$8$ people including $A,B,C$, and $D$ will be rearranged. In how many ways can they be rearranged such that $B$ and $C$ will be between $A$ and $D$?

Question:
$8$ people including $A,B,C$, and $D$ will be rearranged. In how many ways can they be rearranged such that $B$ and $C$ will be between $A$ and $D$?

I tried to break this into cases. However, it didn't work.
Regards
Answer: First choose 4 places, that you can do on ${8\choose 4}$ ways, then on edge put $A$ and $D$, you can do that on 2 ways and in the midle $B$ and $C$, again on 2 ways. Then arrange all others on remaining places, that is on 4! ways. Now multiply all these. So the answer is $${8\choose 4}\cdot 2\cdot 2\cdot 4! = 8!/6$$
Comment: @Mr.Maxwell You have to put $B,C$ between $A,D$. This doesn't mean $A,D$ have to sit on the edges. The arrangement can also look like this: _ _ A _ B _ C D
Comment: That seems a bit unclear. Could you please make it more clear?
Comment: Which part of it?
Comment: $${8\choose 4}$$ And what's also the reason you put B and c in the middle?
Comment: What about it? You don't know what it means?
Comment: I actually do know what it means. However, why did you do that? Like you can draw something like $$A \_ \space \_ \space  \_ \space  \_ \space \_ \space  \_ \space  D$$ As a point of my view, you put A and D on edges in $2$ ways.
Comment: To put those 4 people on that 4 places.
Comment: I'm still unable to get it.
Comment: No, you put A,D on the edges of those 4 places you choose
Comment: and then B,C in other two places you choose
Answer: We first choose among the $8$ positions the $4$ ones for $A,B,C,D$. This can be done in $\binom{8}{4}$ ways. These positions cab be filled in $4$ ways: $ABCD$, $ACBD$, $DBCA$, $DCBA$. The remanining $8-4$ positions can be filled in $4!$  ways. Hence the total number of arrangements is
$${8\choose 4}\cdot 4 \cdot 4!$$
Comment: No. With $\binom{8}{4}$ we choose just the positions where we are going to put the letters $A$, $B$, $C$, $D$ (no order is specified). In the complement we will put the other letters.  Then we place in chosen positions the letters $A$, $B$, $C$, $D$ in one of the $4$ orders and the remaining letters in the remaining positions in one of the $4!$ orders.
Comment: Can you help me understand the first part. I'm a bit confused. When you do 8c4 doesn't that include instances such as: _D_CAB__? If so, shouldn't you remove them because BC should be between AD. Thanks
Comment: ohh. Right! Got it
Answer: This answer assumes that ABCD is unique from ACBD, though I can answer for the other case as well if that is the case.
N = number of spots (8 in your case)
2 * summation(n = 1, N - 3){n! * (N+1-n)!}
In your case, this simplifies to
Take the two boundary tokens and orient them on the edges. There are two "symmetric" orientations for each orientation of the boundary tokens (A******D is distinct from D******A, and there are only two orientations of two tokens with two spots.) From there, you have the inner spaces, designated as "Sep". For each value Sep, there are Sep! permutations inside the boundary tokens, and (N - (Sep + 2))! permutations outside the boundary, so for each value Sep, there are Sep! * (N - (Sep + 2))! * 2 * (number of "slide" positions). The "Slide" positions are the orientations you can bring A and D to through sliding; in simple terms, if the string was A****D**, Slide is 3, since you can have A****D**, *A****D*, **A****D. Factor the 2 out and do some refactoring, plug in N = 8, and you get 2976.
EDIT: My final sum of 2976 was a quick desmos plugin, so if anyone would like to plug in the equation I provided above to verify, you can go ahead and do that.


compare two different size array ( component of A is in B )

Question: What I want to do is there are two different size array
Let
A = [[1,3,1],[2,4,1],[3,3,1]]
B = [[1,3,1],[2,5,1],[2,3,1],[3,3,1],[5,5,1]]

Row of B may in A or not( [1,3,1], [3,3,1] is in A )
What I want to do from these arrays is compare two arrays in order to construct
array "C" which is consist of the rows of B that is in A
C = [[1,3,1], [3,3,1]]

I have tried pandas.isin but I have failed.. Any idea would be very helpful
Comment: in short, you want common elements from both lists.. correct??
Comment: If you treat the arrays as sets, this is intersection.
Comment: oh, yes.. that is common elements
Answer: Use set.intersection
Ex:
A = [[1,3,1],[2,4,1],[3,3,1]]
B = [[1,3,1],[2,5,1],[2,3,1],[3,3,1],[5,5,1]]

A = map(tuple, A)
B = map(tuple, B)
print(set(A).intersection(set(B)))

Output:
{(3, 3, 1), (1, 3, 1)}

Answer: You can get common elements from both lists:
>>> A = [[1,3,1],[2,4,1],[3,3,1]]
>>> B = [[1,3,1],[2,5,1],[2,3,1],[3,3,1],[5,5,1]]
>>> print([data for data in A if data in B])
[[1, 3, 1], [3, 3, 1]]

Comment: You can remove the `if data in A`, since it is already `for data in A`.
Comment: Thank you so much. I got hind from eariler comments from others, and this is it!
Comment: @Keldorn : Yeah..true that.. no need to add `if data in A` as I am looping on `A` only.. Modified answer..
Answer: Or with filter:
print(list(filter(lambda x: x in A.tolist(), B.tolist())))

Output:
[[1, 3, 1], [3, 3, 1]]


Ctools include ajax

Question: What is the best way of include ctools_include('ajax'); into a page already defined by another module?
I'm trying to use ctools ajax framework to replace content (similar to how the ctools example hello world works).
I've tried to integrate the ctools hello world example into my existing code but it looks like none of the setup is run. When clicking on the the link it forwards to the menu link defined rather than changing the content.
EDIT:
I've used ctools_include('ajax') on the following hooks (all hooks confirmed correct by added debug to them):

nodeapi (alter)
preprocess node

This is my hook_nodeapi('alter') code, which inserts the text as per the example module.
ctools_include('ajax');

$links =array();
$links[] = ctools_ajax_text_button(t('Hello world!'), "defs/nojs/test", t('Replace text with "hello world"'));
$node->body .= theme('item_list', $links, t('Actions'));
$node->body .= theme('ctools_ajax_sample_container', '
' . t('Sample Content') . '
');

This is my menu item.
 $items['defs/%ctools_js/test'] = array(
    'page callback' => 'br_defintions_funkyajax',
    'access arguments' => array('access content'),
    'page arguments' => array(1),
    'access callback' => TRUE,
    'type' => MENU_CALLBACK,
  );

This is my callback function from the menu item.
function br_defintions_funkyajax($js = NULL) {
  $output = '
' . t('Hello World') . '
';
  if ($js) {
    ctools_include('ajax');
    $commands = array();
    $commands[] = ctools_ajax_command_html('#ctools-sample', $output);
    ctools_ajax_render($commands); // this function exits.
  }
  else {
    return $output;
  }
}

Answer: Can't you just do
  ctools_include('ajax');

At the start of your existing code.
Ctools should put classes in your code which are fairly easy to spot.
Edit
Thanks for the code. I'm not sure what it is it could be one of a couple of things, but may be neither.
you are using $op 'alter' in hook_nodeapi, this is quite a late stage hook, and it is possible that add_js doesn't work then. 'view' may be a better $op for you to use.
Also you are using theme 'ctools_ajax_sample_container' this is part of the ctools_sample module. Do you have that module installed.
It is also possible that you have a js error somewhere else which is knocking out the ctools js functionality. Check any errors you have with firebug.
Comment: Yeah I should but for some reason done of the classes are appearing e.g. the classes appended to body.
Comment: Do you know if your code is actually running. Can you output any debug? Posting a code sample may help.
Comment: Updated the original post with code used.
Comment: I added it into a the view op and no change. No javascript errors. I have the sample module enabled and can hit the page and confirm that it works correctly. It's really odd that I can't get my implementation to work.
Comment: In firebug you can see if ctools has processed the link it will have a class ctools-use-ajax-processed what classes are on your link?
Comment: the only class is "ctools-use-ajax" and if I look in  I can't see any references to ctools ajax-responder.js

Ctools include ajax


Correct statistical test to answer: "did my intervention help my patients?"

Question: I am doing something to a set of $N$ patients (keeping this vague for generality).
In order to assess if my intervention/treatment is making any difference I did the following:
Before my intervention I measured something about the patients (call that measurement $X_i$ for patient $i$), and repeated the same measurement after my intervention ($Y_i$). So I have $N$ pairs $(X_i, Y_i)$
Let's say my measurements/test spits out a number from 0 to 5 for each patient. Assume I only know high numbers are good and low numbers are bad. (e.g. the difference between a 5 and a 4, might not be necessarily the same as the difference between the 2 and a 1). (I think this is what is called an ordinal measurement). Furthermore assume that when looking at the results of my test for the whole set of $N$ patients we observe that they don't look normally distributed.. (e.g. their most common result could be a 1 while the least common could be a 5)
What is the correct statistical test to see if we could say that my intervention is improving my patients score on those measurements?
I'm also interested in understanding how slight changes to my problem would affect the choice of a statistical test
Does this depend on $N$?
What if actually my measurements are a continuous variable that actually looks normally distributed?
I'm also curious to know if I had first divided the group into 2 groups and only did my intervention to one of the groups, (while for the other group I also measured twice but without any intervention), if this would have been better somehow, and in this case if the test would be different
Thank you
Comment: There isn't "the" best given only the information in your question.  There may be some reasonable choices as long as the way they measure 'larger' fits with the sense of larger you had in mind.
Comment: Sorry, not sure how I missed responding before. "I would assume any test able to deal with ordinal data will not depend on the way my test measures high/low" -- this is not the case, since different models, fits and tests will still measure different aspects of the comparison.
Comment: You will need to determine how "improving" would be reflected in the measurements.  Is "improving" reflected in the percentage of 5 values reported?  Is it reflected in the percentage of 4 and 5 values reported?  Is it "the percentages of 1 and 2 values didn't increase, but the percentage of 3, 4, and 5 values did?"   Or is it something else?
Comment: Hi @Glen_b, thanks for taking time to comment. I have read several of your excellent answers! But this time I didn't fully understand your comment, as I would assume any test able to deal with ordinal data will not depend on the way my test measures high/low. Also, could you suggest some of those tests that you mention as reasonable? Thank you again
Comment: @Glen_b any help is much appreciated!
Comment: no worries, thanks for the response. Though I still don't understand what to do or what test to run. if it helps, these are psycological tests whose output is a number from 1-5. I would like to see if an intervention is on average improving (and by improving I mean increasing) this value
Comment: hmm I thought defining improvement as: "a subject improves if their measurement increases, (and doesn't improve if otherwise)" would be sufficient to choose a test? (my intuition is that if around half the participants increased whereas half decreased then one couldnt say my intervention is causing anything. Whereas if most people increased then I would hope a statistical test would either flag it as significant or non-significant
Comment: in other words, improving is reflected in the measurements if the values increase
Comment: How do you want to treat "no change"?    One alternative is "more improved than got worse" which, in effect, throws out everyone who saw no change.   Thinking through the statistic you want to use is key.
Answer: Fix your experimental design first; then use an ordinal regression model
Firstly, before you get to the inference phase, you need to alter your experimental design.  Since you are trying to infer the causal effect of a medical treatment, you should use randomised control and treatment groups (e.g., using block randomisation) to allow you to infer the causal effect from a comparison of these groups.  Without a control group, you have no way of knowing if a statsitical change in the outcome occurred because of your treatment or because of some contemporaneous change in the patient cohort from another source.  Depending on the natue of the medical treatment, you should also consider a "blinding" protocol to prevent the patients from knowing which group they are in, and possibly even a "double blinding" protocol to also prevent the treating doctor from knowing which group each patient is in.  Randomisation and blinding are protocols used to sever the statistical connection between the treatment variable and any possible confounding variables in the analysis.
Once you have fixed your experimental design, you will then have data that can be used to infer a causal effect.  It sounds like you are using a five-point ordinal scale for your outcome variable, so you would model this using some kind of ordinal regression model.  Typically, you would form a regression model that includes the treatment variable, relevant demographic variables, and any other useful variables relating to the medical state of the patients.  (Make sure to exclude any mediator variables --- i.e., variables that are intermediate medical outcomes from the treatment to a change in the outcome.)  You would then use this model to infer the effect of the treatment variable on the ordinal outcome variable when other (non-mediating) variables are held constant.  In answer to your question, yes, the strength of the inference will depend on the number of data points in your analysis --- roughly speaking, the more patients you have in your sample the more accurate your inferences about the effect of the treatment.
If you will permit me to speculate a little, it appears from your question that you are probably a novice in the field of medical experimental design.  In almost all institutions that perform medical research (universities, hospitals, etc.) there are consulting statisticians available to give assistance on experimental design, modelling and inference.  Even if you are outside these institutions (e.g., if you are a GP in private practice) you will probably find that consulting statisticians in these institutions will be happy to help you with some basic advice.  I strongly recommend going to talk to a consulting statistician working in the medical/population health sector for assistance with planning and executing your medical experiment.  Medical experimentation generally also requires ethical clearance from relevant bodies and so you should ensure you have gone through an appropriate ethics process for your study prior to executing it.
Comment: This sounds like very good advice in general. and I thank you for your answer. My case however is different in the sense that I had no voice in the experimental design phase and I was only later asked to help with the analysis. I will surely pass all your comments to the person that is requesting my help.
Answer: This question is conflating two distinct issues, which both are at the core of statistics: 1) Is this difference just due to sampling error and 2) what does this difference mean?
You seem to be mostly focused on the first one, but the second one is the important one if your real question is "how can I help patients?"
In general, a statistical test is a procedure that is designed to tell you how confident you can be that a difference (e.g. the difference between two groups on some outcome, or the difference between a measurement before and after a treatment)
observed within a random *sample actually reflects an underlying difference in the population from which the sample is drawn, as opposed to being just due to "random chance." That is, a statistical test is a way to guard against sampling error that occurs when you try to use random samples to generalize about populations. And the answer almost always depends on "N" the number of people in your sample. The more people in your sample the less likely it is that a given difference you see is just due to sampling error. In your case a paired sample t test - which tests whether the averages of two measurements from the same sample are different or not, is probably fine. Even though the underlying measurement is ordinal, the average of those measurements is continuous.
However, none of what I just said tells you - by itself - whether the treatment actually helped patients. Whether a treatment works or not is a question of causal inference. And while statistical tests can tell you if a difference is real or not, they can't (by themselves) tell you anything about why that difference existed. To answer that question you have to think about research design. In your case, you are describing what's called a pre-post, no-control group design. You did a measurement of a single group, gave them a treatment, and then measured them again. The problem with this approach is that, even if the group got "significantly" better when you measured them the section time (according to whatever test you ran), that doesn't mean the treatment helped. It might be that they just got better on their own - and would have done just as well if you didn't give them the treatment! No statistical test can solve this problem. You need a different research design.
And indeed what you proposed at the end of your answer is one way to (maybe) solve this problem: "First divided the group into 2 groups and only did my intervention to one of the groups, (while for the other group I also measured twice but without any intervention)." This is what is called a difference-in-differences design. The assumption here (which might be wrong!) is that the chance you observe among the group that didn't get the treatment is what would have happened to the treatment group if they didn't get the treatment. So to estimate the effect of the treatment, you compare the change observed among the control group to the change observed among the treatment groups, the "difference between those differences" is the estimated effect of the treatment. Of course you still have to test if that difference is big enough that it's unlikely to be due to sampling error. In practice people often analyze "Diff-in-Diff" designs with regression models and interaction terms (which you should read up on, since these are critical to the question of causal inference), but the actual "test" that is used at the end of the process is often just a different form of the "t test" used above.
That probably wasn't a very satisfying answer, but your question really gets at the two most fundamental challenges in statistics. Picking the right statistical test for the job is usually not the hardest part of research. It often depends on the measurement level of the variable and a few other things, but you can always figure it out with some googling, and often it doesn't even matter so much if you choose the wrong one. But causal inference is the core of using statistics to actually figure stuff out in the world - and doing it well or poorly is what separates good research from bad.
Comment: That was a clear explanation and it makes sense. However if I only want to focus on the first issue, (of whether this difference is due to sampling bias or not), I am still unsure which statistical test to perform. you mention the paired t-test which in my understanding is the standard test for this case, Having googled for my situation I did find a set of other tests that, on the surface, seemed more adequate for my case, but overall I'm still confused. (Thank you for the time writing this answer)
Comment: Wilcoxon signed-rank test or the paired sign test looked like more proper tests


How to style summary row in Element-UI Table?

Question: I'm making the numbers in red or blue in table cells according to the value using :cell-class-name. It works fine in the table, body but I also want to have the same effect in the summary row. Is there any way to do that?
 <template>
  <el-table
     :data="products"
       style="width: 90%"
       show-summary
     :header-cell-style="{
       'background-color': '#f7f7f7',
       color: '#6e6e6e',
       'font-weight': '400',
       'text-align': 'end',
     }"
     :cell-class-name="classChecker" >
  <el-table-column width="50" prop="profit" align="center"> </el-table-column>
  <el-table-column width="50" prop="cost" align="center"> </el-table-column>
  <el-table-column width="50" prop="price" align="center"> </el-table-column>
 </template>

  methods: {
    classChecker({ row, column }) {
      const val = row[column.property];
     if (val > 0) {
      return "blueClass";
     } else {
       return "redClass";
     }
  }
}

Answer: I had the same requirement a while back, my understanding is that they don't have this feature, so you'll have to implement it yourself.
Here's how I did it:
codepen: https://codepen.io/rugia/pen/ZEajGqg
basically what it does is to add color class on target columns' sum cell every time tabledata updates.
methods: {
            async applyFooterColor () {
                await this.$nextTick()

        const table = this.$refs.table
        const cols = ['amount1', 'amount2']

                table.columns.forEach(col => {
                    const cells = document.getElementsByClassName(col.id)
                    const cell = cells[cells.length - 1].children[0]
                    if (cols.includes(col.property)) {
                        const val = +cell.textContent
                        if (+val !== 0) {
                            const color = val > 0 ? 'blue' : 'red'
                            cell.classList.add(color)
                            return
                        }
                    }
                    cell.classList.remove('red', 'blue')
                })

            }
    },
        watch: {
            tableData: {
        immediate: true,
        handler: function() {
          this.applyFooterColor()
        }
      }
        }


Julialang package REQUIRE specify remote public Git repository

Question: Suppose there is a package with the following REQUIRE file:
julia 0.4
PackageFromGithub

AFAIU from the julia package docs Making Your Package Available one could not specify that PackageFromGithub should be also cloned from Github. Either one have to clone it manually or make the PackageFromGithub available via registering it in METADATA.
Is there a way to specify something like:
PackageFromGithub https://github.com/user-x/PackageFromGithub

?
Answer: Originally I was looking for way to make my build passing on Travis. This is possible to do by adding something like this into the build script .travis.yml:
script:
  - if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
  - julia -e 'Pkg.clone("https://github.com/user-x/PackageFromGithub.jl.git"); Pkg.clone(pwd()); Pkg.build("YOUR_PACKAGE"); Pkg.test("YOUR_PACKAGE"; coverage=true)'

In case somebody could be interested.


hide the content when the mouse out

Question: html

Home

...

...

...

when the mouse hove on the nav-tabs li. the corresponding div content in tab-content show. when leave out the nav-tabs li. the corresponding div content show.
jquery:
$( ".nav-tabs li" ).hover(
  function() {
     $( "#tab-content div" ).css( "display", "block" );
  }, function() {
     $( "#tab-content div" ).css( "display", "none" );
  }
);

there is something wrong with the code. when the the corresponding div in tab-content  part show. i can't move the mouse on its child a link.
2, when i hove on another nav-tabs li. the other li content also shows.
 how to correct it? thank you.
Comment: Please cleanup your code
Answer: Try
$(".nav-tabs li").hover(function () {
    $("#tab-content div." + this.className.match(/(tab\d+)/)[0]).show();
}, function () {
    $("#tab-content div." + this.className.match(/(tab\d+)/)[0]).hide();
});

Demo: Fiddle
Comment: thank u. all of are ok.but when the mouse leave out ".nav-tabs li". the #tab-content div... not hide.
Comment: in the demo example. first,tab2 tab3 content are hidden. when i leave out the li. tab2 content are hidden.thank u.
Comment: @stack2013a sorry can you explain it again

hide the content when the mouse out


Let S be the set of all 8 -bit strings (a sequence of 1s and 0s)...

Question:

So I understand that (i) in that it would be f(10000001)=0 as there are two 1's. However I dont understand the reasoning I should put forward for the (ii) and (iii).
For (ii) my thought is that there is only 0 and 1 which can be the output then it would be an onto function because for all Y(0 and 1) there does exist an X which is the set of binary numbers possible.
Comment: An onto function is one that takes on all elements of the codomain as values. What's the codomain here? Is every element of the codomain a value of $f$?
Answer: You were off to a good start with (ii) in recognizing that all of the possible outputs were $0$ and $1$.  However, you need to also notice that the codomain of the function is $\mathbb Z$, the set of all integers.  If the function were given as $f:S\to\{0,1\}$, then it would be onto.  However, since there are many many integers that are not possible outputs of $f$, the function that was given is not onto.
To determine if the function is one-to-one, you need to investigate whether it is possible for two different inputs to the function to have the same output.  That's definitely the case for this function, so $f$ is not one-to-one.  To demonstrate this formally, all you need to do is to show two different inputs that have the same output and you're done.


Is it okay to do ANOVA on group that is compared with itself and the rest?

Question: I have 4 clusters that is obtained from clustering i.e. Group A, Group B, Group C, and Group D. Now, the comparison that other groups had done was Group A vs. Rest, Group B vs Rest, Group C vs Rest, Group D vs Rest. They're asking me to do ANOVA to find any differences between the groups. According to them, the rows in the groups were oversampled i.e. some rows in Group A could also be in Group B or Group C or in both, for example. Is it even okay to do this? Then is it also okay to perform post-hoc comparison? Could you please help me understand this?
EDIT
The shape of the Dataframe with ties is listed below. The data is joined on 'ID':
Merged data (Inner-Join)
=========================

Comparison              Shape
1. Group A vs Group B:  (15, 107)
2. Group A vs Group C:  (0, 107)
3. Group A vs Group D:  (0, 107)
4. Group B vs Group C:  (15, 107)
5. Group B vs Group D:  (0, 107)
6. Group C vs Group D:  (16, 107)

Actual total rows: 211
Total rows after oversampling: 270

Comment: I don't quite follow the process - how do observations fit in multiple groups when you clustered? It is almost certainly not appropriate to have an algorithm cluster data, and then ask if groups are then different as this is extremely cyclic. If there are shared observations (i.e. the $ith$ observation is in both group A and B) then you definitely cannot make any meaningful comparisons... I am not even sure how you end up with that.
Comment: When you say "*Now, the comparison that other groups had done was Group A vs. Rest, Group B vs Rest, Group C vs Rest, Group D vs Rest. They're asking me to do ANOVA to find any differences between the groups.*" Does the first "group" refer to another group of people? If they have already done comparisons why are they asking you to do more? Do they mean pairwise comparisons?
Answer: Data for illustration: Here are some fake data (simulated in R) to use for illustration: four groups, each with 30
observations, common population standard deviation $\sigma = 3,$ and various
population means. (There are several ANOVA procedures in R. In some of them, it is essential to make gp a factor variable.)
set.seed(1234); n = 30; sg = 3
a = rnorm(n, 20, sg)
b = rnorm(n, 20, sg)
c = rnorm(n, 25, sg)
d = rnorm(n, 35, sg)
x = c(a,b,c,d);  gp = as.factor(rep(1:4, each=n))

In 'notched' boxlots, the notches in the sides of the boxes are calibrated
for making nonparametric comparisons between two groups at a time. If notches overlap between any two groups, then those two groups may not be significantly different.
boxplot(x ~ gp, col="skyblue2", pch=20, notch=T)

Tentatively, it seems that groups 1 and 2 may have the same population means, and that
groups 3 an 4 may have different means, both greater than in groups 1 and 2.
ANOVA Table and main F-test: The one-way ANOVA table is made as follows:
aov.out = aov(x ~ gp)
summary(aov.out)
             Df Sum Sq Mean Sq F value Pr(>F)
gp            3   5554  1851.4   245.8 <2e-16 ***
Residuals   116    874     7.5
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

With such a small P-value, there is strong evidence that not all four group
population means are equal.
Multiple comparisons: There are many methods of adjusting P-values for pairwise comparisons of
group sample means in order to avoid false rejections. Below I show the
results from the Bonferroni method. You can look online for several other
methods.
pairwise.t.test(x, gp, p.adj="bon")

        Pairwise comparisons using t tests with pooled SD

data:  x and gp

  1       2       3
2 1       -       -
3 5.4e-14 < 2e-16 -
4 < 2e-16 < 2e-16 < 2e-16

P value adjustment method: bonferroni
Group 1 does differ significantly from group 2 (P-value 1), but does
differ significantly from the other groups (P-values near 0).
Group 2 differs significantly from both of the other groups, which in
turn differ significantly from each other.
Potential design errors in your data: From a vague description of 'oversampling' that 'may have' taken place, it is
difficult to know what effect such design errors may have on multiple comparisons.
If a few Group A values were repeated in other groups, that
would tend to make the groups look more alike than they really might be.
So you may be safe in believing the significant differences you find in
your data. However, if such repetition is extremely widespread it may make
the variance estimate too small which might lead to unwarranted deflation
of P-values.
If values are expressed to several decimal places, you might
see what happens if you look for ties between the groups. Presumably, there will be few naturally occurring ties, and presumably repeated values would show as ties.
In my fake data, there is no such 'oversampling'. If I round my 120 observations to three place accuracy, there are no ties. (If I round to two
places, there are four ties among the 120 rounded values.)
x1 = round(x, 3)
length(unique(x1))
[1] 120

If you have few tied data (just speculating, say fewer than 5%), you might consider removing all
tied observations from your data and seeing if the that makes any
difference in what is declared significant.
Comment: If you have IDs, then you can forget about ties as a way to correct the data. If the same participant IDs show up in multiple groups, you need to decide which group each belongs to. If that is not clear, and there are only a few, maybe you can remove them altogether. My fake dataset is 'balanced' with the same number of subjects in each group. But it is OK if some groups have more subjects than others. // There are several procedures in R for doing one-way (one factor) ANOVA, and several reasonably clear demos online for each.
Comment: Also there are several different multiple-comparison methods. (I used Bonferroni.) Some of them work better with a balanced design.
Comment: +5: thank you so much for a great explanation. I have edited the question above to reflect the IDs that are shared between groups. My difficulty with understanding the concept is with the way it has been done. I had been doing ANOVA with `Group 1` vs `Group 2` but the term `oversampling` confused me. I'm not entirely sure how they did it, but the idea is that IDs in `one group may also exist in another group based on some degree of similarity`. Vague, but lets go with it. Then, is it okay to perform ANOVA (or equivalent)? You mentioned about removing ties. In our case, should I?


Configure MSSQL 2008 to use kerberos authentication

Question: I would like to set my current mssql db (2008 and 2012) to authenticate using a linux kerberos server that I have in my environment .
The windows station is in a workgroup and I would like to avoid having to setup a domain controller and active directory .
How can I accomplish that ? what do I need to configure on the windows side and the kerberos server side (assuming it's configured already to work with other dbs running on linux based OSes) ?
Answer: Yes this is manageable. All you need is a trusted connection to an odbc connector and you can read about that here: http://technet.microsoft.com/en-us/library/hh568450.aspx
Comment: Thanks @robwahl for the answer. I need to configure it for jdbc which I did , but I'm getting some error message
'Server not found in Kerberos database
Application/Function: Anything that makes a service ticket request.
Potential Causes and Solution: The account for the service principal name being requested doesn't exist in Active Directory or is incorrect in Active Directory. This could also indicate a DNS problem.'
any hints?
Answer: I suspect that you're going to have to "trick" windows into natively authenticating against the linux kerberos server, which might be more effort than you're looking for.  Things I found include:

This O'Reilly article
PGina is old and doesn't include Kerberos out of the box, but you
could write a plugin (yikes).
Use SAMBA and FreeIPA to create a trust with your linux kerberos server

Or, you could use SQL Server authentication instead.


What is the direct limit of $\mathbb Z$ with $f_{ij}(x)=x \cdot j$?

Question: I'm just learning the concept of direct limit. I did problem $8$a fromt Dummit Foote section $7.6$.
As an example, someone suggested I look
$$\mathbb{Z} \xrightarrow{\times 2} \mathbb{Z} \xrightarrow{\times 3} \mathbb{Z}\xrightarrow{\times 4} \mathbb{Z}\xrightarrow{\times 5} \mathbb{Z}\xrightarrow{\times 6} \cdots$$
and I should get the rationals.
I've tried playing around a bit, seeing which elements are equivalent in the quotient of the disjoint union modulo the relation, but not sure how to get the rationals.
Any help on how to get the rationals out of this direct limit?
Answer: Here is a directed system of abelian groups which is isomorphic to yours:
$$
\mathbb{Z} \to \frac{1}{2}\mathbb{Z} \to  \frac{1}{6}\mathbb{Z} \to \ldots \to \frac{1}{n!} \mathbb{Z} \to \ldots
$$
(here we mean the cyclic subgroup of $\mathbb{Q}$ generated by the element $1/n!$). Here, the arrows are just inclusion. To prove this is isomorphic to your sequence, write your sequence under it and the necessary vertical maps to make the diagram commute (which will be multiplication by $n!$).
Now this directed system has $\mathbb{Q}$ as a limit. (The inclusion into $\mathbb{Q}$ commutes with all the maps, and is the initial thing commuting with all the maps since any element of $\mathbb{Q}$ is eventually in $\frac{1}{n!}\mathbb{Z}$ for some $n!$).
Comment: Could you please elaborate on the last paragraph? Not sure what you mean with the sentence in the parenthesis.
Comment: Also, if we have a disjoint union of abelian groups, then I know this is the direct sum of them. So, how do we go from direct sum of the the abelian groups you have listed to modding out by the equivalence relation to get $\mathbb Q$?


Question Regarding Zero Conditional Mean

Question: Hi I am a beginner to econometrics!
I have been dealing with bivariate regression. We use the formula $y = \beta_0 + \beta_1 x$.
I am told that if $E(u\mid x) \ne 0$ then the estimate of the slope parameter $\beta_1$ will be biased.
My question is: if $E(u\mid x) = 1$ for all $x$'s, then why would the estimate of the slope parameter be biased? I thought this would mean that only the intercept estimate will be biased.
I thought that the slope parameter will only be biased if the expected value of the errors is different for each $x$. For example if $E(u\mid x_1) = 1$ and the $E(u\mid x_2) = 3$, etc. etc.
Any help will be immensely appreciated! Thank you!
Answer: You are correct that the condition for unbiasedness of the slope coefficient is that
$$E(u \mid X) = const.$$
However, if it is different than zero, we will have bias in the estimation of the constant term.
If you want to go deeper, this thread may be useful, Conditional mean independence implies unbiasedness and consistency of the OLS estimator
Answer: This is my attempt. I think you are confusing $X_s$ to observations $n$. There is no reason for $\beta_1$ to be biased if for all observation the $E(u|X)=1$, as that would mean you simply subtract the constant term, 1 in this case from the error term and the constant, and your equation becomes $y=\beta_0 -1 +\beta_1x + u-1$. So, your $\beta_0$ is biased but not $\beta_1$.
Answer: Let's see what's going on:
$$\begin{align}E[\hat{\beta_1}|x] &=E\left[\frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{S_{xx}}\bigg|\ x\right]=\frac{1}{S_{xx}}\sum E[(x_i-\bar{x})(\beta_1(x_i-\bar{x})+(u_i-\bar{u}))]\\ &=\frac{1}{S_{xx}}\beta_1 S_{xx}+\sum(x_i-\bar{x})E[u_i|x]-E[\bar{u}|x]\sum(x_i-\bar{x}) \\ &= \beta_1+\sum(x_i-\bar{x})E[u_i|x]\end{align}$$
The second term won't go off when $E[u_i|x]\neq c$ (thanks to the correction by @Laconic), and introduce a bias to the expected value. So, it seems you're right.
Comment: Won't the second term be zero if E[u|x] is constant?


How do I create a paragraph using a bundle class defined in another module?

Question: I have a controller in one module that generates paragraphs that are using unique bundle classes that I want to be able to use in other modules.
In my controller I have this
<code>$bundle_class = \Drupal::service('class_resolver')->getInstanceFromDefinition('Drupal\other_module\Entity\Paragraph\StandardHero');
$paragraph = $bundle_class::create([
  'type' => 'standard_hero',
  'field_title' => [
    'value' => 'Test'
  ]
]);
</code>
It does recognize the class because if I change the class sting to gibberish it complains that that class is not found. But I get this error

ArgumentCountError: Too few arguments to function
Drupal\Core\Entity\ContentEntityBase::__construct(), 0 passed in
/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php on line 31
and at least 2 expected in
Drupal\Core\Entity\ContentEntityBase->__construct() (line 189 of
core/lib/Drupal/Core/Entity/ContentEntityBase.php).

How can I create a paragraph of a specified bundle class that appears in a different module?
Comment: create() is a static method to create an instance, you don't need the class resolver to get an instance. BTW it is not important to identify the right class, the entity storage will do this automatically if the module providing the class is implementing the bundle class correctly.
Answer: 4uk4 was correct. Defining the class was unnecessary. I was able to generate the paragraph with the proper bundle class applied automatically by just defining 'type' in the create method of the paragraphstorage class, like so
<code>  public function __construct(EntityTypeManagerInterface $entity_type_manager) {
    // Injected.
    $this->entityTypeManager = $entity_type_manager;
    $this->paragraphStorage = $this->entityTypeManager->getStorage('paragraph');
    $this->paragraphViewBuilder = $this->entityTypeManager->getViewBuilder('paragraph');
  }

  public function show($type) {
    $query_params = \Drupal::request()->query->all();
    $paragraph_values = [
      'type' => $type
    ];

    $paragraph_values += $query_params;
    $paragraph = $this->paragraphStorage->create($paragraph_values);
    $build = $this->paragraphViewBuilder->view($paragraph);
    return $build;
  }
</code>


Problem with defintion of an equivalence relation induced by an arbitrary class

Question: I think I have a problem with a definition of an equivalence relation induced by an arbitrary class -  I am not sure where I picked the definition up from, indeed I may even have made it up and convinced myself I had seen it somewhere;
Definition (wrong?)
For an arbitrary class $\mathcal{A}$ of subsets of $\Omega$ let $\omega'\sim_{\mathcal{A}}\omega''$ denote that $\omega'$ is equivalent to $\omega''$ (mod $\mathcal{A}$);
$\omega'\equiv\omega''(\text{mod }\mathcal{A})\iff 1_{A}(\omega')=1_{A}(\omega''),\forall A\in\mathcal{A}$
Then $\sim_{\mathcal{A}}$ is an equivalence relation on $\Omega$ induced by the class $\mathcal{A}$, where $[w]_{\mathcal{A}}$ denotes the equivalence class represented by $\omega$, that is $\omega'\sim_{\mathcal{A}}\omega$ for all $\omega\in [w]_{\mathcal{A}}$. Let the class of all equivalence classes (the quotient set) be denoted $\Omega\backslash \sim_{\mathcal{A}}:=\{[w]_{\mathcal{A}}:\omega\in\Omega\}$. Since $\sim_{\mathcal{A}}$ is an equivalence relation then it partitions $\Omega$, that is $\Omega=\dot{\cup}_{[w]_{\mathcal{A}}\in \{\Omega\backslash \sim_{\mathcal{A}}\}}[w]_{\mathcal{A}}$ where the dot denotes a disjoint union.
I mention here my understanding about duplicate equivalence classes given the statement about partitions (and partitions become my issue a little later): Although $\omega\sim_{\mathcal{A}}\omega'$ implies $[w]_{\mathcal{A}}=[w']_{\mathcal{A}}$, the quotient set $\Omega\backslash \sim_{\mathcal{A}}$ is, from a set theoretic perspective, still always a disjoint collection of equivalence classes since $\Omega\backslash \sim_{\mathcal{A}}$ is just a set (where the elements are sets), and sets do not "recognise" duplication (i.e the set $B=\{a,a,b\}$ satisfies the axiom [?] that $a\in B \Longleftrightarrow \exists a\in B$ is true, even though there is duplication of the element $a$). My understanding may be off here, although I do not see the quotient set being explicitly defined to contain only the unique equivalence classes so I figure the definition of a set does the work behind the scenes?.
So what is the problem with the above definition of $\sim_{\mathcal{A}}$? Suppose $\Omega=\{\omega_{1},\omega_{2},\omega_{3},\omega_{4}\}$ and $\mathcal{A}=\{A_{1},A_{2},A_{3}\}$ where $A_{1}=\{\omega_{1},\omega_{2}\}$, $A_{2}=\{\omega_{1},\omega_{2},\omega_{4}\}$ and
$A_{3}=\{\omega_{2}\}$. Then using the above definition $[w_{1}]_{\mathcal{A}}=\{\omega_{1}\}$, $[w_{2}]_{\mathcal{A}}=\{\omega_{2}\}$, $[w_{3}]_{\mathcal{A}}=\emptyset$ and $[w_{4}]_{\mathcal{A}}=\{\omega_{4}\}$ and so $\Omega\backslash \sim_{\mathcal{A}}=\{\emptyset,\{\omega_{1}\},\{\omega_{2}\},\{\omega_{4}\}\}$. The problem here is that $\omega_{4}\not\in\Omega\backslash \sim_{\mathcal{A}}$ so $\Omega\backslash \sim_{\mathcal{A}}$ does not partition $\Omega$. I have seen another definition of the relation $\sim_{\mathcal{A}}$ that states (again I cannot recall where);
Definition (correct?)
$[\omega]_{\mathcal{A}}:=\bigcap\{A:\omega\in A\in\mathcal{A}\}\cap\bigcap\{\Omega\backslash A:\omega\not\in A\in\mathcal{A}\}$
If I apply this definition I get $[w_{3}]_{\mathcal{A}}=A_{1}^{c}\cap A_{2}^{c}\cap A_{3}^{c}=\{\omega_{3},\omega_{4}\}\cap \{\omega_{3}\}\cap\{\omega_{1},\omega_{3},\omega_{4}\}=\{\omega_{3}\}$. The other equivalence classes remain the same as before and so $\Omega\backslash \sim_{\mathcal{A}}=\{\{\omega_{1}\},\{\omega_{2}\},\{\omega_{3}\},\{\omega_{4}\}\}$ which is a partition of $\Omega$ (the finest partition).
My question therefore is what is missing form the first definition, since the second must be correct given it produces a partition as claimed by the theory? Clearly $\omega_{3}\sim_{\mathcal{A}}\omega_{3}$ fails in the first definition.
Answer: I assume that in your example $[w_k]_{\mathcal{A}}$ is supposed to be $[\omega_k]_{\mathcal{A}}$, as otherwise the objects $w_k$ are undefined. It is not true that $[\omega_3]_{\mathcal{A}}=\varnothing$: the relation that you defined at the top is reflexive, so $[\omega_3]_{\mathcal{A}}=\{\omega_3\}$. The fact that $1_A(\omega_3)=0$ for each $A\in \mathcal{A}$ simply says that $\omega_3$ is equivalent to those members of $\Omega$ that belong to none of the sets in $\mathcal{A}$. If we add a fifth element, $\omega_5$, to $\Omega$ without changing $\mathcal{A}$, then
$$[\omega_3]_{\mathcal{A}}=[\omega_5]_{\mathcal{A}}=\{\omega_3,\omega_5\}\,.$$
Comment: @dandar: I took the original definition to apply to all $\omega\in\Omega$: in this context that's how it would normally be understood, and you ensured that interpretation by saying that $\sim_{\mathcal{A}}$ is an equivalence relation on $\Omega$. Yes, the notation $[\omega]_{\mathcal{A}}$ is fine.
Comment: Thanks! I forgot to apply to all points of $\Omega$ rather than only those contained in $\mathcal{A}$-sets. Should the definition be amended: $\forall\omega',\omega''\in\Omega:\omega'\equiv\omega''(\text{mod }\mathcal{A})\iff 1_{A}(\omega')=1_{A}(\omega''),\forall A\in\mathcal{A}$. Then $\omega'=\omega''=\omega_{3}$ gives your result: $1_{A}(\omega_{3})=1_{A}(\omega_{3})=0,\forall A\in\mathcal{A}$, whilst for all $\omega_{k}\in\omega$, $\omega_{k}\not=\omega_{3}$ we have $0=1_{B}(\omega_{3})\not=1_{B}(\omega_{k})=1$ for some $B\in\mathcal{A}$ giving $[\omega_{3}]_{\mathcal{A}}=\{\omega_{3}\}$
Comment: My notation $[\omega]_{\mathcal{A}}$ is supposed to be pertain to the general case when $\Omega$ is left unspecified - i.e $[\omega]_{\mathcal{A}}$ is the equivalence class for any $\omega\in\Omega$. Presumably this is OK?
Comment: Thanks again Brian for your help
Comment: @dandar: You're very welcome.


Sending attachment via Web Service using java script

Question: I want to attach PDF file to Domino/Notes document using Web service. Currently i have done saving the Notes document by web service using java script. The problem is the sending PDF file to domino server.
How to achieve this?
Comment: please have time to accept answers of your previously asked questions. This way people will be encouraged to help you.
Answer: You need to send the attachment base64 encoded, as provided for by the XSD_BASE64BINARY class that is defined in the lsxsd.lss file. See Practical Web services in IBM Lotus Domino 7: Writing complex Web services for an example on how to deal with attachments.


Chi-square test on regression model

Question: I have a regression model to build, with data having few categorical variables and my target variable is continuous. I wish to implement chi-square test for feature selection of categorical variables in my model. So converting target variable into classes and then implementing chi-sqr upon it, is a correct approach or there is any other better technique.
Comment: Selecting predictors on the basis of their statistical significance in bivariate comparison is not a good approach to feature selection. Search around this website for feature selection and you'll find better recommendations.
Comment: Why would you need to convert transform you dependent variable into a categorical one?
Comment: Chi square should be performed upon 2 categorical variables, correct me if I'm wrong. New to this field.
Comment: Is chi-square a resctriction, could you use another test?
Comment: yes you can do it after binning the continious target variable
Comment: Binning is a bad idea that involves throwing away useful information.
Answer: This is a bad idea. Binning a continuous variable throws away information and increases both type I and type II error. It also imposes a kind of "magical thinking" - that something amazing happens at the breaks.
I'm not sure why you want to use chi-square to do feature selection.
Model building in regression is an extensive topic and has been dealt with here a lot. You will find many good answers here, but, briefly, I'd say it's best to use substantive knowledge and that if you must use an automated method (that is, you want to avoid thinking) then LASSO isn't bad.

Chi-square test on regression model


How to submit my C# code in Code Forces? Getting runtime error

Question: This is my code and it works fine in VS 2022 but when I try to submit it in CodeForces, I get a runtime error.
How I can solve that ????
<code>using System;

class Program
{
    public static void Main(string[] args)
    {
        string? Val1 = Console.ReadLine();
        int Number1 = Convert.ToInt32(Val1);

        string? Val2 = Console.ReadLine();
        int Number2 = Convert.ToInt32(Val2);

        if (Number2 <= (Number1 + 1) / 2)
        {
            Console.WriteLine(Number2 * 2 - 1);
        }
        else
        {
            Console.WriteLine((Number2 - (Number1 + 1) / 2) * 2);
        }
    }
}
</code>
Comment: What is the error, and what are the `args` you are providing?
Comment: When I put my code in CodeForces, it writes this message "running time error" to me and I don't know what's wrong with the code
Answer: <code>     //if the test inputs in one line
    var line = Console.ReadLine();
    var data = line.Split(' ');
    Number1  = int.Parse(data[0]);
    //first integer
    Number2  = int.Parse(data[1]); //second integer
</code>
Comment: It looks like you propose to use only one `Console.ReadLine()`, where both numbers are input separated by a space and then split it. Please clarify how this answers the question.


Debian iptables: Block all Incoming except special IPs or Ports

Question: I play around with the rules for my web server, but it doesn´t work as i want. The ports seems to be reachable, but a connection won´t happen, here´s the script:
#!/bin/bash

# Drop all Incoming
iptables -A INPUT -j DROP

# Allow all Outgoing
iptables -A OUTPUT -d 0.0.0.0/0 -j ACCEPT

# Allow Incoming for special IPs to all Ports
iptables -A INPUT -s 220.127.116.11/32 -j ACCEPT
iptables -A INPUT -s 22.214.171.124/32 -j ACCEPT
iptables -A INPUT -s 18.104.22.168/32 -j ACCEPT

# Allow Incoming from Internet to Ports
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow localhost
iptables -A INPUT -i lo -j ACCEPT

Comment: @Daniel, please take the time to write this out as an answer, don't correct the mistake in the question. Please read the [help].
Comment: Your first rule drops all incoming packets. Even those looping back to the host itself. There is no way anything is ever going to work when your firewall rules won't let even one single packet pass.
Comment: thanks for hinting me to the right position...working 3 days in a row makes blind.^^changing the first rule to iptables --policy INPUT DROP solves it!
Answer: Solution was simple, just replace the first rule
iptables -A INPUT -j DROP

with
iptables --policy INPUT DROP

This doesn't block all, instead sets the policy to default block for Input connections.


Product display and taxonomy term ID filter (with depth)

Question: I have a view for nodes that show my products by product reference field.
All of these products tagged with "Catalog" vocabulary.
I have added a reference to the field_product in the view.
Is there any way to use contextual filter «has taxonomy term ID (with depth)» for this view?
Answer:
Clone the default view called "taxonomy term" and start from there.
Add a relationship to your product, and save.
Add fields as you like.
Answer: At the moment there is no easy/correct way of doing things as far as I know.
You could try to:
use Commerce Taxonomy Reference (sandbox project). Excerpt from the page:

If you want to add the Contextual Filter 'has taxonomy term ID (with
  depth) to a Commerce Product View this module will accomplish that.

This does what you need, but I can't get it to work: there are some versions mismatching etc.
OR
You could move the Taxonomy Reference field away from the Commerce Product to the Product Display node (content type) and use the normal filter (Content: has taxonomy term ID (with depth)).
Answer: Use the Taxonomy Entity Index module
This creates a more complete taxonomy index for all entity types, and provides useful Views filters.
Note: if installing on a site with existing content, you need to rebuild the index from the Configuration page before it'll work.


Count em Intervalos de Tempo do Passado

Question: Bom dia !
Tenho uma view com contratos, e nela há uma coluna para DtInicioVigencia (DATE) e uma coluna DtFimVigencia (DATE). Eu precisava contar o número de contratos ativos para cada um dos treze últimos meses. Tinha pensado na query abaixo, e até deu certo, porém o tempo de execução foi alto (8Min) e para somente um dos treze meses que eu gostaria de puxar:
DECLARE @mm13 DATE
SET @mm13=DATEADD(MONTH,DATEDIFF(MONTH,0,GETDATE()),-1)
SELECT
 COUNT(NmContrato)
 FROM Corporativo.VwComercial
 where
DtFimVigencia>@mm13
and
DtInicioVigencia<@mm13

Answer: O problema é um pouco complexo.
Avalie a seguinte sugestão:
-- código #1
declare @Inicio_leitura date, @Inicio_mes_atual date;
set @Inicio_mes_atual= dateadd (month, datediff (month, 0, cast (sysdatetime() as date)), 0);
set @Inicio_leitura= dateadd (month, -13, @Inicio_mes_atual);

with
Calendario as (
SELECT @Inicio_leitura as Data
union all
SELECT dateadd (month, +1, Data)
  from Calendario
  where dateadd (month, +1, Data) < @Inicio_mes_atual
),
Contrato_Periodo as (
SELECT vC.NmContrato,
       dateadd (month, datediff (month, 0, vC.DtInicioVigencia), 0) as MesAno_inicio,
       dateadd (month, datediff (month, 0, vC.DtFimVigencia), 0) as MesAno_fim
  from Corporativo.vwComercial as vC
  where vC.DtInicioVigencia < @Inicio_mes_atual
        and vC.DtFimVigencia >= @Inicio_leitura
        --and vC.DtInicioVigencia < vC.DtFimVigencia
)
SELECT year (C.Data) as Ano, month (C.Data) as Mes,
       count (CP.NmContrato) as Ativos
  from Contrato_Periodo as CP
       cross join Calendario as C
  where C.Data between CP.MesAno_inicio and CP.MesAno_fim
  group by year (C.Data), month (C.Data);

Não testei; pode conter erro(s).
Na CTE Calendario são gerados os 13 últimos meses, excluindo o mês em curso.
Na CTE Contrato_Periodo são selecionados os contratos que estavam em vigência em pelo menos um dos 13 meses anteriores.
Ao final, através de produto cartesiano são geradas n linhas por contrato, uma para cada mês do período de emissão em que o contrato estava ativo. Após, são contabilizados os contratos por mês/ano.

Count em Intervalos de Tempo do Passado


Origin of terms "sensitivity" and "specificity"

Question: Who coined the terms "sensitivity" and "specificity"—meaning the complements of false positives, and false negatives, respectively in tests and measures—and when did they first do so?
Bonus points if there is a motivating etymology.
Comment: @NickCox Gilman, R. L., Boerner, F., & Lukens, M. (1940). A SIMPLIFIED COMPLEMENT FIXATION TECHNIC FOR THE DIAGNOSIS AND TREATMENT OF SYPHILIS: ITS SENSITIVITY AND SPECIFICITY. *Archives of Dermatology and Syphilology*, 41(1), 32-37. is certainly earlier. :)
Comment: Any retreat from this? Yerushalmy, J. 1947. Statistical Problems in Assessing Methods of Medical Diagnosis, with Special Reference to X-Ray Techniques. _Public Health Reports_  62(40), 1432-1449. doi:10.2307/4586294
Comment: You got me there. I just Googled and found a confident assertion that Yerushalmy was first, which is evidently wrong. I've found a few earlier occurrences before documented first occurrences myself for quite different terms. .
Comment: @TannerPhillips "How Specific is the test in detecting the illness?" **Specificity** is about detecting **not** the illness.
Comment: As far as etymology, it makes the most sense in a medical sense (I don't have a source, this is just how it was taught to me), when viewing the terms as relative to the desire to detect an illness. How well does a test Sense the presence of an illness? How Specific is the test in detecting the illness? Is it highly specific or all over the place?
Comment: I thought the terms came from radar operation in World War Two. The "receiver operator" in receiver operator characteristic curve was the operator of a radar who was looking for enemy planes.
Comment: I wonder if this goes back as far as the first telling of the Princess and the Pea.
Answer: Binney, N., C. Hyde and P. M. Bossuyt. 'On the origin of sensitivity and specificity'. Annals of Internal Medicine, 174 (2021): 401–7. 10.7326/M20-5028.
Comment: While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference.  Link-only answers can become invalid if the linked page changes. - [From Review](/review/late-answers/308321)
Answer: tl;dr (this answer is a bit of a book report): The terms sensitivity and specificity have their origin in the roots of immunology in the first years of the 20th century, and work towards antibody testing for syphilis.

As @NickCox and others in the published literature have suggested (e.g., Lilienfield, 2007), Yerushalmy was an exponent of the biostatistical concepts of sensitivity and specificity in a 1947 article on radiological tests for tuberculosis. However, it is reasonably easy to find earlier uses of these terms in the context of medical tests (e.g., Gilman, Boerner, and Lukens, 1940) using academic search engines.
In the 2021 article by Binney, Hyde, and Bossuyt cited by @PeterMilne (and well worth the read!), the authors trace the genesis of these terms' use to immunology, and

the development of serology in the early 20th century. They are closely linked to the development of the Wassermann reaction as a test for syphilis, and to notions of species of pathogen and of specific disease entities. […Where it was] used to distinguish diseased from non-diseased patients.

The terms were already in established use in the syphilis serology literature by the late 30s and early 40s. According to these authors, the concepts of sensitivity and specificity spread to disciplines like clinical epidemiology (and presumably from there to other epidemiologies :)—including for other diseases, such as tuberculosis, decision making in medicine, and disciplines where evaluating tests is important.
According to Binney, Hyde, & Bossuyt, the term specificity has its origins in experiments around 1901 demonstrating the capacity for blood serum from a living animal previously exposed to a known pathogen to successfully destroy that specific pathogen (in contrast to other pathogens). They go on to describe Jules Jean Baptiste Vincent Bordet's advancement of the immunological concepts of antibody and complement from this specific destructive power, where antibodies were described by him as a mechanism to sensitize an organism's immune response to a specific pathogen. From these definitions, Binney, Hyde, & Bossuyt argue, come the concept that "specific entities must be determined to identify infected individuals," and the development of the Wasserman antibody test* first embodied these concepts in technology. These authors cite work by Craig a few years later in 1910 presenting tables of numbers testing positive and negative among cases of syphilis, and percent testing positive among cases of syphilis, where the term specificity of the test was used to describe "healthy individuals in whom [syphilis] could be absolutely excluded, we tested seventy individuals, with a negative result in every instance."
The concept of sensitivity of the test has a blurrier origin, with Binney, Hyde, and Bossuyt writing:

Through the 1920s, the term "sensitive" referred both to the fraction of true positives and, sometimes in the same article, to the delicacy of an immunologic reaction: its propensity to return positive reactions, whether these were true or false positives.

They note that this overloaded meaning extended through to the 40s in the literature, until 1942 when American immunologist Reuben Leon Kahn wrote "Strictly speaking, the term 'sensitivity' when used in connection with tests for syphilis should apply only to cases of syphilis."

* The Wasserman test is not actually specific immunologically, but is diagnostically useful.

References
Craig, C.F. (1910). Observations upon the Noguchi modification of the Wassermann complement fixation test in the diagnosis of lues in the military service. Journal of Experimental Medicine. 12, 726–45.
Gilman, R. L., Boerner, F., & Lukens, M. (1940). A Simplified Complement Fixation Technic for the Diagnosis and Treatment Of Syphilis: Its Sensitivity and Specificity. Archives of Dermatology and Syphilology, 41(1), 32–37.
Kahn, R. L. (1942). Serology in Syphilis Control: Principles of Sensitivity and Specificity, With an Appendix for Health Officers and Industrial Physicians. Williams & Wilkins.
Lilienfeld, D. E. (2007). Abe and Yak: the interactions of Abraham M. Lilienfeld and Jacob Yerushalmy in the development of modern epidemiology (1945-1973). Epidemiology. 18, 507–14.
Yerushalmy, J. (1947). Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Public Health Reports. 62,1432–1449.


Construction of mollifiers to generate a desire function

Question: I am currently reading a book on analysis but do not understand the following:
It claims that we can construct a function $\phi: \mathbb{R}^3 \rightarrow \mathbb{R}$ such that
$1.$  $\quad $ $\phi \in C^2$ and has bounded derivatives (including itself) up to second order.
$2.$  $\quad $ $\phi (y) = \frac{1}{|y|}$ for $\epsilon \leq |y| \leq R$, where $R> \epsilon >0$.
I assume that we have to multiply the reciprocal function by some kind of mollifier with compact support, but can't figure out the details. Any suggestions?
Answer: It is straightforward (and tedious) to find an even polynomial $p$ of degree 4 such that $p(\epsilon) = {1 \over \epsilon}$, $p'(\epsilon) = -{1 \over \epsilon^2}$ and $p''(\epsilon) = 2{1 \over \epsilon^3}$.
Define $f:\mathbb{R} \to \mathbb{R}$ by $f(x) = \begin{cases} p(x), & |x| < \epsilon \\
{1 \over x}, & \text{otherwise}\end{cases}$. It should be clear that $f$ is $C^2$ and $f$ is bounded (not that it matters, but it is bounded by $f(0)$).
Now let $\phi(x) = f(\|x\|)$.
It should be clear that $\phi$ is $C^2$ for $x \neq 0$, since the function $x \mapsto \|x\|$ is smooth for $x \neq 0$.
Since $p$ is even, we can write $p(x) = q(x^2)$ for some polynomial of degree 2, and so, if $\|x\| < \epsilon$, we can write
$\phi(x) = q(\|x\|^2)$, and since $x \mapsto q(\|x\|^2)$ is smooth everywhere we see that $\phi$ is $C^2$ everywhere.
If $\|x\| \ge \epsilon$, we have $\phi(x) = {1 \over \|x\|}$, hence we see that $\phi, \phi', \phi''$ are bounded for $\|x\| \ge \epsilon$. It follows by continuity that $\phi, \phi', \phi''$ are bounded everywhere.


How can the support of proposal distribution impact convergence of RH-MH algorithm?

Question: In the book Introducing Monte Carlo Methods by Casella and Robert, there's a sentence with which I'm having some trouble to understand.
«If the domain explored in $q$ [proposal] is too small, compared with the range of $f$(the target's distribution density), the Markov Chain will have difficulties in exploring this range and will converge very slowly»
The acceptance probability depends on the proposed state $y$, through $$\frac{f(y)}{q(y|x)}$$ The higher this proportion, the higher the chances of accepting the proposed move.
Unintuitive to me at least, a very low transition prob. $q(y|x)$ w.r.t. $f(y)$ would increase the chances of accepting the new move. So, if I'm at $x$ and I'm proposed a move to a lower transition probability state, then I'll want to move...
Is that to make me explore the whole domain of $q$?
How can the support of proposal distribution impact convergence of RH-MH algorithm?
Comment: You want to explore $f$, not $q$. And yeah, if $q(y|x)$ is small and $f(y)$ big you gotta "seize the opportunity" and jump before you miss your chance.
Answer: While obviously biased, I have trouble to see the difficulty with that sentence! If the proposal distribution has a bounded support, the Markov chain can only move to a point within that support and nowhere else. This means connecting two values $y_1$ and $y_2$ that have positive densities, $f(y_1)>0$ and $f(y_2)>0$ but are far away will require many iterations of the Metropolis-Hastings algorithm. In the worst case, if the support of $f$ is made of two disjoint sets $A_1$ and $A_2$ with $y_1\in A_1$ and $y_2\in A_2$ and if the distance between both sets is larger than the diameter of the support of $q(\cdot|\cdot)$ the Markov chain cannot leave the $A_i$ it started from.
Here is an illustration on a basic example:
<code>target<-function(x){
  (sum((x-c(1,1))^2)<1)+(sum((x-c(-1,-1))^2)<1)
}

propal<-function(x,y){
  prod((x-y)^2<.01)
}

hastin<-function(x){
  prop=x+runif(2,-.1,.1)
  if (runif(1)*target(x)*propal(x,prop)>target(prop)*propal(prop,x))
    prop=x
  return(prop)}

T=2e3
mark1=matrix(0,T,2)
mark1[1,]=c(1,1)
for (t in 2:T)
  mark1[t,]=hastin(mark1[t-1,])

mark2=matrix(0,T,2)
mark2[1,]=-c(1,1)
for (t in 2:T)
  mark2[t,]=hastin(mark2[t-1,])

plot(mark1,col="sienna",type="l",axes=FALSE,xlim=c(-2,2),ylim=c(-2,2),xlab="",ylab="")
lines(mark2,col="steelblue")
</code>
with outcome

This poor behaviour of $q$'s with small support is unrelated with the point that when $f(y)/q(y|x)$ is large the chain should move there, which is correct.
Comment: Now I get it. My confusion came from the (mis?)use of the words domain/range in the sentence. I would write something like «If the space explored by $q$ ( $\text{supp}(f)$ ) is too small when compared with the domain of $f$, the Markov Chain will have difficulties in exploring $f$'s domain and thus will converge very slowly, if at all for practical purposes.
Comment: I hope you didn't take it badly, I'm not a native, and I'm self-studying this. So, I must absorb the most I can from each specific word. Since I have no one to ask in case of doubt, and sometimes using this forum helps, but it's not a perfect substitute for a physical professor. Also, I'm a really slow learner... ;)


How to set a timeout between each iteration in Javascript?

Question: I am trying to set a timeout(wait time) between each call of drawLines() inside a setTimeout() in a while-loop.
while(i < j){
            while(i < lastElIndex && array[i] < pivot){
                const iCopy = i;
                const lastCopy = lastElIndex;
                i++;
                const arrCopy = array.slice();
                setTimeout(() => {
                    drawLines(arrCopy, iCopy, -1, lastCopy, -1);
                });
            }

            while(j > firstElIndex && array[j] >= pivot){
                const jCopy = j;
                const firstCopy = firstElIndex;
                j--;
                const arrCopy = array.slice();
                setTimeout(() => {
                    drawLines(arrCopy, -1, jCopy,-1, firstCopy);
                });
            }

I already tried setting the wait time like this
setTimeout(() => {drawLines(arrCopy, iCopy, -1, lastCopy, -1);}, 1000);

...but the timeout happens before the calling drawLines() repeatedly without waiting between calls.
I also tried
setTimeout(async() => {
           drawLines(arrCopy, iCopy, -1, lastCopy, -1);
           await sleep(1000);
});

...with
function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

...but nothing happened
Here is the whole function:
function quickSort(arr, firstElIndex, lastElIndex){
    let array = arr;
    let currentIndex = 0;
    function partition(firstElIndex, lastElIndex) {
        let i = firstElIndex;
        let j = lastElIndex;
        let pivot = array[lastElIndex];
        while(i < j){
            while(i < lastElIndex && array[i] < pivot){
                const iCopy = i;
                const lastCopy = lastElIndex;
                i++;
                const arrCopy = array.slice();
                setTimeout(() => {
                    drawLines(arrCopy, iCopy, -1, lastCopy, -1);
                });
            }

            while(j > firstElIndex && array[j] >= pivot){
                const jCopy = j;
                const firstCopy = firstElIndex;
                j--;
                const arrCopy = array.slice();
                setTimeout(() => {
                    drawLines(arrCopy, -1, jCopy,-1, firstCopy);
                });
            }

            if(i < j){
                array.swap(i, j);
            }
        }
        if(array[i] > pivot){
            array.swap(i, lastElIndex);
        }
        return i;
    }
    if(firstElIndex < lastElIndex){
        currentIndex = partition(firstElIndex, lastElIndex);
        quickSort(array, firstElIndex, currentIndex - 1);
        quickSort(array, currentIndex + 1, lastElIndex);
    }
    setTimeout(() => {drawLines(array, -1, -1);}, 1000);
}

And here is drawLines():
function drawLines(arr, x, y, rightPivot, leftPivot){
    let array = arr.slice();
    let container = document.getElementById("centerContent");
    let line = undefined;
    container.innerHTML = '';
    for(let i = 0; i < array.length; i++){
        let my_length = (array[i]/6.7).toString() + "vw";line = document.createElement("div");
        line.className = "line";
        if(i === x || i === y){
            line.style.borderLeft = (1/7).toString() + "vw solid red";
        }
        else if(i === rightPivot){
            line.style.borderLeft = (1/7).toString() + "vw solid aqua";
        }
        else if(i === leftPivot){
            line.style.borderLeft = (1/7).toString() + "vw solid orange";
        }
        else{
            line.style.borderLeft = (1/7).toString() + "vw solid black";
        }
        line.style.height = my_length;
        container.appendChild(line);
    }
}

I apoligize if the question is poorly structured. It's the my first one. Many thanks in advance.
Comment: `setTimeout` doesn't wait for the time to pass. It puts the function on the stack to get run after the timeout delay expires, but then it continues with your function immediately
Comment: Yes, i realised that. I want to put the function + wait time in the stack, so there is wait time between each function call.
Answer: It sounds like you just want a delay to happen before the line is drawn and also before the loop continues. Using async/await can do that for you fairly easily. This is a structure you can use:
// to use await you must define the function to be async
async function partition(firstElIndex, lastElIndex) {
  let i = firstElIndex;
  let j = lastElIndex;
  let pivot = array[lastElIndex];
  while(i < j){
    while(i < lastElIndex && array[i] < pivot){
      const iCopy = i;
      const lastCopy = lastElIndex;
      i++;
      const arrCopy = array.slice();
      // this line pauses execution for 1 second
      await new Promise(resolve => setTimeout(resolve, 1000);
      drawLines(arrCopy, iCopy, -1, lastCopy, -1);
    }

    while(j > firstElIndex && array[j] >= pivot){
      const jCopy = j;
      const firstCopy = firstElIndex;
      j--;
      const arrCopy = array.slice();
      // this line pauses execution for 1 second
      await new Promise(resolve => setTimeout(resolve, 1000);
      drawLines(arrCopy, -1, jCopy,-1, firstCopy);
    }
    if(i < j){
      array.swap(i, j);
    }
    return i;
  }
}

and since partition is now an async function you need to do this when calling it:
if(firstElIndex < lastElIndex){
  currentIndex = await partition(firstElIndex, lastElIndex);
  quickSort(array, firstElIndex, currentIndex - 1);
  quickSort(array, currentIndex + 1, lastElIndex);
}

That requires that quickSort is also and async function which means it returns a Promise object that can be either waited for too, or use .then to know when it's complete.
Comment: This is exactly what I was trying to achieve. Thank you very much!
Answer: The first thing is that the setTimeout needs a 2nd parameter with the amount of time to wait, so written like this:
setTimeout(() => {
  drawLines(arrCopy, iCopy, -1, lastCopy, -1);
});

will not wait for any amount of time, unless you pass it a 2nd argument like:
setTimeout(() => {
  drawLines(arrCopy, iCopy, -1, lastCopy, -1);
}, 500);

I think you where on the correct path when creating that sleep function; indeed it wouldn't work when running the timeout inside the loop. But there's no need to return a Promise. Just try:
function sleep(callback, ...callbackParameters) {
  setTimeout(() => callback(...callbackParameters), 500)  // remember your timer
}

And then pass whatever function you want to run after the timeout to your sleep function like:
sleep(drawLines, arrCopy, iCopy, -1, lastCopy, -1)


Turning on verbose logging

Question: Is the "Verbose logging" in Drupal actually the PHP errors that can be activated in various ways (.htaccess, php.ini, settings.php) to be seen on screen?
Is it drush ws --full ?
From the material I've seen I can't say for sure if there is a unique definition for this concept in Drupal.
I would thank an experienced Drupal programmer to give the final word on this.
Answer: Navigate to admin/config/development/logging and set "All messages, with backtrace information".
Comment: This answer may be actually what you are asking: http://drupal.stackexchange.com/questions/127182/how-to-enable-developer-debug-mode-in-drupal-8?rq=1
Comment: Actually, these are equivalent.

Turning on verbose logging


Hitting times of a biased continuous time random walk

Question: Let $X_{s \geq 0}$ be a continuous time random walk on $\mathbb{Z}$, i.e. waiting times between jumps are exponentially distributed with mean one. The random walk is biased: $\mathbb{P}(X_s\text{ jumps up}) = 2/3$, $\mathbb{P}(X_s\text{ jumps down}) = 1/3$. Let $a \in \mathbb{Z}_{+}$. I am trying to derive a lower-bound on $\mathbb{P}(\tau_a < s)$ for $a$ and $s$ large, where $\tau_a$ is a hitting time of $a$. I tried embedding this random walk into a Brownian motion and then deal with hitting times of a BM of a line, however the calculations of time rescaling are very technical.
Answer: Not a complete answer, but this might give you some ideas about that probability:
If the jumps arrive with exponential waiting time, your process can be expressed as a compound Poisson process:
$$
X_t = \sum^{N_t}_{i=1}J_i
$$
where $N_t$ is a Poisson process with intensity $\lambda=1$ and $J_i$ are iid random variables which are $1$ with probability $2/3$ and $-1$ with probability $1/3$.
To find the probability, condition on number of jumps in the interval $[0, s]$:
$$
\mathbb P(\tau_a <s) = \mathbb E[\mathbb E[1_{\tau_a<s} \mid N_s = n]] = \sum^{\infty}_{n=0}\mathbb E[1_{\tau_a<s} \mid N_s = n] \mathbb P(N_s=n)\\
=\sum^{\infty}_{n=0}\mathbb P(\tau_a<s \mid N_s = n) e^{-\lambda s}\frac{(\lambda s)^n}{n!}
$$
where $\lambda=1$ in your case. The probability inside the sum is the same as
$$
P(\tau_a<s \mid N_s = n) =\mathbb P \left( \max_{1\leq k\leq n}\sum^k_{i=1} J_i \geq a \right)
$$
which can be viewed as the probability that a simple asymmetric random walk reaches the level $a$ within $n$ steps. That is a combinatorial problem which most probably has been solved by someone.
Comment: Yeah, I forgot to mention that conditioning on the number of jumps yields a solution, however estimating such a sum is a struggle for me.


Question related to approximation of measurable functions by simple functions

Question: I am self-studying measure theory from the book by Sheldon Axler.There I found a theorem before integration is introduced:
Let $(X,\mathcal S)$ be a measurable space and $f:X\to [-\infty,\infty]$ be a measurable function.Then there exists a sequence of simple measurable functions $(f_n)$ such that:
$1.$ $|f_n(x)|\leq |f_{n+1}(x)|\leq |f(x)|$ for all $n\in \mathbb N$ and $x\in  X$.
$2.$ $f_n\to f$ pointwise on $X$.
Now I have a number of questions regarding this theorem:
First one is that why the codomain of $f$ is taken as $[-\infty,\infty]$ instead of $\mathbb R$.Will it affect the theorems related to integrals,if we choose $\mathbb R$?
Second one is that why is modulus sign given in $(1)$.Can I simply omit the moduli by taking $f\geq 0$.
Now comes the next section of the question,about the proof of this theorem(even if it is assumed that codomain is $\mathbb R$ and $f\geq 0$).I still do not get how to prove this result.Can someone guide me a bit by providing intuition so that I can visualize the idea behind the proof,because the proof I found in the book is a bit technical and constructive.
Comment: There is some repetition in 1., perhaps one of the indices is off by 1?
Answer: The intuition is that you are essentially discretizing $f$ with granularity $h > 0$: Let $f : X \to [0, \infty]$ be measurable. Let $h > 0$. Define $f_h : X \to [0, \infty]$ by
$$f_h(x) = \lfloor\frac{f(x)}{h}\rfloor h.$$
It's clear that $f_h \nearrow f$ as $h \to 0$. $f_h$ takes the values $\{0, h, 2h, 3h, \dots\} \cup \{\infty\}$. So $f_h$ is not a simple function since it takes on countably many values instead of just finitely many. To fix this, use $\tilde{f_h} = f_h \land \alpha(h)$, where $\alpha(h)$ is any function with $\alpha(h) \nearrow \infty$ as $h \to 0$, e.g. $\alpha(h) = 1/h$. Now $\tilde{f_h}$ takes the values $\{nh : n \in \mathbb{N}, n \leq \alpha(h)\}$, which is a finite set, and $\tilde{f_h} \nearrow f \land \infty = f$ as required.
The most common proof of the above approximation theorem uses the sequence
$$f_n = \left(\lfloor\frac{f(x)}{2^{-n}}\rfloor 2^{-n}\right) \land n.$$
In my notation above, this corresponds to $h = 2^{-n}$ and $\alpha(h) = -\log_2(h) = \log_2(1/h)$.
Comment: Please correct me if I am wrong.I am defining $f_k\to f$ as follows:
Comment: For $k\in \mathbb N$,define $f_k(x)=k$ if $f(x)>k$ and and if $0\leq f(x)<k$ then $f_k(x)$ is defined in the following manner: We first divide each interval of the form $[n,n+1)$ within $[0,k]$ to $2^k$ subintervals and define $f_k(x)$ to be the left endpoint of the subinterval in which $f(x)$ falls.
Comment: I want to know why $f_k$ is measurable provided $f$ is measurable.
Comment: @KishalaySarkar You have $f_k = \left(\lfloor\frac{f(x)}{2^{-k}}\rfloor 2^{-k}\right) \land k$. So you just have to show that the floor function is measurable and that the minimum of two measurable functions is measurable. Both of these are straightforward.
Comment: what about my first question?Can we replace the codomain by $\mathbb R$?
Comment: @KishalaySarkar The proof works when the codomain is $[0, \infty]$, and by subtraction, you get the result when the codomain is $[-\infty, \infty]$. It is occasionally useful to deal with functions that take infinite values, e.g. $\limsup_{n}f_n$. There is no point in only restricting attention to real valued functions since the proof does not change.


SQL query in Node.js server looping

Question: I am trying to set up a server using Node.js.
For the database I am using Xampp.
I want to run the updateEngine() function when /update is provided.
Everything is working , except the code is looping and after a while I get an error.
(I used node --max_old_space_size=2000000)

Output
server is up!!
connected as id 38
into the query
connected as id 39
into the query

Error
 69777 ms: Mark-sweep 1398.8 (1456.7) -> 1398.8 (1456.7) MB, 1871.5 / 0 ms [allocation failure] [GC in old space requested].
   71705 ms: Mark-sweep 1398.8 (1456.7) -> 1398.8 (1456.7) MB, 1928.0 / 0 ms [allocation failure] [GC in old space requested].
   73346 ms: Mark-sweep 1398.8 (1456.7) -> 1398.8 (1456.7) MB, 1640.7 / 0 ms [last resort gc].
   75283 ms: Mark-sweep 1398.8 (1456.7) -> 1398.8 (1456.7) MB, 1937.4 / 0 ms [last resort gc].

==== JS stack trace ========FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Code
http = require('http');
var connect = require('connect');
var mysql = require('mysql');
var app = connect();

app.use('/update',updateEngine);

http.createServer(app).listen(8080);
console.log("server is up!!");

function updateEngine(request,response,done) {
    var connection = mysql.createConnection({
        host: 'localhost',
        port: '3306',
        user: 'root',
        password: '',
        database: 'test'
    });
    connection.connect(function(err) {
        if (err) {
            console.error('error connecting: ' + err.stack);
            return;
        }
        console.log('connected as id ' + connection.threadId);
    });

    connection.query('SELECT * FROM `testtable`' , function (error, results, fields) {
        console.log("into the query");
    });
    connection.end();
}

Xampp
The MySQL server
[mysqld]
user = mysql
port=3306
socket      = /Applications/XAMPP/xamppfiles/var/mysql/mysql.sock
skip-external-locking
key_buffer = 16M
max_allowed_packet = 500M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M

Answer: Suggestions:

Send a response to your requests (even if it's just a simple 200 OK with no body). Right now it's never responding and so the client is probably timing out and automatically retrying the request.
Use a database connection pool that you create on startup instead of creating a new database connection for every single request. Using a connection pool is much more efficient since you are not creating and tearing down a connection every time (which adds overhead and latency to your requests).
Comment: I sent a simple 200 response but the error is still there...The path will be rarely used. Is this a node issue or mysql configuration ?
Comment: There is nothing else "wrong" with the code you've posted so far. The problem lies elsewhere.
Comment: testtable has 100.000 entries , is this a problem ? The response is presented in the browser before the query is finished
Comment: That *might* explain the OOM error, but it certainly does not explain the duplicate calls to `updateEngine()`.
Comment: I updated my question with sql details. can I avoid the error ? connection.end() stops the query ? It seems that in the activity monitor node is hitting 100% and then I get the error.

SQL query in Node.js server looping


Vba/Excel criando uma função no Excel

Question: Estou tentando criar uma função, onde coloco uma vírgula depois de cada número ou palavra.
Segue o script do vba:
Public Function Virgula(Texto1, Texto2)
    Virgula = Texto1 & "," & Texto2 & ","
End function

É como concatenar.
Só consigo até dois, mas queria até 1000, se possível.
Comment: Você pode usar a função [`JOIN()`](https://learn.microsoft.com/en-us/office/vba/language/reference/user-interface-help/join-function) para juntar um array de strings usando um separador. Veja se esse exemplo em VB.NET o ajuda https://ideone.com/JYZ3Sy
Answer: No VBA já existe uma função que acredito que faça o que está querendo, é a função join.
Ela recebe um array com as palavras que quer unir, e o segundo parâmetro é o delimitador.
No exemplo abaixo, a função está unindo os itens com uma vírgula entre eles:
Dim itens(0 To 2) As String
itens(0) = "Chave"
itens(1) = "Livro"
itens(3) = "Mouse"

Dim listaItens As String
listaItens = Join(itens, ",")

O resultado será "Chave, Livro, Mouse"
Comment: Como está escrito atualmente, sua resposta não é clara. Por favor, pode [edit] para adicionar mais detalhes, que ajudarão outras pessoas a entender como isso resolve a pergunta feita. Você pode encontrar mais informações sobre como escrever boas respostas [na Central de Ajuda](/help/how-to-answer).
Comment: Faça isso dentro de uma função usando ParamArray pois a pergunta é sobre criar uma função com parâmetros variáveis. e.g. https://pt.stackoverflow.com/questions/575525/vba-excel-criando-uma-fun%c3%a7%c3%a3o-no-excel#comment1009947_575525

Vba/Excel criando uma função no Excel


When were chords invented?

Question: I'd like to know in what period/year chords were invented and by whom.
Did musicians like Bach, Mozart, and Beethoven think in chords to harmonize their melodies or did chords only emerge in the 1900s?
note: this is a secondary question to my question Why aren't chords written in classical sheet music?
one of the commenters said that chords weren't invented during that time, which was very surprising for me to hear. so I want to know when they were invented.
Comment: The comment in the other question is that chord ***names*** hadn't been invented. Chords being played and heard likely dates back at least to Ancient Greece. If you count only two notes as a chord, then the aulos was used to play two note chords as early as 5000 years ago!
Comment: As others have mentioned, it depends on your definition of "chord". Certainly chorales were full of what we would consider chords, but the were treated as a collection of "voices", each leading to its own resolution. That's why they had such strict rules regarding intervallic movement.
Answer: It depends on how you define "chords," but I assume you mean "tertian sonorities" like triads and seventh chords.
If so, the music theorist Johannes Lippius discussed the "harmonic triad" in his Synopsis musicae novae (Synopsis of New Music) in 1612. In a quote from this article:

The concept of triadic generation and invertibility, so clearly and firmly established by Johannes Lippius in 1610 and again in 1612, constitutes a milestone in the history of harmonic theory.

In that same article, the author makes a case that Ramos de Pareja preceded Lippius. In his Musica practica of 1482, Ramos discussed fixing the intonation of thirds and sixths in order to create triadic structures.
Comment: Don't misquote me. I said **the modern concept of chord notation** wasn't invented in the Baroque period. Well, actually it *was* invented *during* the Baroque period, and first published in a book written by Rameau in 1744 - and that book was widely considered to be avant-garde nonsense when it first appeared in print! Bach and his contemporaries certainly didn't think of "chord progressions" in the sense of "C Em F G7" or whatever - that notation was non-existent at the time.
Comment: … If you want to know how Bach and his contemporaries *did* think of chords, read the relevant chapters of C P E Bach's "True Art of Playing Keyboard Instruments" (C P E was one of J S Bach's sons). FWIW this thread https://music.stackexchange.com/questions/27308/iii9-i-progression-in-bach?rq=1 is a good example of several people (including Walter Piston, who ought to have known better IMO) making a complete hash of analysing something that is really very simple to understand, by trying to explain it using "modern" chord theory, which is *not* the way its composer thought about it!.
Comment: @foreyez While I don't think that Bach thought about melodic lines in terms of harmony either, it's also true that he tends to adhere to many of the rules of harmony without thinking about it. If you want to really feel like you understand the subject, I would suggest that you consider following alephzero's advice about reading CPE Bach, and form your own opinions after doing so.
Comment: hmm, in my last question, @alephzero said that chords weren't invented in the Baroque period. but according to you it seems they were.
Comment: So I assume (until someone tells me different) that Bach, for example, knew every little thing there is to know about chords when composing his music.
Comment: Actually, Gioseffo Zarlino described triads as the basis of harmony in his _Institutioni Harmoniche_ in 1558.  But triads had arguably been part of musical practice since the time of Dufay (ca 1400- 1474).
Comment: @user19146 I assume the above comment by the mysterious numbered user is referring to [Treatise on Harmony](https://en.wikipedia.org/wiki/Jean-Philippe_Rameau#Treatise_on_Harmony,_1722) from 1722?  In any case, thanks to this comment I just discovered a really interesting composer, so cool
Comment: @foreyez  yes chords were certainly known in the Baroque period. It was normal to write a bass line to be used as the notation of the chords. Examples: If the key signature was two sharps and the bass note was a D, the keyboard player was supposed to play a D major chord. If the key signature was one flat and the bass note a D, the chord to be played was supposed to be D minor. If something else was supposed to be played there was a figure, a number, under the bass note indicating what kind of chord to play. And regarding Bach: Yes no doubt that Bach's knowledge of chords was thorough.
Comment: @LarsPeterSchultz what you said conflicts with the third comment. I think thorough bass/rule of the octave was the main thing back then.
Comment: @foreyez I don't know what you mean by "bass/rule of the octave". Anyway what I wrote is correct. Find a sonata by Händel written for violin and piano. Today it is often published with a full written out piano part. But Händel only wrote a bass line with figures, a socalled "figured bass". Then it is up to the piano player to invent the right hand based on that. If a publisher has printed a right hand written out it is only one possible way to play it. But since many keyboard players in our time can not invent the right hand based on a figured bass they need an edition where it is written out.

When were chords invented?


line break formatting in IF/ELIF statement [Python3]

Question: How would you format this code? I know there should be two line breaks before defining a function. Would you put another line break after if new_requests_objectid > 0: for readability? How about after def assign_uniquekey_request(): or assign_uniquekey_request()?
Or maybe it's fine the way it is.
if new_requests_objectid > 0:
    # get last data object from Request table
    last_request = Popen(['curl',
                          '-H', 'application-id: %s' % backendless_appid,
                          '-H', 'secret-key: %s' % backendless_sk,
                          '-H', 'application-type: REST',
                          '-H', 'Content-Type: application/json',
                          '-X', 'GET',
                          '-v', 'https://api.backendless.com/v1/data/Request/last'],
                         stdout=PIPE).communicate()[0]

    last_request_str = last_request.decode(encoding='utf-8')

    last_objectid = json.loads(last_request_str, strict=False)['objectId']

    # generate unique key for last entry in Request table
    uniquekey_request = ''.join(random.SystemRandom().choice(string.ascii_uppercase + string.digits)
                                for _ in range(6))

    # assign unique key to last entry in Request table
    def assign_uniquekey_request():
        Popen(['curl',
               '-H', 'application-id: %s' % backendless_appid,
               '-H', 'secret-key: %s' % backendless_sk,
               '-H', 'application-type: REST',
               '-H', 'Content-Type: application/json',
               '-X', 'PUT',
               '-d', '{\"requestId\":\"%s\"}' % uniquekey_request,
               '-v', 'https://api.backendless.com/v1/data/Request/%s' % last_objectid])

    assign_uniquekey_request()
else:
    pass

Comment: Aah! Don't use cURL like that. If you _really_ want to use cURL, use [PycURL](http://pycurl.sourceforge.net/). But I would **highly** recommend using [requests](http://python-requests.org/).
Comment: @fingaz - That's not just readability; it can affect the result of the code. However, I would agree that classes and functions should be defined unconditionally, and then called conditionally if necessary.
Comment: I think that it is opinion of each person so it should be marked somehow...
Comment: Your indentation is off, but [no, there is no need to insert a new line after an `if` statement or function signature](https://www.python.org/dev/peps/pep-0008/#blank-lines).
Comment: Well, first, you need to post a code sample that's actually correctly indented in the first place. Second, you have some wonky things going on here. Are you really defining that function inside your if statement? Does that function really only need to be defined conditionally? Also, why `elif` and not `else`? Why not just rewrite your `if` condition if you're only going to pass anyway?
Comment: The only readability change could be to place the function outside your if/elif statement. Otherwise, it's opinion.
Comment: @TigerhawkT3 I was unsure if that was valid to say. Thank you for clarifying, I knew it didn't agree with me.
Comment: @Two-BitAlchemist Fixed for indentation. Also changed `elif` to `else`. The `pass` was a placeholder but I think `else` works in my case. And you are right about defining the functions outside the statement. Noob here. Thanks for the advice.
Comment: @Cyphase Will definitely look into `requests`. I was using `curl` to capture response times and just used it for consistency. I will research to see if `requests` can do that.
Answer: From PEP 0008:

Surround top-level function and class definitions with two blank lines.
Method definitions inside a class are surrounded by a single blank line.
Extra blank lines may be used (sparingly) to separate groups of related functions. Blank lines may be omitted between a bunch of related one-liners (e.g. a set of dummy implementations).
Use blank lines in functions, sparingly, to indicate logical sections.

Your code's newline formatting looks fine. See the sample below:
class A:
    def __init__(self):
        pass

    def actions(self):
        if 1:
            pass
        else:
            pass

        print(1)
        print(37)
        print(22)

        return

Note that newline rules are rigidly-defined and don't follow this guideline if you are working in the interactive interpreter, where you should only add an extra newline when you want the block you just finished writing to execute. The class above would need to be entered into the interpreter like this:
>>> class A:
...     def __init__(self):
...         pass
...     def actions(self):
...         if 1:
...             pass
...         else:
...             pass
...         print(1)
...         print(37)
...         print(22)
...         return
...
>>>


Algebriac curves in $\mathbb P_1\times\mathbb P_1$

Question: I know that in $\mathbb P_2$, If we have an algebraic curve which a line intersects at one point(counting multiplicities) then it is a line, this is a direct consequence of bezout's theorem.(we are over an algebraically closed field)
Is a version of this true in $\mathbb P_1\times\mathbb P_1$?
What characterizes a curve in $\mathbb P_1\times\mathbb P_1$ such that if we fix one of the coordinates it has only one point?
Answer: Say $\mathbf{n}:=(n_1,n_2,\dots,n_k)$ and $\mathbb{P}^{\mathbf{n}}$ denotes the space $$
\mathbb{P}^{\mathbf{n}} := \prod_{i=1}^k \mathbb{P}^{n_i}.
$$ One defines a linear subspace of $\mathbb{P}^{\mathbf{n}}$ as a product $L:=L_1\times L_2\times\dots L_k$ of linear subspaces $L_i\subset\mathbb{P}^{n_i}$. Call $\dim L:=\sum_{i=1}^k \dim L_i$ the dimension of $L$ and the tuple $(\dim L_1,\dim L_2,\dots,\dim L_k)$ the format of $L$. Then the following is true.
Assume that $V\subset\mathbb{P}^{\mathbf{n}}$ is an algebraic variety of dimension $m=\dim V$. Then, $V$ is a linear subspace if and only if the following conditions are satisfied:

There exists a format $(\alpha_1,\alpha_2,\dots,\alpha_k)$ satisfying $\sum \alpha_i = \dim\mathbb{P}^{\mathbf{n}}-m$ such that any $L$ of format $\alpha$ intersect $V$ at a single point.

For every format $(\beta_1,\dots,\beta_k)$ with $\sum_{i=1}^k \beta_i = \dim\mathbb{P}^{\mathbf{n}}-m$ except $(\alpha_1,\dots,\alpha_k)$, a generic subspace $L$ of format $\beta$ does not intersect $V$.

In the case that $\mathbf{n}=(1,1)$, what this says is the following:
If $V\subset\mathbb{P}^1\times\mathbb{P}^1$, then there are three possibilities:

$V$ intersects $p\times\mathbb{P}^1$ for every $p\in\mathbb{P}^1$ at a single point but it does not intersect with $\mathbb{P}^1\times q$ for generic $q\in\mathbb{P}^1$. In this case, $V$ is linear and $V=\mathbb{P}^1\times q$ for some $q$.

$V$ intersects $\mathbb{P}^1\times p$ for every $p\in\mathbb{P}^1$ at a single point but it does not intersect with $q\times\mathbb{P}^1$ for generic $q\in\mathbb{P}^1$. In this case, $V$ is again linear and $V=p\times\mathbb{P}^1$ for some $q$.

Other cases. Then $V$ cannot be linear. For example take $V=\mathcal{Z}( x_0 y_0 + x_1 y_1 )\subset\mathbb{P}^1\times\mathbb{P}^1$. Then, $V$ intersects with linear subspaces $p\times\mathbb{P}^1$ at a single point for every $p$ and also with linear subspaces $\mathbb{P}^1\times q$ at a single point for every $q$. But, $V$ is not linear.
Comment: What is $V$ if not linear?
Comment: @razivo Well, it can be many things. Consider the example $V=\mathcal{Z}(x_0 y_0+x_1 y_1)$ that I gave. It intersects every line at a single point but it is not itself a linear subspace. But this is very general, any other variety with this property is somehow 'equivalent' to $V$.


Azure function to send push notifications (Notification Hub + NodeJs)

Question: I'm working on an Azure function that will use the Azure notification hub module to send push notifications. The official documentation for the notification hub contains only samples using .Net. I could not find anywhere the official documentation for the Notification Hub NodeJs library. Can anyone who had the chance to work on something similar and can shed some light on the path to follow? That would be highly appreciated. Thanks!
Answer: The official provides a sample document about nodejs using notificationhub.
1. Sending push notifications with Azure Notification Hubs and Node.js
2. Azure Notification Hubs modules for JavaScript
You also can read this blogs about Push Notifications Tutorial, and you can read related links, hope it can help you.
Comment: Thank you very much. I had the same references and still was not successful (the sample provided by MS did not work for me). HOWEVER, and this is for anyone that comes across the same issue, the article provided here (https://www.intertech.com/push-notifications-tutorial-node-js-and-azure-setup/) (your third link) did specifically mention using the DefaultFullSharedAccessSignature endpoint in the notification hub instead of the DefaultListenSharedAccessSignature, and that was the root cause for not working for me.