Trying to understand math behind "Checking whether a coin is fair"

Question: I would like to understand the standard math done to check whether a coin is fair.
There is a wikipedia article on it, and it outlines two different ways of answering the question using a Baysian and Frequentist approach.
I would like to understand the Frequentist approach, which calculates an "estimator of true probability."
To quote wikipedia:

The best estimator for the actual value r is the estimator
$p\,\!={\frac {h}{h+t}}.$ This estimator has a margin of error (E)
where $|p-r|<E$ at a particular confidence level.

What does this mean? How do we know that this is the case?
One answerer in this question does the math with the following formula:

The formula to calculate the approximate confidence limits for a
binomial test is: $z_{a l p h a
> / 2}  \sqrt{p q / n}$

How do I find this formula and how is it related to the "best estimator for r"?
In general, I have a rough understanding of central limit theorem, so I get that some kind of normal distribution might pop up if I start looking at a combination of bunch of trials of independent random distributions, but I would like to understand the mathematical derivation for this "estimator" for determining if a coin is fair.
Answer: Let's start with a simple example. Suppose you want to know whether a coin is fair; that is $p = P(Head) = 1/2.$ You toss the coin $n = 50% times and get
$x = 28$ Heads.
Then the point estimate for $p$ is $\hat p = x/n = 28/50 = 0.56.$
Various styles of confidence intervals can be used to give an idea how far
from $0.56$ the actual value of $p$ might be.
Confidence intervals. One of these is a 95% confidence interval (CI) of the form $\hat p \pm 1.96\sqrt{\frac {\hat p(1-\hat p)}{n} },$ which computes to $(0.422, 0.698).$ [Computation below, using R as a calculator.] Over many such experiments, this style of CI
will include the true Heads probability $p$ of the coin about 95% of the time.
The expression qnorm(c(.025,.975)) amounts to $\pm 1.96$ and is based on
a normal approximation to the binomial distribution of $X.$ [Wald CI; see Addendum.]
x = 28;  n = 50;  p.hat = x/n
CI = p.hat + qnorm(c(.025,.975))*sqrt(p.hat*(1-p.hat)/n)
CI
[1] 0.4224111 0.6975889

Experience has shown that a slightly different style of CI (due to Agresti and Coull) comes closer to the 95% goal than the one above. It gives the 95% CI $(0.423, 0.688).$ [Sometimes shorter than the Wald CI, sometimes longer, but with greater precision attuned to the circumstances.]
x = 28;  n = 50;  p.est = (x+2)/(n+4)
CI = p.est + qnorm(c(.025,.975))*sqrt(p.est*(1-p.est)/(n+4))
[1] 0.4230227 0.6880885

Notice that both styles give intervals that contain $p = 0.5.$ So
data $x = 28, n = 50$ with $\hat p = 0.56$ seems compatible with a fair coin.
(However, with $x = 2792$ heads out of $n = 5000$ tosses, the CI's would
not contain $p = 0.5.$ With such experiments, it is not just the proportion
of heads that matters, but also the total number of tosses.)
Test of hypothesis. Another possibility is to test the null hypothesis $H_0: p=1/2$ against
the alternative $H_a: p\ne 1/2,$ at the 5% level of significance.
There are several versions of this test, some using exact binomial
distributions and some using normal approximations. In R, an exact
test is implemented as binom.test, illustrated for data $x = 28, n = 50$ from above. The P-value $0.4799$ tells us that a fair coin would give a result
as far $28 - 25 = 3$ from the expected number of heads almost half the time.
So 28 heads in 50 tosses would not be an unusual result for a fair coin.
binom.test(x = 28, n = 50, p = 0.5)

        Exact binomial test

data:  28 and 50
number of successes = 28, number of trials = 50, p-value = 0.4799
alternative hypothesis:
 true probability of success is not equal to 0.5
95 percent confidence interval:
 0.4125441 0.7000928
sample estimates:
probability of success
                  0.56

Here is how the P-value $0.4799$ is computed: If $H_0$ is true then
the observed number of Heads is $X \sim \mathsf{Binom}(n=50,p-0.5).$
The P-value is the probability that $X$ is farther from $E(X) = np = 25$
that what we observed. That is, $P(X \le 22)+P(X\ge 28) =$ $2(0.23994) = 0.4799.$
pbinom(22, 50, .5)
[1] 0.2399438
1 - pbinom(27, 50, .5)
[1] 0.2399438

One more style of CI: Notice that the 95% CI from this procedure is $(0.413, 0.700),$ based on
yet another style of confidence interval (Clopper-Pearson CI). [About half a dozen styles are
in common use. For more about this see this Wikipedia page.]
Finally, if I had as many as $x=2792$ heads out of $n=5000$ tosses, then the
P-value would be much smaller than $0.05 = 5\%$ and we would reject $H_0$
at the 5% level of significance. And the 95% CI would be  $(0.545, 0.572),$
which does not contain $p = 0.5.$
Just from your question, I am not sure what aspects of the coin-testing problem
interest you most. If there are parts of the above that don't make sense to
you, or other topics of interest that I omitted, then you can ask questions
in comments or post a more specific Question. Also, the margin of this question has links to 'Related' Q&A's on this site that may be of interest.
Addendum per comment. Argument for the Wald CI (which was originally intended only for very large $n.)$
Begin with $E(\hat p) = p; SE = SD(\hat p) = \sqrt{\frac{p(1-p)}{n}}.$ Assuming $\hat p$ is normal, standardize to get $Z = \frac{\hat p - p}{SE} \stackrel{aprx}{\sim}\mathsf{Norm}(0,1),$ so that $P(-1.96 < Z < 1.96)\approx 0.95.$ Use algebra to transform the event, obtaining
$P(\hat p - 1.96SE < p < \hat p + 1.96SE)\approx 0.95.$ Then provided we knew SE, an approximate 95% CI for $p$ would be of the form
$(\hat p - 1.96SE,\, \hat p + 1.96SE).$ However $SE = \sqrt{\frac{p(1-p)}{n}}$ contains unknown $p.$ For sufficiently large $n,$ one has $\hat p \approx p,$ so approximate $SE$ as $\widehat{SE} = \sqrt{\frac{\hat p(1-\hat p)}{n}}.$ Then pretend $\left(\hat p - 1.96\widehat{SE},\, \hat p + 1.96\widehat{SE}\right)$ is 95% CI for $p.$
Comment: thanks for the write-up. I am still struggling to understand the derivation of the [Binomial proportion confidence interval](https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval). Is there a straightforward way to arrive at $\hat{p} \pm z \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$?
Comment: Tried to show that in Comment beneath your Q, Will try again in Addendum. A difficulty is that there are many approximations.
Comment: even a reference resource is good, ideally one with the derivation and doesnt require reading an entire textbook.
Comment: Addendum is finished. Deleting comment after your Question.
Comment: I learnt from this answer. The Wikipedia link says "[t]he Normal approximation interval and its presentation in textbooks has been heavily criticised, with many statisticians advocating that it be not used. The principal problems are overshoot (bounds exceed [0, 1]), zero-width intervals at ${\hat {p}} = 0$ and $1$ (falsely implying certainty), and overall inconsistency with significance testing." So the other methods you mention may be better for actual data anaysis.
Comment: Yes, Others are better. Agresti-Coull, using $\hat p = (x+2)/(n+4)$ is noticeably better. Exact method implemented in R's `binom.test` gives wider intervals (sometimes unnecessarily wide), but true 95% coverage. See my Wikipedia link for discussion/formulas of several types. // Simple to use in R and surprisingly good is to take quantiles $.025$ and $.975$ of $\mathsf{Beta}(x+.5, n-x+.5),$ which gives $(0.422, 0.690)$ for my example with $x=28, n=50.$ [Jeffries CI.]
Comment: @BruceET, Thanks for the explanation. I'm still conceptually confused - as I get lost conceptually in your derivation. I wrote a new question about it to hopefully make that more clear. [which is here](https://stats.stackexchange.com/questions/530789/struggling-to-understand-what-a-confidence-interval-is-in-problem-involving-m)


Program To Check If A Number Is Present In An Array

Question: I wrote the below C code to check if a number is present in an array whose elements are input by the user. But weirdly it's skipping the the third printf statement, directly taking the input and printing Enter the number you wish to look for after taking that input. What is causing this? Included input and output box below code.
CODE:
 #include <stdio.h>
 #include <stdlib.h>

  void main() {
    int arr[30], size, i, num, flag=0;

    printf("Enter size of array. \n");
    scanf("%d",&size);

    printf("Enter %d array elements one by one. \n",size);
    for (i=0; i<size; i++) {
        scanf("%d \n",&arr[i]);
    }

    printf("Enter the number you wish to look for. \n");
    scanf("%d",&num);

    for(i=0;i<size;i++) {
        if (num == arr[i]) {
            flag++;
        }
    }

    if (flag>0) {
        printf("The number %d is present in the array.",num);
    } else {
        printf("The number %d is not present in the array.",num);
    }
 }

INPUT/OUTPUT:
Enter size of array.
5
Enter 5 array elements one by one.
1
2
3
4
5
5
Enter the number you wish to look for.
The number 5 is present in the array.

You can see that Enter the number you wish to look for. should come before 5, but it is not so.

Solved
Simply fixed by removing \n from scanf.
Comment: You are using scanf() without checking. Be more paranoid. Read the spec and look for things you did not expect. If you find some, prove that they do not occur in your situation.
Comment: I'm sorry, i don't get your comment. The program works, the only thing is that it's taking the input first and then printing that line. Could you point out why that's happening? Sorry again.
Comment: Well both upvoted answers talk about your use of scanf(). The problem is behaviour you did not expect. I recommended to read the spec and look for things you did not expect. You might find this interesting for future applications: http://sekrit.de/webdocs/c/beginners-guide-away-from-scanf.html
Answer: The space in the input format string "%d \n" tells the input system to

consume... all available consecutive whitespace characters from the input

(described here)
So when you enter your last number 5, the system now tries to consume all whitespace characters. To do that, it waits for additional input, until it's not a whitespace. So, paradoxically or not, to consume spaces, the system has to read a non-space, which is the second 5 you input.
To fix this behavior, you can tell your system to input only a number, without consuming whitespace:
    scanf("%d",&arr[i]);

However, this will leave the whitespace in the buffer, which may interfere with later input. To discard the whitespace, you can use various techniques, described e.g. here.
In my opinion, the most correct technique (however, maybe the most cryptic one) is
    scanf("%d%*[^\n]%*c",&arr[i]);

%d - read the number
%*[^\n] - read a string, terminated by a newline byte; discard it and don't store it anywhere
%*c - read a byte (which is a newline byte); discard it and don't store it anywhere

BTW in your format string "%d \n", there are two whitespaces: a regular space and an end-of-line. They both tell scanf to consume all whitespaces in input. The effect is exactly the same as with one space "%d " or with one end-of-line "%d\n", so this particular format string may be highly confusing to whoever reads your code (including yourself).
Answer: In scanf, a space character already represents any whitespace. So in your "%d \n" the function already processes the new line right after the last digit, but then you force it to wait for another newline.
This causes the program to wait for yet another line. After it's input, the program continues and asks for the number to search, and at that point the input was already entered.
Just use only one space in scanf, it will already work for the newline, ideally before the digit itself so that you don't need one extra line to complete the operation:
scanf(" %d", arr + i);

Comment: Thank you! Cleared it up for me.


How to use data in fields from one node to create a new node of different content type?

Question: I have a content type called "Submit a photo" where anonymous users can upload a photo alongside with their name, email and short description. And I have another content type where an admin uploads images and publishes them as "Looks", and Looks are more complicated content type, with image tags, categories ...
Than I have a view which lists all of the submitted images so that an editor can pick and create new node "Look" using existing fields. But I'm looking for a way to somehow pre populate fields in that new content type with fields of "Submit a photo".
So editors can review all the images that are submitted by anonymous users. If they like one, they click "Add new Look" or something like that, which opens a new "Add Look" form using that image.
Should I look at nodereference or something like that?
Answer: Have a look at the Node_convert module. From it's project page:

adds a menu tab "Convert" on the node view page, i.e. node/1/convert, which gives the ability to convert the node from its current node type to another node type. The module provides the means to transfer all the field values of the current node type into fields on the destination node type, or discard them.

I'm using Node_convert already for long time, works great. It allows for implementing a lifecycle (or workflow?) to start from a node that is only just something like an idea (or suggestion, question, issue, abstract, etc). So that would be like your "Submit a photo" content type. When the node's content evolves (read: the more effort/details added to it), I convert its content type to finished products like (eg) a recipe, a redbook (already from my Drupal 6 days a few years ago). In your case it'd be the "Look" content type.
By combining it with an access module such as Content Access (my preference) or Taxonomy Access Control you can make it more restrictive to give access to it. E.g. public (anybody) to start from, then "authenticated", and in the end only for roles such as "subscribers", "students", etc.
Comment: Thanks for this, i tried it and looks fine, but im having issues making it work with my custom made dashboard, i tried with VBO but it cant convert straight away, instead it gives me offer to pick themplate but i have only one...
Answer: Another way is using the Rules module like so:

Event: after saving new node
Conditions : content type
Action : create new node of "look", fetch images from "sumbit a photo" ,  update the created node.


set onClick or onItemSelected for item in list view

Question: I have a requirement that, in my list view I am having an image icon, a text view and a button.
when I click on any one of these it should move me to next screen. I tried implementing these methods, but no success.
Can anybody help me?
Note:My class does not extends ListActivity rather only Activity.
Here is my code....
<code>ListView listView = (ListView)findViewById(R.id.listView1);
listView.setOnItemSelectedListener(new OnItemSelectedListener() {
            @Override
            public void onItemSelected(AdapterView<?> arg0, View arg1,
                    int arg2, long arg3) {
                Intent in2 = new Intent(City.this, TourDescription.class);
                startActivity(in2);

            }
            @Override
            public void onNothingSelected(AdapterView<?> arg0) {

            }

        });
</code>
Answer: You need to set OnItemClickListener, not OnItemSelectedListener


How to tell whether a predictor in a survival model useful?

Question: For the question below (e), I think I need to compare the median survival times for two groups so that I can decide whether AG is a useful predictor. To do so, I  wrote the code below, but do not know how to do that in R:

zaman  <- c(65,156,100,134,16,108,121,4,39,143,56,26,22,1,1,5,65,
56,65,17,7,16,22,3,4,2,3,8,4,3,30,4,43)
test <- c(rep(1,17),rep(0,16))
WBC <- c(2.3,0.75,4.3,2.6,6,10.5,10,17,5.4,7,9.4,32,35,100,
100,52,100,4.4,3,4,1.5,9,5.3,10,19,27,28,31,26,21,79,100,100)
status <- c(rep(1,33))
data <- data.frame(zaman,test,WBC)

# test positive
surv3 <- Surv(zaman[test==1], status[test==1])
fit3 <- survreg( surv3 ~ log(WBC[test==1]),dist="w")

#test negative
surv4 <- Surv(zaman[test==0], status[test==0])
fit4 <- survreg(surv4 ~ log(WBC[test==0]),dist="w")

Any idea how to do that in R?
Answer: The predict method for survreg objects is helpful here:
predict(fit3, newdata=subset(data,test==1), type='quantile', p=0.5, se=TRUE)

predict(fit4, newdata=subset(data,test==0), type='quantile', p=0.5, se=TRUE)

But if you really want to see if test produces different survival curves, it is probably easiest to use the log-rank test to compare two survival curves.
km.test <- survdiff(Surv(zaman,status)~test,data=data)
km.test

Call:
survdiff(formula = Surv(zaman, status) ~ test, data = data)

        N Observed Expected (O-E)^2/E (O-E)^2/V
test=0 16       16      9.3      4.83      8.45
test=1 17       17     23.7      1.90      8.45

 Chisq= 8.4  on 1 degrees of freedom, p= 0.00365

km.fit <- survfit(Surv(zaman,status)~test,data=data)
km.fit

Call: survfit(formula = Surv(zaman, status) ~ test, data = data)

       records n.max n.start events median 0.95LCL 0.95UCL
test=0      16    16      16     16    7.5       4      43
test=1      17    17      17     17   56.0      22     121

plot(km.fit)

Comment: In this case how do we tell whether AG is useful?
Comment: What is AG in this case?
Comment: Sorry, I forgot to add the question. Added now


Erro de importação ao enviar e-mail simples com python

Question: Estou aprendendo a enviar e-mail em python e estou me deparando com vários problemas. Um deles é já na importação do módulo smtplib. Meu código é o seguinte:
from smtplib import SMTP
smtp=SMTP('smtp.live.com',587)
smtp.starttls()
smtp.login('firstname.lastname@example.com','senha')
msg='ola mundo'
smtp.sendmail('firstname.lastname@example.com',['email@example.com'],msg)
smtp.quit()

Este script gera o seguinte erro:
Traceback (most recent call last):
  File "C:\Users\Benedito\Desktop\email.py", line 1, in <module>
    from smtplib import SMTP
  File "C:\Users\Benedito\AppData\Local\Programs\Python\Python35- 32\lib\smtplib.py", line 47, in <module>
    import email.utils
  File "C:\Users\Benedito\Desktop\email.py", line 1, in <module>
    from smtplib import SMTP
ImportError: cannot import name 'SMTP'

Pelo que eu entendo, o erro trata o módulo smtplib como se ele não tivesse o método SMTP nele. Porém, quando executo a importação direto na linha de comando do Python, esse erro de importação não ocorre. Porque está dando erro quando executo um script e não está dando erro quando executo escrevendo linha por linha na linha de comando do python?
Estou no Windows 8 e uso Python 3.5.2
Answer: Troque o nome do teu arquivo de email.py para enviaremail.py (ou qualquer coisa), ele parece estar conflitando com o import email.utils (usado pela lib SMTP)
E depois tente:
python enviaremail.py

É sempre bom evitar estes nomes para não conflitar com as classes e namespaces nativos


How to pass a parameter in such scenario

Question: Say tp0 is the lowest hierarchy, all real code omitted just leave a parameter
module tp0;
parameter num=0;
endmodule

Where it is instanced
module tp
(
input id
);
//what I want to do is below code, but this will not pass compile since parameter need this is a constant!

tp0 #(id) tp_i();

endmodule

id is an input but this is actually be assign as fixed value in tp's instance.
module tb
.....
tp tp_0
(
 .id(0)
);

tp tp_1
(
 .id(1)
);
....
endmodule

How to resolve this if the pass of id is such a way and need to be passed into a parameterized module tp0?
Thanks

Answer: Verilog is a RTL language, which means that you are describing hardware that will be eventually synthesized.

When you use a parametrized instance, the parameter is analyzed at compile time and that affects how in synthesis the code will be transformed. The parameter NUM (capital letters suggested for params) usually refers to number of bits or instances, thus it needs to be known to calculate for example how many flip-flops should be instanced.

An input is something that dynamically changes while the application is running (ASIC or FPGA), thus it cannot be defined at compile time by connecting it to a parameter.

In your case, the best solution is to add a parameter also to tp module and then connect it to tp0.NUM, but it is not clear from this example what is your target or if this code is for simulation only (various quick alternatives) or if needs to be synthesized.

An elaborate alternative might be that in tp module you instance two submodules with each value of the parameter (0 and 1) and then mux the outputs depending on the value of id.
module tb();
    tp tp0 (.id('0));
    tp tp1 (.id('1));
endmodule

module tp(
    input logic id
);
    logic out0, out1, out_f;

    tp_sub #(0) tp_0(.out(out0));
    tp_sub #(1) tp_1(.out(out1));

    always_comb begin
        if (id) begin
            out_f = out1;
            //or other things
        end
        else begin
            out_f = out0;
            //or other things
        end
    end

endmodule


Is it worth playing this game of St. Petersburg paradox?

Question: A gambler offers you the following deal. You have to keep tossing a fair coin until you get a heads, at which point you stop and collect your winnings: if it happens after n throws, the gambler will give you $2^n$ cents.
However, in order to play the game you first have to agree to pay an entrance fee, but the amount you need to pay is not known to either of you. Instead, you each ask a stranger (before the game starts) to secretly write a random amount of money on a piece of paper and seal it in an envelope.
Once the game has ended, and having paid you your winnings, the gambler will open both envelopes and you then have to pay him the average of the two amounts that the strangers have written (rounded up to the nearest penny).
From a financial point of view, is it worth playing the game?
$$
P(getting \space heads)=P(getting\space tails)=1/2
$$
If we obtained head after say, 5 throw, then it will be $2^5$ cents
This question is seriously baffling me. There's little to no numerical information that I can go on besides the 1st paragraph.
Though from overall looks of it, it seems that this is a Markov chain process on a countable state space that uses a random walk.
I'm completely lost. Could you help me out.
Comment: The lack of information about the details of the entrance fee seems to make this unsolvable.
Comment: I'm confused, we don't know the distribution of the entrance fee. Hence, we can't say anything about the Expected payment?
Answer: With respect to the game, this is exactly the St. Petersburg paradox.
reference:
http://en.wikipedia.org/wiki/St._Petersburg_paradox
The game has infinite expected value but is worth only a finite amount in practice if that makes sense?


Program Activity Home Button to go back to previous Activity

Question: After trying so hard to make Home button appear in my ActionBar it finally showed up, but when clicking it nothing happens. What method or OnClick listener should I add to make it functional.
I admit this question has been asked frequently and after going through most of them, am still not satisfied so please moderators take it easy on me, please.
Anyway your help is greatly appreciated.
Here is the code

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_second);

        ActionBar actionBar = getSupportActionBar();
        if (actionBar != null) {
            actionBar.setDisplayHomeAsUpEnabled(true);

        }
//Method to listen on the home button tap and go back to previous activity
}

Answer: The button in the upper left of the Action bar is the Up button. If you've already added the android:parentActivityName name to your AndroidManifest, then the system does the correct thing already - navigate you back to that parent activity as per the documentation on the onSupportNavigateUp() method, which is the method that is triggered when that button is clicked.
Comment: Actually lemme experiment on that now, Thanks for thre Answer @ianhanniballlake
Comment: You hit the jackpot with your reply buddy, keep up with the good work. Thanks in Advance. You deserve that upvote...


Mudar conteúdo de um form sem precisar abrir outro no c#

Question: Eu gostaria de saber, como faço pra mudar o conteúdo do Form sem precisar abrir outro, quando eu clico em Criptografia por exemplo, o conteúdo muda sem precisar abrir outro Form. Como faço isso?
Comment: você pode usar um tabControl, ou abrir os outros forms dentro desse principal
Answer: Manipulando UserControl(s) você consegue criar um objeto herdado de Control. Assim, crie seu formulário neste UserControl e faça que funcione como um Form.
Após isso, crie um Panel cliente que irá exibir as páginas, e declare os controles como no exemplo abaixo:
public UserControlHome ucHome = new UserControlHome();
public UserControlCripto ucCripto = new UserControlCripto();

E crie este método para alterar o Control que está no Panel cliente conteudo:
public void AlterarConteudo(ref Control controle) {
    // considere 'conteudo' o Panel que irá ter os controles dentro
    conteudo.Controls.Clear(); // - remove os controles dentro do Panel
    { // altera propriedades do controle que irá ser colocado
        controle.Dock = DockStyle.Fill;
    }
    // adiciona o controle ao cliente
    conteudo.Controls.Add(controle);
}

E ao clicar no botão de Criptografia, chame o método com a expressão:
AlterarConteudo(ref ucCripto);

E para o botão Home:
AlterarConteudo(ref ucHome);

Lembre-se de implementar um método para salvar informações nos controles ao serem descartados.


How to click on checkbox filter with selenium Python?

Question: I'm trying to click on a checkbox filter on a website with selenium python. This is a part of its HTML code linked to one of the checkbox options.

        <input type="checkbox" name="" value="Jabodetabek">

        Jabodetabek

I tried following, but it didn't work.
import time
from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get('https://shopee.co.id/search?keyword=baju%20laki-laki')

time.sleep(5)
driver.find_element(By.XPATH, "//input[@value='JABODETABEK']").click()

I read the answers to similar questions, mostly they suggest using 'id' and 'name' to find the element. However, in this input tag, there are only 'type' and 'value'.
Answer: Here is one way to select that checkbox:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
[...]
wait = WebDriverWait(driver, 25)
[..]
wait.until(EC.element_to_be_clickable((By.XPATH, '//span[text()="Jabodetabek"]'))).click()
print('clicked on Jabodetabek')

Selenium documentation can be found here.
Comment: This works very well! Thank you very much!
I didn't know that must use the 'span', because I followed the documentation here: https://selenium-python.readthedocs.io/locating-elements.html
Answer: @value for the checkbox in the xpath is incorrect. The selenium web locator techniques are case sensitive. So make sure to use exact values while identifying objects. Changing your xpath to below would fix the issue.
driver.find_element(By.XPATH, "//input[@value='Jabodetabek']").click()

Comment: still not working. This is the error that I got:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable


How can I assess performance of a semi-supervised learning method?

Question: I'm working with a semi-supervised learning task, where I only have positive and unlabelled data (PU learning).
I've tested a few algorithms and would like to assess their performance.
For supervised learning, I normally use:

Misclassification error
Accuracy
ROC curves

to assess performance.
However, since I only have positive and unlabelled data in my training and validation sets I'm not sure these metrics make sense.
What metrics can I use to properly assess the performance of a semi-supervised learning method?
Answer: We have addressed this problem in Assessing binary classifiers using only positive and unlabeled data. Specifically, we show how to compute strict bounds on any metric based on contingency tables (accuracy, precision, ROC/PR curves, ...). Our work was accepted by all reviewers at this year's NIPS conference, but then rejected by the editor for lack of significance (go figure). We will submit it to the upcoming KDD.
Our approach is based on the reasonable assumption that known positives are sampled completely at random from all positives. If you can't rely on this assumption, any form of performance evaluation is infeasible. Additionally, we require an estimate of the fraction of positives in the unlabeled set, which you can often acquire via domain knowledge or by explicitly obtaining labels for a small, random subset of the unlabeled set.
Answer: Here's a sideways thinking idea: You have some positive labels and you can estimate the natural grouping of data using unsupervised learning. Try to measure the overlap between the known information and the way the data groups together, use the overlap as a ground truth measure.
So, perform unsupervised learning, see how the labeled data corresponds to the clusters. If your're in luck, then the labels will correlate to only one of the clusters or to outliers (which might turn out to be clusters given more data).
Outcome A - disjoint groups of data
Let's say that you have 10 labels from 100 unlabeled examples and after clustering it turns out that the 10 labels belong to a cluster with 20 data points. This is the happy case and you can now label all 20 with 1 and everything else as 0. Problem solved, just use AUC.
Outcome B - more than 2 groups, fuzzy clusters
What if this is not the case? What about the other groups?
If not, let's say you have 9 labels in cluster with 20 and 1 in one of the other clusters (hopefully the only other one). Repeat multiple times and count how many times did a label 'land' in a certain group. Compute the mutual information between the labeled data (positive examples) $X$ and the other groups $Y$ over multiple clusterings.
$$
I(X;Y) = \sum_{y \in Y} \sum_{x \in X}
                 p(x,y) \log{ \left(\frac{p(x,y)}{p(x)\,p(y)}
                              \right) }, \,\!
$$
So, with $K=3$ clusters you will finally have $I_k(X;Y)$ for each group.
Assume that these values are the ground truth (target values) when you evaluate your final model.
This is based on the assumption that your prediction will also have the positive labels (now, more of them) distributed in a certain way in the unsupervised grouping of data.


Favored methods for overcoming selection bias (special attention to healthcare fields)?

Question: I am frequently measuring the effect of behavioral health treatment interventions on outcomes of interest. However, comparing the relative efficacy of different types of treatment is tricky - more intensive interventions may indicate clients with more severe issues, whose outcomes will be more frequently negative anyway. RCTs are generally unethical in the areas I'm studying.
What are your favorite approaches for addressing this sort of selection bias - where level of need determines intervention type, but level of need also plays a role in determining outcome? What are your critiques of common approaches?
Some of the approaches I've explored (note that when I say covariates indicating severity, there is no magic variable I have that shows "this treatment is what a person needs"; this is all based in theory and observed/available data, but are just likely indicators that have to be taken into account with other factors):
Multivariate models including covariates for severity of condition (e.g., primary diagnosis, history of emergency services, etc.);
Propensity-score matching, with the same factors predicting treatment type and outcome (but can only examine one treatment type at a time);
Latent class analysis (built off covariates that may indicate severity);
Only running models on tightly-defined groups (e.g., only on people with one specific type of diagnosis).
Comment: Latent class analysis classifies respondents based on the indicator variables you select. When you say "built off covariates that may indicate severity", it sounds like you might be thinking of forming classes like low severity, medium severity, high severity. I don't think this gains you anything in terms of causal inference. You might as well just enter the variables into a regression. Thus, I respectfully disagree that this post should get the latent class tag. However, can you show articles that are a counterexample?
Comment: Point of information: running models on tightly-defined groups could be a subset of stratified analysis.
Answer: There is no single magic bullet to estimate treatment effects in the context of confounding (note: "selection bias" can mean something else). There is also no agreement in the field about the best method, and the best method for a given problem may differ from the best method for another (and neither will be immediately apparent). My understanding is that some of the best performing methods are the "multiply robust" methods, which include targeted minimum loss-based estimation (TMLE) and Bayesian additive regression trees (BART) with a BART propensity score. I describe these methods with references in this post.
These methods are multiply robust in that there are numerous forms of misspecification that they are robust to (i.e., they will give you an unbiased or low-error estimate even if you get some things wrong about the relationships among variables). The more standard doubly robust methods are those that give you two chances to correctly specify a model in order to arrive at an unbiased estimate of the treatment effect. Augmented inverse probability weighting (AIPW) with parametric outcome and propensity score models is one such example; if either the outcome model or propensity score model is correct, the effect estimate is unbiased. Multiply robust methods are robust to these misspecifications but also to misspecifications of the functional form of the relationship between the covariates and the treatment or outcome. They gain this property through flexible nonparameteric modeling of these relationships. Such methods are highly preferred because they require fewer untestable assumptions to get the right answer, in contrast to propensity score matching or regression, which require strong assumptions about functional form.
I would check out the best performers of the annual Atlantic Causal Inference Conference competition, as these represent the cutting edge of causal inference methods and are demonstrated to perform well in a variety of conditions. TMLE and BART were two of the best performers, and are both accessible and easy to use.
I'm not going to write off the other methods you mention, but they do require many assumptions that cannot easily be assessed or they have been demonstrated to perform poorly in a number of contexts. They are still the standards in the health sciences, but that is slowly changing as the advanced methods become better studied and more accessible.
Comment: I don't recall (I was there though!), but I do remember BART was high up there. They tend to be slow to release the results, but you can attempt to ask the organizers of the conference.
Comment: Do you know which methods performed best at the 2019 ACIC Data Challenge?
Answer: I don't disagree with Noah's answer. I have never heard of Bayesian Additive Regression Trees or with targeted minimum-loss estimation, so I can't comment on those specifically. Methods involving weighting and propensity scores are well-accepted in epidemiological circles.
You should also consider instrumental variable and regression discontinuity approaches.
In the former, there are sometimes cases where you have a variable that influences the probability of receiving treatment but not the outcome. For example, McClellan et al (1994) noted that some hospitals treated acute myocardial infarction (the fancy term for heart attack) more intensively than others (i.e. they were more prone to use cardiac catheterization and revascularization, as opposed to what I guess is medical management). They used the differential distance as their instrument: for each patient, what was the distance to the nearest high-catheterization hospital minus the distance to the nearest low-catheterization hospital?
IVs are not without untestable assumptions - just like all observational methods, really. Also, they answer a subtly different question than a randomized trial would. Quoting McClellan et al

Thus, IV methods are ideally suited to address the question, "What would be the effect of reducing the use of invasive procedures after AMI in the elderly by, for example, one fourth?" They do not address the question, "What would be
  the expected effect of treating a particular patient aggressively rather than with noninvasive therapies alone?" For clinical decisions involving treatment of individual patients, the answer to the latter question is more useful. For policy decisions affecting the treatment of patient populations, the answer to the former is likely to be more useful.

Alternatively, sometimes you have cases where treatment is given to people at or above a cutoff point on some sort of score, and withheld from everyone below the cutoff. You can exploit that in a regression discontinuity design. You'd compare people just above the cutoff to people just below it. The inherent assumption is that because all scores are measured with error, the people just above the cutoff and the people just below it are pretty similar. This does also require that the participants did not game the score - which is an assumption that you should really think about. In some ways, being above versus below the score is an instrument.
The issue is that it may be difficult to find an instrument, and that the treatments you're interested in may be not be assigned according to some score.
Comment: I encourage you to look into BART and TMLE! They hold a lot of promise and are slowly gaining traction in the methodological world.


Mudar o arquivo ( jpeg , gif ou swf ) de acordo com tamanho de tela?

Question: Tenho um Site em PHP e nele possui um campo para anúncios publicitários ( Onde eu posso colar códigos )
Como é um site que se redimensiona eu não faço ideia de como fazer um código HTML ou JAVA usar diferentes imagens para o mesmo espaço publicitário ( vários arquivos para diferentes dimensões - Pc ou Mobile  )
Por exemplo :
No Campo X eu poder usar ( colar )  um código
que me permita usar dois arquivos
para diferentes dimensões
por exemplo:
Dimensão 230x90 px   para Celular
Dimensão 978x90 px   para PC
alguém pode me ajudar ?
ficarei muito grato kkkkkkkkk
encontrei vários códigos, mas nao consegui
Comment: acho que essa idéia vai funcionar para vc  https://pt.stackoverflow.com/questions/57828/mudar-banner-de-acordo-com-tamanho-de-tela
Comment: Eu vi aqui, porém nao consegui entender onde direciono os arquivos nesses códigos.
Comment: Há uma infinidade de tamanhos de tela, não seria melhor já detectar se é mobile ou não e inserir o arquivo correspondente?
Comment: isso !!!  pq eu quero falar td para ser entendido e acabo é atrapalhando.

busco um código que me ajude nisso, que ao acessar no celular ele busque o arquivo x

e ao acessar no pc ele busque o arquivo y
Comment: seria mais ou menos isso
mas não imagino como chamar ou dizer qual arquivo ele deve chamar quando tiver nessa dimensão.

@media(max-width: 500px){
    .texto{
        font-size: 10px;
        color: black;
    }
   }

ignore o código, apenas um exemplo
Comment: uai, a pergunta é mudar ( jpeg , gif ou swf ) de acordo com tamanho de tela, e no comentário trata-se de um CSS
Comment: sim mudar, independente de como fazer.
Answer: Você pode usar imagens como background em divs e usar media-query para ajustar, um exemplo se tiver duas imagens:

Ajuste o @media (max-width: 420px) para o que achar adequado para a tela dos celulares, já o swf não teria como trocar, mas geralmente os próprios swf podem ser expanciveis, então bastaria adicionar ele width: 100%; para ajustar ao elemento DIV que recebe o width: e height da media-query por exemplo, no entanto devo lhe alertar que Google Chrome e Firefox já bloqueiam por padrão arquivos swf (creio que tenha como ativar ou ainda tem o click to play, mas é algo que realmente vai só funcionar dependendo do usuário, flash já não é um bom caminho faz tempo)
Comment: vou testar aqui, mas vi que vc entendeu o que busco.
Comment: OBRIGADOOOOOOOOOOOOOOOOOOOOOOO !!
Comment: @jhonatansantos por nada, e seja bem vindo a nossa comunidade.
Answer: Veja se atende, vai detectar se é mobile e chamar um arquivo e caso contrário chama outro arquivo
function isMobile(){
    var a = navigator.userAgent||navigator.vendor||window.opera;
    if(/android|avantgo|blackberry|blazer|compal|elaine|fennec|hiptop|ip(hone|od|ad)|iris|kindle|lge |maemo|midp|mmp|mobile|o2|opera m(ob|in)i|palm( os)?|p(ixi|re)\/|plucker|pocket|psp|smartphone|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce; (iemobile|ppc)|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|e\-|e\/|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(di|rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|xda(\-|2|g)|yas\-|your|zeto|zte\-/i.test(a))
        return true;
    else
        return false;
}

if(isMobile()){
    document.write('<link rel="stylesheet" type="text/css" href="mobile.css" />');
}else{
    document.write('<link rel="stylesheet" type="text/css" href="desktop.css" />');
}

.No    document.write pode colocar o que desejar, uma imagem, chamada de arquivo css, etc... ou até mesmo um HTML inteiro.
document.write('');

Comment: vou testar e te aviso , mas desde ja agradeço pela ajuda man
Comment: obrigado cara deu certo


How to get ID from master table then record to details table

Question: I'm junior in MVC programming I have this code in controller:
<code>     [HttpPost]
        public JsonResult SaveOrder(PonudiVM O)
        {
            bool status = false;
            if (ModelState.IsValid)
            {
                try
                {
                    using (leskaEntities dc = new leskaEntities())
                {

                    ponudi ponuda1 = new ponudi { klID = O.klient, datum = O.datum, description = O.description };
                    foreach (var i in O.DetalnaPonuda)
                    {
                        ponuda1.ponudaDet.Add(i);

                    }

                    dc.ponudi.Add(ponuda1);
                    dc.SaveChanges();

                    //int last_id = db.ponudi.Max(item => item.pID);
                    status = true;
                }
            }
                catch (BllException bllEx)
            {
                ModelState.AddModelError("", bllEx.Message);
            }
            catch (Exception ex)
            {
                //todo da se zapise vo LOG file
            }
        }
            else
            {
                status = false;
            }
            return new JsonResult { Data = new { status = status } };
        }
</code>
What I need to add in the code, to take new ID from master table then record ID to details table.
My data I send by Javascript and jQuery.
Comment: Welcome to Stack Overflow!  You can read up on how to [ask] a question and create a [mcve]. That makes it easier for us to help you.
Answer: To get the last id simply do:
<code>dc.ponudi.Add(ponuda1);
dc.SaveChanges();

int last_id = ponuda1.id;
</code>
If the detail table is ponudaDet, by the association, the ID should be already saved.


Copy a directory from remote resource using Chef

Question: I want to copy a directory from a remote respository. To copy a single setup/file I am using remote_file resource. but this doesn't work if you want to copy a complete directory. I also checked chef's remote_directory resource, but that can be used only if the directory is in the cookbook itself. Any ideas as how to achieve the same.
Answer: Create an archive for the files and transfer the archive using remote_file.
Comment: yea, I too thought of that solution. My client is on a windows 2008r2 system, and there's no native way to archieve a directory in windows.
Answer: Since there is no portable way of listing the remote files to download (or is there?), a more specific approach has to be taken, such as running a script or remote_file resource, or using the 'artifact' cookbook to download and extract an archive.


Reading CSV input file by field name instead of position in Pentaho Data Integration

Question: I have a CSV file with four fields:
FIELD_A, FIELD_B, FIELD_C, FIELD_D
1, 2, 3, 4
5, 6, 7, 8

I also created a transformation in PDI that has a Read CSV Input file step and a following step that inserts the values in a database table.
This works fine, however I may get other combinations in the input file (without notice), such as:
FIELD_A, FIELD_B, FIELD_D
1, 2, 4
5, 6, 8

I was assuming that PDI was going to be intelligent enough to map the fields by name, and send to the next step FIELD_A, FIELD_B, and so on, depending on the position of the column header. Instead, it takes the first three fields in the file and maps them to the first three fields in the step. As a result, I get in FIELD_C the values of FIELD_D.
Is there a way to have the transformation map by field name instead of position? I cannot change the transformation every time an input field is received.
Answer: There have many way to do this. Use 'Select Values' step before insert to DB is the simple way. You can find the sample transformation from  Here. Where in the 'CSV file input', you can first call file with 4 field and after that you can call file with 3 field for testing.
Let me know if its ok or not.
Comment: Thanks for your answer. My issue is in the 'CSV file input' step, what do you mean by "call file with 4 fields" ? Note that in the real scenario I have 200 fields.
Comment: Yes, at first you need to prepare 'CSV file input' step with the file (which has maximum fields 200) and complete the 'Select Values' step. After that, if you found the csv file with less then 200 field than others field value will return as null in the 'select values' step automatically. And matching filed name will return expected values.
Comment: or you can share your transformation you have try so far.
Comment: [this is what I have so far](https://1drv.ms/u/s!Au4Vkx6gmEf-5Ev34Y2OsoK7J7qV?e=kTA44j), a step that reads the csv file, and a step that writes the fields in the database. The fields in the file may come in any order, and sometimes some fields will be missing. How do I add the 'Selected Values' step?
Comment: you can find the updated transformation from https://drive.google.com/drive/folders/1IR7CHy3BlXS-IVUxsBDXehlzHNKcDlyA?usp=sharing        I have tested and found its working.
Comment: Sorry, it didn't work. I have a column named X in position 60 in my Read CSV Input step, and in the CSV file is column 63. The only thing that matches is the column name, not the position. Instead of taking the value from column in position 63 it took the values from position 60.
Comment: I know i can help you. I think there have something messing between our requirement understanding. You can send me a document with your requirement in brief. and also send 2 sample csv file with your problem. [Then if require we will use metadata injection as well]
Answer: You have also the option of injecting metadata to the transformation to read the CSV file and insert in the table.
You first create a transformation to read which columns are present in the CSV file (reading only the first line of the CSV with the column headers) and then injecting that information to the Read CSV step and Output table step in your transformation.
Comment: Hi Ana, assuming I get the list of columns somehow by reading the file, how do I inject it to the Read CSV step?
Comment: I also found [this](https://help.pentaho.com/Documentation/5.2/0R0/0V0/020/020) to create dynamically the transformation reading the list of columns first
Comment: Well, this is the official Pentaho Documentation for the last version (although it's been working the same for past versions, with each version they have added steps and fields supporting metadata injection: https://help.pentaho.com/Documentation/9.1/Products/ETL_metadata_injection
Comment: Also, in your PDI installation you have a samples directory with examples of use of common steps, in the samples/transformations/meta-inject you have exactly an example on how to use metadata injection to read a CSV file.


Prophet trend model

Question: Can someone help me understand how Prophet automatically detects the change points in its piecewise linear trend model?
Please check page 10 of Prophet paper (Forecasting at Scale: https://peerj.com/preprints/3190.pdf)
Thank you so much
Comment: L1 drops out whole changepoints, whereas L2 will set the coefficient to small values.
Comment: its perhaps more complicated than it needs to be but essentially its fitting a series of 'hockey stick' functions with l1 (lasso) regularisation so it will drop out (set coefficient to zero) minor changes.
Comment: and what's the idea behind choosing L1 and not L2?
Answer: With default options, Prophet detects changepoints by first specifying 25 potential changepoints at which the rate/slope of the trend is allowed to change. They are uniformly placed in the first 80% of the time series. It then puts a sparse prior on the magnitudes of the rate changes, so most of them get shrunk to zero. You can also manually specify the locations of potential changepoints. Slope changes will then be allowed only at these points, with the same sparse regularization as before.
This info can be found here, with some guidance on how to set these three parameters.

Prophet trend model


General solution of $AX=B$ for rectangular matrices

Question:
Let $$A = \begin{bmatrix} 1 & 0 & 2 &-1 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 &0 \end{bmatrix}, \qquad B = \begin{bmatrix} \frac{3}{2} &-\frac{3}{2} &0\\
\frac{3}{2} & \frac{3}{2} & 1\\ 0 & 0 & 0 \end{bmatrix}$$ Find the general solution for $AX=B$.

My textbook says we can construct new equation $CX=D$,where
$$C=\left[\begin{matrix}
    1 & 0 & 2 &-1 \\
    0 & 1 & -1 & 0 \\
    0 & 0 & 1 &0 \\
    0 & 0 & 0 & 1\\
    \end{matrix}\right],\qquad D=\left[ \begin{matrix} \frac{3}{2} &-\frac{3}{2} &0\\
\frac{3}{2} & \frac{3}{2} & 1\\
c_1 & c_3 & c_5\\
c_2 & c_4 & c_6\\ \end{matrix}\right]$$
where $c_1,\dots,c_6$ are any constants. Then solve this get
$$X=\left[\begin{matrix}
\frac{3}{2} & -\frac{3}{2} &0\\
\frac{3}{2} & \frac{3}{2} & 1\\
0 & 0 & 0\\
0 & 0& 0
\\
\end{matrix}\right]+\left[ \begin{matrix}-2 & 1\\ 1&0\\1& 0\\0 & 1 \end{matrix}\right]\left[ \begin{matrix}c_1& c_3 &c_5\\
c_2 &c_4 &c_6 \end{matrix}\right]$$
I really can't see why this will work.
Comment: Have you tried Gaussian elimination on augmented matrix $[A B]$?
Answer: In terms of components, we have
$$\sum_{j}A_{ij}X_{jk}=B_{ik}.$$ The matrix $X$ has $4\times 3 =12$ components, while $i$ can take values $1,2,3$ and $k$ can take values $1,2,3$, so it seems like we have $9$ equations for $12$ unknowns and will expect $3$ free constants. Then, we notice that $A_{3,j}=0$ for all values of $j$, so three of our equations give us no information and we expect $3+3=6$ free constants, as in the textbook's answer. The only equations we have with which to constrain $X$ are $\sum_{j}A_{ij}X_{jk}=B_{ik}$ for $i=1,2$ and $k=1,2,3$.
Now, does the equation $CX=D$ encompass all of the above equations? Yes! The equation
$$\sum_j C_{ij}X_{jk}=D_{ik}$$ has the same constraints as $AX=B$ for $i=1,2$ and $k=1,2,3$. The free constants are incorporated into the equation in the final two rows of $D$, which guarantees that the required constraints are untouched (this is not the only possible way of doing things), so any solution of $CX=D$ for any set of constants will yield a matrix $X$ that solves the desired $6$ relations of $AX=B$.


Which DB/DB Engine supports search well?

Question: I'm starting a site which relies heavily on search. While it's probably going to search basic meta data in the beginning, it might grow to something bigger in the future.
So which DB/DB Engine is best in your opinion when it comes to search performance and future scalability?
Appreciate your help
Answer: It depends on what you are searching.
If you doing a lot of text searching then you want more than just a database - you also want a search algorithm. You can find them around the web and they can use several databases as backends.
However, if you only simple text searches then MySQL MyISAM offers full-text searching which I use for small amounts of text (less than a few GB).
Other searches include using keys and indexes which might lead you to PostgreSQL for it's rock solid ACID compliance or MySQL with INNODB.
Answer: What is "search" ? What are you looking for and what kind of queries do you expect?
PostgreSQL is very powerfull, has full text search, btree, hash, gin and gist indexes. You can also configure you own types and operators, everything is there to optimize your searches in the database. It's up to you to use and tweak it for you situation.
PostgreSQL is easy to use with PHP, no problem at all. And it's free, sort of BSD-licence.
Comment: initially it'll search the fields of microposts (body, title, date...etc). Forgot to mention that the "did-you-mean" feature is a must.
Answer: Depending on whatever you mean by "search" any database system might work. (MySQL is a well know and fast RDBMS).
If what you are looking for really is "full text search" then you should take a look into MySQL FULLTEXT indices (only usable with the MyISAM backend, IIRC), Lucene or Xapian.
The Zend Framework (written in PHP) has a ready adapter for lucene, see: http://devzone.zend.com/article/91
Comment: initially it'll search the fields of microposts (body, title, date...etc). Forgot to mention that the "did-you-mean" feature is a must.
Comment: Please read the article I mentioned above. This is exactly what you need.


How do I deal with ties when using rank-based normalizing transforms, e.g. Blom?

Question: I would like to transform heavily skewed data with range (-Inf, Inf) and heavily zero-inflated into a form suitable for using GLMs for significance testing.  Zero-inflation precludes the effective use of standard blom-family transforms (e.g. Blom, Elfving, etc) because they require data not to have ties.  It occurs to me that I can randomly break ties and fit an ensemble of glms on the resulting sampled datasets, and then average the significance result.  Therefore, I have two questions:

Is there any theory on normalizing copula transforms for continuous data with ties?  I can't find anything?
Does this bootstrap-driven correction make sense to folks?  Would it lose statistical power?

Someone has proposed that I run a Wilcoxon test.  That doesn't work for me for two reasons:

I need to run some multivariate fits and
Wilcoxon will heavily extrapolate the significance test by excluding 0s from ranks, which is roughly 2/3 of the measurements (yes, I know, the data sucks, but it's the data)
There's a modified wilcoxon test that addresses this problem but I can't find an implementation (Pratt, J. W. (1959). Remarks on Zeros and Ties in the Wilcoxon Signed Rank Procedures. Journal of the American Statistical Association, 54(287), 655–667. doi:10.1080/01621459.1959.10501526)

Thanks in advance!
Comment: Yes, you can. There are different ways to overcome the tie problem. Look at copula package for some solutions. One way is to use: `copula::pobs(data,tie.method="random")`.
Comment: Copula ia a package in R program. There is another one in Paython.
Comment: I've run a quick simulation, doing blom transform on ranks with ties broken randomly, and it seems that, for pairwise comparisons, p-value on real data is better with spearman correlation than with a pearson run on "my own" blom-transformed data - typically roughly 85th percentile of simulations.  this is not to say multivariate data will behave the same way - stay tuned
Answer:
Note that spearman p-values are typically at 85th-90th quantile of blom simulations.  Still looking into multivariate predictors, which may benefit from gaussianity.  X-scale: lower is better.  See my last comment above for details.


$H_q(X;\mathbb{Z})=0$ when X spherical complex with $H_q(X;F)=0$ for all $q>0$ and for all $F=\mathbb{Q}$ or $F=\mathbb{Z}/p\mathbb{Z}$

Question: Suppose that X is a spherical complex with $H_q(X;F)=0$ for all $q>0$ where $F=\mathbb{Q}$ or $F=\mathbb{Z}/p\mathbb{Z}$, as $p$ runs over all primes. I want to prove $H_q(X;\mathbb{Z})=0$.
I know Singular Homology Theory and some of Singular Cohomology. I know that $H_q(X;\mathbb{Z})$ is a finitely generated $\mathbb Z$-module for every q, since $\mathbb{Z}$ is a Noetherian ring and $X$ is a spherical complex. But I don't know how to continue proving.
I give a definition of a spherical complex: Start with a finite discrete set of points, and successively attach cells, possibly of varying dimensions, but finite in number. The resulting Hausdorff space is called a spherical complex.
Thank you for your help!
Comment: The term I know for what you describe is a "finite CW complex". On a separate note, do you know of the Universal Coefficient Theorem?
Comment: I believe you should use the universal coefficient theorem to show that $H_*(X,\mathbb{Z})$ tensor with the rational and the finite fields are zero. This will imply that  $H_*(X,\mathbb{Z})$ has no torsion-free elements and no elemets of finte order.
Comment: I am reading Algebraic Topology by Greenberg and Harper, and this book gives that definition. It seems that Hatcher and many references call those "finite CW complex", as you said.
Comment: Thank you for your comments! I will try to prove it with the universal coefficient theorem.
Comment: @MikeMiller I saw the latter sentence just now. Actually there is the universal coefficient theorem for singular cohomology, not for singular homology in the book. I found the homology version on the internet and it looks similar to the cohomology version. Is it possible to convert the problem to cohomology version and prove it?
Comment: @S.Ha I put a sketch of a sketch of the second argument below.
Comment: Actually the definition of a CW complex has another condition on top of what the OP is considering, namely, when a $k$-dimensional cell is attached, the image of its $k-1$-sphere boundary must be contained in the union of the cells of dimension $\le k-1$.
Answer: Hint: by the universal coefficient theorem for homology, we have an isomorphism $$H_q(X, F) \cong \left(H_q(X, \mathbb Z) \otimes F\right) \oplus \text{Tor}(H_{i-1}(X,\mathbb Z), A).$$ Necessarily both terms vanish, and the fact that $H_q(X, \mathbb Z)$ is a finitely generated abelian group gives it an explicit form you can easily deal with.
EDIT: here's an approach with the UCT for cohomology. We know that $$H^i(X, F) \cong \text{Ext}_R^1(H_{i-1}(X,R),F) \oplus \text{Hom}_R(H_i(X,R),F)$$
Calculate $H^i(X,F)$ in two ways: first by using $R=F$, and then after seeing that $H^i(X,F)=0$ calculate it using $R=\mathbb Z$; again because $\text{Ext}$ and $\text{Hom}$ are well-behaved and you can explicitly write down what $H_i$ must be (by the classification of finitely generated abelian groups), the theorem you want falls out.
Comment: Thank you very much! So it seems that the core is the UCT.


How to implement parcelable for List in Kotlin

Question: I am trying to make Parcelable of below class
data class Movie(
    @SerializedName("popularity")
    @Expose
    var popularity: Double,

    @SerializedName("vote_count")
    @Expose
    var voteCount: Int,

    @SerializedName("video")
    @Expose
    var video: Boolean,

    @SerializedName("poster_path")
    @Expose
    var posterPath: String?,

    @SerializedName("id")
    @Expose
    var id: Int,

    @SerializedName("adult")
    @Expose
    var adult: Boolean,

    @SerializedName("backdrop_path")
    @Expose
    var backdropPath: String?,

    @SerializedName("original_language")
    @Expose
    var originalLanguage: String?,

    @SerializedName("original_title")
    @Expose
    var originalTitle: String?,

    @SerializedName("genre_ids")
    @Expose
    var genreIds: List<Integer>,

    @SerializedName("title")
    @Expose
    var title: String?,

    @SerializedName("vote_average")
    @Expose
    var voteAverage: Double,

    @SerializedName("overview")
    @Expose
    var overview: String?,

    @SerializedName("release_date")
    @Expose
    var releaseDate: String?
) : Parcelable {
    constructor(parcel: Parcel) : this(
        parcel.readDouble(),
        parcel.readInt(),
        parcel.readByte() != 0.toByte(),
        parcel.readString(),
        parcel.readInt(),
        parcel.readByte() != 0.toByte(),
        parcel.readString(),
        parcel.readString(),
        parcel.readString(),
        TODO("genreIds"),   <-------------- Which is generated by "Android Parcelable code generator" plugin
        parcel.readString(),
        parcel.readDouble(),
        parcel.readString(),
        parcel.readString()
    ) {
    }

    override fun writeToParcel(parcel: Parcel, flags: Int) {
        parcel.writeDouble(popularity)
        parcel.writeInt(voteCount)
        parcel.writeByte(if (video) 1 else 0)
        parcel.writeString(posterPath)
        parcel.writeInt(id)
        parcel.writeByte(if (adult) 1 else 0)
        parcel.writeString(backdropPath)
        parcel.writeString(originalLanguage)
        parcel.writeString(originalTitle)
        parcel.writeString(title)
        parcel.writeDouble(voteAverage)
        parcel.writeString(overview)
        parcel.writeString(releaseDate)
    }

    override fun describeContents(): Int {
        return 0
    }

    companion object CREATOR : Parcelable.Creator<Movie> {
        override fun createFromParcel(parcel: Parcel): Movie {
            return Movie(parcel)
        }

        override fun newArray(size: Int): Array<Movie?> {
            return arrayOfNulls(size)
        }
    }
}

I was getting an error so I have found solution from this website How to implement parcelable for List<Long>
and done like below
parcel.readList("genreIds",null),

but I am still getting an error like

And also by follwing this page
How to parcel List<Int> with kotlin
I have added extension functions like below
fun Parcel.writeIntList(input:List<Int>) {
    writeInt(input.size) // Save number of elements.
    input.forEach(this::writeInt) // Save each element.
}

fun Parcel.createIntList() : List<Int> {
    val size = readInt()
    val output = ArrayList<Int>(size)
    for (i in 0 until size) {
        output.add(readInt())
    }
    return output
}

Hi @Eugen Pechanec first of all thank you so much for your help, as per you suggestion I have done Int instead of Integer but in readArrayList still I am getting an error
Comment: check this answer https://stackoverflow.com/questions/48371013/how-to-parcel-listint-with-kotlin
Comment: Thanks @MahmoudElsakka but it does not help me :(
Comment: Kotlin doesn't have `Integer`. Use `Int`. Here: `var genreIds: List`
Comment: You also forgot to write the list in `writeToParcel`, careful.
Answer: Move createIntList() and writeIntList outside the scope of your Movie class like below:
package com.acs.mtnprodapp.data.local.entity

import android.os.Parcel
import android.os.Parcelable
import com.google.gson.annotations.Expose
import com.google.gson.annotations.SerializedName

data class Movie(
        @SerializedName("popularity")
        @Expose
        var popularity: Double,

        @SerializedName("vote_count")
        @Expose
        var voteCount: Int,

        @SerializedName("video")
        @Expose
        var video: Boolean,

        @SerializedName("poster_path")
        @Expose
        var posterPath: String?,

        @SerializedName("id")
        @Expose
        var id: Int,

        @SerializedName("adult")
        @Expose
        var adult: Boolean,

        @SerializedName("backdrop_path")
        @Expose
        var backdropPath: String?,

        @SerializedName("original_language")
        @Expose
        var originalLanguage: String?,

        @SerializedName("original_title")
        @Expose
        var originalTitle: String?,

        @SerializedName("genre_ids")
        @Expose
        var genreIds: List<Int>,

        @SerializedName("title")
        @Expose
        var title: String?,

        @SerializedName("vote_average")
        @Expose
        var voteAverage: Double,

        @SerializedName("overview")
        @Expose
        var overview: String?,

        @SerializedName("release_date")
        @Expose
        var releaseDate: String?
) : Parcelable {
    constructor(parcel: Parcel) : this(
            parcel.readDouble(),
            parcel.readInt(),
            parcel.readByte() != 0.toByte(),
            parcel.readString(),
            parcel.readInt(),
            parcel.readByte() != 0.toByte(),
            parcel.readString(),
            parcel.readString(),
            parcel.readString(),
            parcel.createIntList(),
    parcel.readString(),
    parcel.readDouble(),
    parcel.readString(),
    parcel.readString()
    ) {
    }

    override fun writeToParcel(parcel: Parcel, flags: Int) {
        parcel.writeDouble(popularity)
        parcel.writeInt(voteCount)
        parcel.writeByte(if (video) 1 else 0)
        parcel.writeString(posterPath)
        parcel.writeInt(id)
        parcel.writeIntList(genreIds)
        parcel.writeByte(if (adult) 1 else 0)
        parcel.writeString(backdropPath)
        parcel.writeString(originalLanguage)
        parcel.writeString(originalTitle)
        parcel.writeString(title)
        parcel.writeDouble(voteAverage)
        parcel.writeString(overview)
        parcel.writeString(releaseDate)
    }

    override fun describeContents(): Int {
        return 0
    }

    companion object CREATOR : Parcelable.Creator<Movie> {
        override fun createFromParcel(parcel: Parcel): Movie {
            return Movie(parcel)
        }

        override fun newArray(size: Int): Array<Movie?> {
            return arrayOfNulls(size)
        }
    }

    //move from here
}

//to here
fun Parcel.writeIntList(input:List<Int>) {
    writeInt(input.size) // Save number of elements.
    input.forEach(this::writeInt) // Save each element.
}

fun Parcel.createIntList() : List<Int> {
    val size = readInt()
    val output = ArrayList<Int>(size)
    for (i in 0 until size) {
        output.add(readInt())
    }
    return output
}

This would make the methods available to other class as well
Comment: I have tried that first but in that I am getting an error as i have shown in last picture " Can not access createIntList....."
Comment: You need to move the methods out of the class like so: `class Movie{} // place them here`. Don't worry methods without classes are allowed in Kotlin. You can read more here https://stackoverflow.com/questions/49013996/why-is-possible-to-write-a-function-outside-a-class-in-kotlin
Answer: With the help of @ Eugen Pechanec & @Networks I found solution which is below,
 data class Movie(
    @SerializedName("popularity")
    @Expose
    var popularity: Double,

    @SerializedName("vote_count")
    @Expose
    var voteCount: Int,

    @SerializedName("video")
    @Expose
    var video: Boolean,

    @SerializedName("poster_path")
    @Expose
    var posterPath: String?,

    @SerializedName("id")
    @Expose
    var id: Int,

    @SerializedName("adult")
    @Expose
    var adult: Boolean,

    @SerializedName("backdrop_path")
    @Expose
    var backdropPath: String?,

    @SerializedName("original_language")
    @Expose
    var originalLanguage: String?,

    @SerializedName("original_title")
    @Expose
    var originalTitle: String?,

    @SerializedName("genre_ids")
    @Ignore
    var genreIds: List<Int>,

    @SerializedName("title")
    @Expose
    var title: String?,

    @SerializedName("vote_average")
    @Expose
    var voteAverage: Double,

    @SerializedName("overview")
    @Expose
    var overview: String?,

    @SerializedName("release_date")
    @Expose
    var releaseDate: String?
) : Parcelable {
    constructor(parcel: Parcel) : this(
        parcel.readDouble(),
        parcel.readInt(),
        parcel.readByte() != 0.toByte(),
        parcel.readString(),
        parcel.readInt(),
        parcel.readByte() != 0.toByte(),
        parcel.readString(),
        parcel.readString(),
        parcel.readString(),
        parcel.createIntList(),  //<-------------------------- change here
        parcel.readString(),
        parcel.readDouble(),
        parcel.readString(),
        parcel.readString()
    )
    override fun writeToParcel(parcel: Parcel, flags: Int) {
        parcel.writeDouble(popularity)
        parcel.writeInt(voteCount)
        parcel.writeByte(if (video) 1 else 0)
        parcel.writeString(posterPath)
        parcel.writeInt(id)
        parcel.writeByte(if (adult) 1 else 0)
        parcel.writeString(backdropPath)
        parcel.writeString(originalLanguage)
        parcel.writeString(originalTitle)
        parcel.writeIntList(genreIds) //<-------------------------- change here
        parcel.writeString(title)
        parcel.writeDouble(voteAverage)
        parcel.writeString(overview)
        parcel.writeString(releaseDate)
    }

    override fun describeContents(): Int {
        return 0
    }

    companion object CREATOR : Parcelable.Creator<Movie> {
        override fun createFromParcel(parcel: Parcel): Movie {
            return Movie(parcel)
        }

        override fun newArray(size: Int): Array<Movie?> {
            return arrayOfNulls(size)
        }

    }
}

            fun Parcel.createIntList(): List<Int> {
                val size = readInt()
                val output = ArrayList<Int>(size)
                for (i in 0 until size) {
                    output.add(readInt())
                }
                return output
            }

            fun Parcel.writeIntList(input:List<Int>) {
                writeInt(input.size) // Save number of elements.
                return input.forEach(this::writeInt) // Save each element.
            }

Answer: as my first comment you could convert your list of integers to List of int
as
class Movie(
...
    var genreIds: List<Int>,
..)

so , in case you want to save movie object as Json  i recommend using Serializable
it doesn't required any implementation method
data class Movie(

    var originalLanguage: String?,

    var originalTitle: String?,

    var genreIds: List<Integer>,

    var title: String?,

    var voteAverage: Double,

    var overview: String?,

    var releaseDate: String?
) : Serializable

and you could save it as following
        val sharedPref: SharedPreferences.Editor = context.getSharedPreferences(Constants.PREF_KEY, Context.MODE_PRIVATE).edit()
        val json = gson.toJson(YOUR_OBJECT)
        sharedPref.putString(Constants.MOVIE, json)
        sharedPref.commit()

and get it
        val sharedPref: SharedPreferences = context.getSharedPreferences(Constants.PREF_KEY, Context.MODE_PRIVATE)
        val json = sharedPref.getString(Constants.MOVIE, null)
        if(json != null) {
            return gson.fromJson(json, Movie::class.java)
        } else {
            return null
        }

Comment: Serialization dumps all its implicit baggage onto you. Every time you compile the app the serialization process may change (the compiler is in charge). You're not required to do anything when you add or remove fields but it will alter the byte representation of the class and fail on deserialization. On the other hand you have `Parcelable` which forces you to take care of it. And is also more performant because it doesn't use introspection - because you had to wrote the serialization code yourself. Final note, `Serializable` has nothing to do with Gson, at all.


Is it possible to wrap the fields of a view inside a  and ?

Question: UPDATE: Colin is right but I'd like to go over what I did to accomplish this.
I added a views-view.tpl.php template and where it said
<?php print $rows; ?>

I wrapped it like this

    <?php print $rows; ?>

I also wanted to prevent each row from being wrapped in a view so I also added a views-view-unformatted.tpl.php template and replaced
<div<?php if ($classes_array[$id]) { print ' class="' . $classes_array[$id] .'"';  } ?>>
    <?php print $row; ?>

with just
<?php print $row; ?>

I'm trying to have a view that starts with a

then contains all the content of my view like so

image

image

image

and then ends with

However I'm having a lot of trouble. Header and footer does not work because it starts and closes it in it's own div. I tried hiding all the contents and doing a global text field like so

[image]

However that gives each item it's own nav and closes it right after each one.
Is wrapping the entire comments in the  and  tags possible in drupal?
Answer: You will need to add a View template to your theme, take a look at Views template files to give you an idea of how to accomplish this
Comment: That worked. But is there a way to make it so it only applies to a specific type of view? I don't want this to affect all of my views.
Comment: Whichever file you have overridden add the view name prefixed with -- before the .tpl.php so forexample if your view name was nav and you overrode `views-view.tpl.php` your file would be `views-view--nav.tpl.php`


Modelform prepopulate foreign key

Question: Trying to set a foreign key per modelform instance i ve got the following models:
#models.py
Column(models.Model):
     name = models.CharField()

Input(models.Model):
    column = ForeignKey(Column)
    text = models.CharField()

In forms.py, how do I set the ForeignKey for the Modelform instance?
#forms.py
class InputForm(forms.ModelForm):

    def __init__(self, column_parameter):
        super(InputForm, self).__init__()
        self.fields['column'].value = calc_column(column_parameter) #this of course does not set the value

    class Meta():
        model = Input

Answer: You should probably not set it in init method. If you have to do it, then set it in save() method.
def save(self, force_insert=False, force_update=False, commit=True):
    m = super(InputForm, self).save(commit=False)
    m.column = calc_column(column_parameter)
    if commit:
        m.save()
    return m

But in case you absolutely have to set this value, then i suggest you do it with initial data:
InputForm(initial = {'column':calc_column(column_parameter)})

Modelform prepopulate foreign key


Simulation study - what error variances to use from empirical data? MS error of an ANOVA or the variance of their residuals?

Question: I want to do a simulation study in R and I have already some empirical data, that gives me a hint about the variance parameters to set. But what should I use for the error variance?
Here's an example of what I mean:
> a <- aov(terms(yield ~ block + N * P + K, keep.order=TRUE), npk)
> anova(a)
Analysis of Variance Table

Response: yield
          Df Sum Sq Mean Sq F value   Pr(>F)
block      5 343.29  68.659  4.3911 0.012954 *
N          1 189.28 189.282 12.1055 0.003684 **
P          1   8.40   8.402  0.5373 0.475637
N:P        1  21.28  21.282  1.3611 0.262841
K          1  95.20  95.202  6.0886 0.027114 *
Residuals 14 218.90  15.636
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> var(residuals(a))
[1] 9.517536

So would I use 15.6 or 9.5 as my empirical error variance?
Comment: Two suggestions: (1) Begin with simple one-way ANOVA where details may be simpler. You can simulate the design with known $\sigma^2,$ several times, and see what best estimates that known value. (2) Think carefully about your more complex design, MS(Resid) has DF(Resid), far fewer than total nr $T$ of obs because several other params est'd. Nr of Residuals  is also $T,$ so `var(residuals(a))` must have donom $T-1.$ Resids can't be indep. obs because constrained to add to $1.$ Which is unbiased for $\sigma^2?$ Look at your 'a': Does it give you "residual std error"? Is square of that MS(Resid)?
Answer: "Halfway" between the two suggestions of my (long, possibly cryptic) Comment,
here is a simulated two-way ANOVA, in which MS(Resid) is a reasonably good
estimate of the $\sigma^2$ used to simulate the model.
set.seed(1234)
e = rnorm(24, 0, 2)
sd(e)
[1] 1.87691

a = rep(1:2, each=12)
b = rep(rep(1:3, each=4), 2)
x = 2*a + 3*b + e
A = as.factor(a); B = as.factor(b)

out = aov(x~A+B); out
Call:
   aov(formula = x ~ A + B)

Terms:
                        A         B Residuals
Sum of Squares   49.67123 132.10586  73.73120
Deg. of Freedom         1         2        20

Residual standard error: 1.920042                  # compare 1.87691
Estimated effects may be unbalanced

anova(out)
Analysis of Variance Table

Response: x
          Df  Sum Sq Mean Sq F value    Pr(>F)
A          1  49.671  49.671  13.474  0.001517 **
B          2 132.106  66.053  17.917 3.478e-05 ***
Residuals 20  73.731   3.687                       # compare 1.92^2 = 3.6864
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Summary: $\sigma^2 = 2^2 = 4.$ The sample SD of simulated errors is 1.88,
Resid SE = 1.92, MS(Resid) = 3.69.
95% CI for $\sigma^2$ based on simulated e of my model is $(2.13, 6.93),$
which includes $\sigma^2 = 4.$
23*var(e)/qchisq(c(.975,.025), 23)
[1] 2.127980 6.931925

95% CI for $\sigma^2$ based on SS(Resid) is $(2.16, 7.69),$ which covers $\sigma^2=4.$
73.731/qchisq(c(.975,.025), 20)
[1] 2.157795 7.687698

Note: Here is information about the 24 residuals.
r = residuals(out)
length(r); mean(r); var(r)
[1] 24
[1] -1.168002e-16
[1] 3.205704

Answer: Summing up what BruceET inspired me to, I found a rather simple answer to my question:
Simulate with parameters that were received in the same way than you analyze later the simulated data.
This is actually self explanatory and I could have found it out earlier but maybe somebody will have the same question one day.
This means: If you analyze empirical values with a linear model (e.g. ANOVA) and you then use the results as your parameters for your simulations (e.g. the residual MS for the error variance), make sure you also analyze the simulated data in the same way, i.e. analyze it with an ANOVA and look at the residual MS in this example.


General solution of $\tan (2x)\tan (x)=1$

Question: For the question, $\tan (2x) \tan x=1$, I divided it by $\tan x$, and got the solution as $\frac{(2n+1)\pi}{6}$.
$\tan 2x= \cot x= \tan\left(\frac{\pi}{2}-x\right)$. So, $2x=n\pi+ \frac{\pi}{2}-x$. So, $3x= \frac{(2n+1)\pi}{2}$
But the book solved using the formula of $\tan (2x)$, and got the solution as  $\frac{(6n \pm 1)\pi}{6}$. I can see that my solution has odd multiples of $\pi/2$, which should be discarded, but I thought of it only after checking the solution. Also, in a way, it suggests that we can't solve these questions in different ways because that way we might get extra solutions. So, how to ensure which method to follow?
Comment: The usual way. tan2x=cotx=tan(pi/2-x). so, 2x=n.pi+pi/2-x. So, 3x=(2n+1)pi/2.
Comment: "but I thought of it only after checking the solution" You should always check whether the solutions you reach actually solve the original equation.
Comment: Dividing by $\tan x$ assumes $\tan x$ is defined, and going from the equation $\tan(2x)=\tan(\pi/2-x)$ to $2x=n\pi +(\pi/2-x)$ assumes the equation is true when both sides are undefined (which is not the case). You can solve these questions in different ways, it's just that tracking assumption-making and discarding extraneous solutions must be considered as part of the way.
Comment: @runway44- I thought from the given equation, it was clear that tan x is non-zero. So, we could divide by it.
Comment: This [tutorial](https://math.meta.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) explains how to typeset mathematics on this site.
Comment: @N.F.Taussig- Thanks for the link.
Comment: @runway44 - Just reread your comment. Understood your point now. Thanks.
Answer: I have used a different approach to solve this and tried to keep everything generalized without any predefined assumptions.
$$\tan (2\theta)\tan (\theta)=1  $$
$$ \Longrightarrow \frac{2\tan^2(\theta)}{1-\tan^2(\theta)}  -1 = 0$$
$$ \Longrightarrow \frac{3\tan^2(\theta) -1}{1-\tan^2(\theta)}   = 0$$
Solving for $3\tan^2(\theta) -1 = 0 $ we get $\theta = n\pi  \pm  \frac{\pi}{6}$.
However if $\frac{1}{1-\tan^2(\theta)} = 0 $ then $\tan(\theta)$ must be undefined which surely won't satisfy our original solution.
Thus to conclude the solution is $\theta = n\pi  \pm  \frac{\pi}{6}$.
Answer: Your way is better, I think:
The domain gives $\cos2x\neq0$,$\cos{x}\neq0$ and $\sin{x}\neq0$ and by your idea we obtain:
$$\tan2x=\cot{x}$$ or
$$\frac{\sin2x\sin{x}-\cos2x\cos{x}}{\cos2x\sin{x}}=0$$ or
$$\cos3x=0$$ or
$$x=\frac{\pi}{6}+\frac{\pi}{3}k,$$ where $k\in\mathbb Z$.
Now, delete from these roots, roots of $\sin{x}=0$ (which is impossible), $\cos2x=0$ (which is impossible again) and $\cos{x}=0.$
We can get an answer by the following way.
From $\cos3x=0$ on the interval $(0,2\pi)$ we got the following numbers:
$$\left\{\frac{\pi}{6},\frac{\pi}{2},\frac{5\pi}{6},\frac{7\pi}{6},\frac{3\pi}{2},\frac{11\pi}{6}\right\}.$$
$\frac{\pi}{2}$ and $\frac{3\pi}{2}$ we need to delete and since
$$\frac{11\pi}{6}=\pi+\frac{5\pi}{6}$$ and
$$\frac{7\pi}{6}=\pi+\frac{\pi}{6},$$ we got the following series:
$$x=\frac{\pi}{6}+\pi k$$ and
$$x=\frac{5\pi}{6}+\pi k$$ or $$\left\{\frac{(\pm1+6)\pi}{6}|k\in\mathbb Z\right\}$$
Comment: You are welcome!
Comment: The initial equation doesn't require eliminating the solutions with $\sin x=0$ (there are none, though).
Comment: @Bernard See please better the starting post. After dividing of the both sides by $\tan{x}$ we obtain $\cot{x}$ and $\sin{x}\neq0$.
Comment: Yes, but this transformation is not necessary to obtain $\cos 3x=0$.
Comment: @Bernard I tried to restore the Ramit's solution. In any case, with $\cos3x$ it seems very smooth, I think.
Comment: Subtracting $(2m+1)pi/4$ from the obtained solution, I get $(2(2k-3m)-1)pi/12$, which is not a very neat presentation. On the other hand, the book's solution gave the answer without having to subtract anything. I have two questions: a) how to obtain a neat solution from our method; b) is there a way we could pick a method in which we don't have to subtract anything from the obtained solution? Thanks.
Comment: @Ramit I added something. See now.
Comment: Very nice presentation. Thanks.
Answer: $$\tan 2x \tan x=1.~~~(1)$$ $$\Rightarrow \tan 2x = \cot x ~\mbox{if} \tan x \ne 0, \pi/2~~~(2)$$ Therefore, $x \ne k \pi~~~(3)$ and $x \ne (m+1/2)\pi~~~(4).$
From (2), we get $$\tan 2x= \cot x= \tan(\pi/2-x) \Rightarrow 2x =\frac{\pi}{2}-x+n\pi\Rightarrow x= \frac{\pi}{6}+\frac{n\pi}{3}.~~~~(3)$$ These solutions cannot satisfy (3) but they do satisfy (4) when $n=3m+1$ So the complete solution of (1) is given by $$x=\frac{\pi}{6}+\frac{n\pi}{3}, n \in I ~\mbox{but}~ n \ne 3m+1, ~\mbox{where}~ m \in I$$
Comment: Can you please elaborate how we got 3m+1?
Comment: @Ramit After getting $\pi/6+n\pi/3$ set it $\ne (m+1/2)\pi$, you get $n \ne 3m+1$Similarly one should  also set it equal to $k\pi$ but then one gets a condition $n=3k-1/2$ which anyway cannot be satisfied by two integers. These two things are required to because we are dividing by eq. (1)  $\tanx$ so it should not be zero or infinity.
Answer: Writing the equation as
$$1=\tan 2x\tan x=\frac{\sin 2x\sin x}{\cos 2x\cos x}\iff\cos 2x\cos x-\sin 2x\sin x=\cos 3x =0$$
with a domain of validity defined by the conditions $\cos x,\cos 2x\ne 0$, i.e.
$$x\not\equiv \tfrac\pi2 \mod\pi,\quad x\not\equiv \tfrac\pi 4\mod\tfrac\pi 2,$$
we obtain at once the solutions
$$3x\equiv \frac \pi2\mod \pi\iff x\equiv \frac\pi6\mod\frac\pi 3.$$
None of these solutions are congruent to $\frac\pi4\bmod\frac\pi2$, but a part of them are congruent to $\frac\pi2\bmod \pi$. Actually, geometrically the solutions are represented by the vertices of a regular hexagon and the solutions to eliminate are represented by the north and south poles of the trigonometric circle. There remains two series of solutions:
$$x\equiv\pm\frac\pi 6\mod \pi.$$
Comment: Oh! yes, of course. Fixed it. Thank you for pointing it!
Comment: You have written 3x not equal to pi/2. I think you don't mean it.
Comment: I didn't understand how you reached the final solution but perhaps it's because I have no idea of geometrical interpretation.
Comment: You simply have to plot the solutions on the trigonometric circle to obtain the hexagon and remove the north and south poles. There remains 4 vertices, which are symmetric by pairs w.r.t. the centre. Is it clear?
Comment: I see now. In spirit, it is similar to what Michael has done. Thanks.
Answer: You got certain solutions, but you must check whether or not the functions that you are dealing with are actually defined at those points. And $\tan x$ doesn't exist when $x$ is of the form $\frac\pi2+k\pi$ ($k\in\mathbb Z$).
Comment: But tanx is non-zero.
Comment: Sorry for the silly answer. What do you think now?
Comment: Thanks, but I was wondering if there is a way which could give us concrete answer without having to counter-check it with the given equation.
Answer: By your method,
$$\tan(2x)=\tan\left(\frac\pi2-x\right)\iff 2x=\frac\pi2-x+k\pi\iff x=\frac{1+2k}{6}\pi$$ and every step is justified.
But the domain of the LHS excludes $$x=\frac{1+2k}{2}\pi$$ and these solutions must be rejected.
By the method of the book,
$$\frac{2\tan^2x}{1-\tan^2x}=1$$ then
$$2\tan^2x=1-\tan^2x$$ and $$\tan^2x=\dfrac13$$ is only allowed if $\tan^2x\ne1$ and this must be mentioned in the resolution.
Comment: Hi. I just dropped a comment on  Michael's answer. I have the same query here.
Comment: @Ramit: there is nothing "not neat" here. The coefficients in the numerator are $1,5,7,11,13,17,\cdots$ and they can't be expressed as a pure arithmetic sequence.
Comment: by using the formula of tan2x, we are getting a neat solution.
Comment: @Ramit: not neater, unless you have a special definition of neat.


Windows Server 2008 R2 allows many local RDS connections, but only one remote

Question: I'm trying to set up an instance of Server 2008R2 with RDS. I have a 20 User and 20 Device CAL that I can choose from, but either one is not enabling me to have multiple remote connections. I can connect via localhost as much as I want, but remotely, I only get one. I have the firewall temporarily disabled, I have the network adapter configured in RDS to allow unlimited connections, and the policy for multiple sign-ons of the same user is allowed.
Comment: You're question doesn't make sense. The `unlimited` option is only available if you have the RDS role installed. Also, what do you mean you can connect via localhost as mush as you want? How are you connecting via localhost? Can you post a screenshot of the Roles you have installed and of the RDP protocol settings showing your unlimited option?
Comment: I have the RDS role installed. If I use Remote Desktop Connection, I can connect locally via localhost as much as I want, for as many sessions as I want. Trying to do this remotely does not work. I will post a screenshot shortly.
Comment: I still don't understand what you mean by connecting to localhost. Do you mean that you are using RDP to connect to the server from itself?
Comment: Yes. I am able to do so with user (non-admin) credentials, for multiple sessions, but if I try to RDP from outside the server itself, it only allows one connection.
Comment: outside of the server, as in from a computer on the same LAN, or is this a hosted server and you are trying to RDP into it from outside of its LAN?
Comment: The server is in the same LAN. I can successfully make a single RDP connection from my workstation to the server, but cannot make another connection afterwards. Within the server, however, I can continue to create sessions.
Comment: @joeqwerty: the screenshots you wanted are added.
Comment: @d_r_w What does your event viewer has to say?
Comment: It mentioned a need to add NetworkService to the Terminal Server License group. I did that, rebooted, still no dice.
Answer: I assume that you try to log in with the same user all the time. Even on RDS, this is by default restricted.
I assume you can logon with a different user.
If you want to use the same user, use this procedure:
http://www.petenetlive.com/KB/Article/0000471.htm

On the server, click Start and in the search/run box type tsconfig.msc{enter}. Locate "Restrict each user to a single session"
  Right click > Properties.
Remove the tick from "Restrict each user to a single session" > Apply > OK.
Answer: You need to setup and configure terminal services for multiple remote desktop connections. By default, you only get 2.
Check this link to read up on what you want to do.
http://technet.microsoft.com/en-us/library/cc770759.aspx
Comment: The Remote Desktop Session Host role is definitely installed and configured, though.
Answer: I think your problem is in licensing. Have you set up TS Licensing services? Have you configured RDS to use this licensing server? It licence type correct?


SQL: is there a query that returns records only when there is an increment of +1 in a field?

Question: sample_table
Hello. I am looking for a query that will only return the case where there was any increase in the 'numoflaptop' field. I would like to identify the family that has acquired any additional laptop.
The end result should have only 2 records
'family1, purchase, 3' and 'family4, purchase, 3'
family2 is not included in the result since it was a decrease of 1 and family3 is not included since there was no change in numoflaptop field.
Thanks in advance.
I am using Teradata
Comment: What should happen if someone is buying two laptops at the same time?
Comment: Do the ones you want, and only the ones you want, all have `reason = 'purchase'`?
Comment: In theory it should, but in the data, the increase doesn't always coincide with a purchase. So I'm hoping to be able to do it based on just an increment of +1.
Comment: I should have clarified, I am OK with any increase, could be 1, 2, 3. I just don't want to see no change, or a decrease.
Answer: Use LAG to see the previous row's value:
select customer, visit_date, reason, numoflaptop
from
(
  select
    customer, visit_date, reason, numoflaptop,
    lag(numoflaptop) over (partition by customer order by visit_date) as prev
  from mytable
) where numoflaptop > prev
order by customer, visit_date;

Answer: Use windowing functions to calculate change in laptops and to identify most recent visit.  Then use them as filters.
Select customer, visit_date
From (
    Select customer, visit_date,
       numoflaptop - ifnull(lag(numoflaptop) Over (Group by customer Order By visit_date),0) as laptopchanges,
       Row_number() Over (Group By customer Order By visit_date desc) as latest visit
    )
Where latest_visit=1 and laptopchanges=1

You did not tell us which data base manager you are using so the ifnull function might be named differently in yours.  For example, in Oracle that same function is called nvl.
Comment: i'm sorry, i'm using teradata
Comment: `IFNULL` is a function propriatary  to some DBMS. The standard SQL function supported by most DBMS including Teradata is `COALESCE`.


How is the double stepback legal?

Question: James Harden popularized the double stepback.
Sometimes, it's called a travel.

Sometimes, it's not.

How is the double stepback legal?
Answer: The double stepback is legal when used with a zero step, also referred to as a gather step.
More details from the NBA below, but a player may take two steps after gathering the dribble. Depending on when the ref determines a gather has taken place, it may or may not be a travel.
For the sake of the question, let's assume Steph Curry's first double stepback attempt was correctly called as a travel. This would mean three steps took place after the gather step had been determined (as annotated below).

The NBA defines a "gather" as:

For a player who receives a pass or gains possession of a loose ball, the gather is defined as the point where the player gains enough
control of the ball to hold it, change hands, pass, shoot, or cradle
it against his body.

For a player who is in control of the ball while dribbling, the gather is defined as the point where a player does any one of the
following:

Puts two hands on the ball, or otherwise permits the ball to come to rest, while he is in control of it;
Puts a hand under the ball and brings it to a pause; or
Otherwise gains enough control of the ball to hold it, change hands, pass, shoot, or cradle it against his body.

This is relevant as the following is incorporated into NBA traveling rules like so:

The gather will be expressly incorporated into the traveling rule to clarify how many steps a player may take after he receives the ball while progressing or completes his dribble:

A player who gathers the ball while progressing may (a) take two steps
in coming to a stop, passing or shooting the ball...

A player who
gathers the ball while dribbling may take two steps in coming to a
stop, passing or shooting the ball.
Answer: In the end it's all up to the referee watching the play.  Many illegal actions are uncalled in basketball as the referee misses them or chooses not to call them.
NBA rules state the zero step as:

The zero step is the first movement a player makes with their foot on
the ground while receiving or gaining control of the basketball. This
rule allows players to make their first and second steps after
receiving the ball, before finishing their play. The zero step only
applies to players that are moving while receiving the ball.

Thus, James Harden commonly uses this rule to his advantage.
Comment: I think you should put that in your answer, then it will make more sense.  I assume zero step is landing from being in the air while the ball is being received?
Comment: I think this would be a better answer if you were clear on whether the double stepback is *legal* or *illegal* - your answer seems to flow both ways.
Comment: A double stepback is legal if used with a zero step, otherwise is termed as illegal,but ultimately its left to the ref umpiring the play, to signal if its a travel or not.

How is the double stepback legal?


Select amount with Paypal donation button

Question: I try to create donate via PayPal button and add custom select with amount.
I tried to add something like this:
Select the amount you wish to donate:
<input type="hidden" name="currency_code" value="EUR" />
<select id="amount" name="amount">
  <option value="25.00">€25.00</option>
  <option value="35.00">€35.00</option>
  <option value="50.00">€50.00</option>
  <option value="75.00">€75.00</option>
  <option value="100.00">€100.00</option>
</select>

but PayPal simply doesn't recieve selected value.
What I'm missing?
What custom variables should I put into button generator form?
Answer: This code works for me:
<form action="https://www.paypal.com/cgi-bin/webscr" method="post">

<input type="hidden" name="business" value="email@example.com">

<input type="hidden" name="cmd" value="_donations">

<input type="hidden" name="item_name" value="Donation">
<input type="hidden" name="item_number" value="Donation">
<select name="amount"><option value="10.00">€10.00</option><option value="25.00">€25.00</option></select>
<input type="hidden" name="currency_code" value="EUR">

<input type="image" name="submit" border="0" src="https://www.paypal.com/en_US/i/btn/btn_donate_LG.gif" alt="PayPal - The safer, easier way to pay online">
</form>

Reference
Comment: Thanks. It works perfectly. I guess error was caused by input "hosted_button_id"