Analyzing Pfizer Vaccine Efficacy: Testing a Claim about 2 Proportions

Question: I teach Introduction to Statistics and would like to show my students how to analyze the Pfizer Phase 3 Vaccine results.  Testing a claim about 2 proportions is straightforward with the Pfizer data:
Given:

Group (1) is the placebo group;
Group (2) is the vaccine group

Data:

https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-conclude-phase-3-study-covid-19-vaccine [A], and

https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-11/C4591001_Clinical_Protocol_Nov2020.pdf [B]

The number of people who got the 2nd dose of vaccine is 41,135 [A].  Assuming that people dropped out equally from the placebo and vaccine group, those groups were created at a 1:1 ratio [B].  So we have approximately 20,567 people in each group.  Using the number of people who got covid from each group published in [A] we have:
Placebo Group:   $x_1 = 162$,   $n_1 = 20,567$
Vaccine Group (BNT162b2):  $x_2 = 8$,  $n_2 = 20,567$
Claim:  $p_1 > p_2 \rightarrow p_1 - p_2 > 0$
$H_0: p_1 - p_2 \le 0$
$H_1: p1 - p2 > 0$
Test Statistic: $z = 11.8  \rightarrow \text{P-Value} < 0.0001$
Thus we have evidence for the claim that $p_1 > p_2$ meaning that we have evidence for the claim that the vaccine lowered the covid rate versus the control group.
So far, so good.  But here's where things break down.  The manufacturer's claim that the vaccine is shown to be 95% effective.  If I use the 162 as the expected number of cases in general, then if the vaccine is 95% effective, the vaccine group should have less than 5% of the expected number of cases.  5% of 162 is 8.1 cases expected in 20567.
Control Group:  $x_1 = 8.1, n_1 = 20,567$
Experimental Group:  $x_2 = 8, n_2 = 20,567$
Claim:  $p_1 > p_2  \rightarrow p_1 - p_2 > 0$
$H0: p_1 - p_2 \le 0$
$H1: p_1 - p_2 > 0$
Test Statistic: $0.0249 \rightarrow  \text{P-Value} = 0.4901.$
This is clearly a fail meaning that with a P-Value of 0.4901, I have NOT shown that the vaccine group has less than 5% of the expected number of cases.  Why is this analysis flawed?  This seems related to (What does 94.5% effective mean?), but even reading this citation, its unclear to me why my hypothesis test is wrong.  Again, an explanation appropriate for an INTRODUCTION to statistics class, please.

UPDATE:  If I redo the analysis with an expectation of 16.2 cases instead of 8.1, then I get a p-value of 0.0477.  This suggests that I can claim with a 5% significance level that the vaccine is 90% effective.  Is the issue just that they can do a more detailed/sophisticated analysis to get to 95% effective or is my analysis just flawed and I've gotten lucky?
Comment: I entered those values into an online [risk ratio calculation tool](https://www.medcalc.org/calc/relative_risk.php) and it gives a 95% confidence interval for effectiveness as 90% to 97.5%.  That page has some details about the confidence interval calculation. It's a little complicated for an introductory class.
Comment: Can you help me to understand why my analysis failed?  What makes the techniques I am using not-applicable to this situation?
Comment: What are you doing in this second computation?  It is not clear how your computation relates to the hypothesis $H_0: p_1=p_2$ how did you compute the statistic 0.0249 and based on what distribution? You seem to be testing the hypothesis $H_0:p_2=8.1/20567$ via a normal approximation of a Poisson distribution? What do you mean that this test fails?
Comment: Aside from that, the test which is performed is not based on a z-test. What they do is condition on the number of cases (by performing the analysis once a certain number is reached) and treat the number of cases from placebo:vaccine groups as binomial distributed. (at least that is what Pfizer described in their protocol) of course you can make it more complex with survival analysis, Bayesian stuff, and computations that take into account age, gender, other background and computations based on time of vaccination and infection. See also https://stats.stackexchange.com/questions/496774
Comment: Interesting. I was approximating binomial as normal which I believe is allowed if $(n)(p)(1-p) >= 10$ for both groups.  In checking this assumption based on your comment, it appears to not hold up, indicating that normal distributions are not appropriate.
Comment: I also added some text to the question to try to better explain my thinking.
Comment: If I treat this as a confidence interval with x=8, n=162, CL=0.95, I get that I am 95% confident that the percentage of people who got COVID and had the vaccine is between 1.6% and 8.2%.  Again, not so far off from the 5% published by Pfizer if all of this hangs together.  Am I looking at this correctly?
Answer: Based on the comments, there are several issues with this analysis.  First, the analysis is based on being able to approximate binomial as normal.  Textbooks I've consulted write this requirement in different ways. Sullivan (Fundamentals of Statistics) says that in addition to being binomial, each of the groups must pass $np(1-p) \ge 10$ for this to be valid.  For the control group this requirement is satisfied.  For the experimental group it is not.    Triola (Elementary Statistics) says the requirement is  $np\ge5$ and $nq\ge 5$ for each of the two groups.  This requirement is met for both groups.
Second, the second analysis also depends on the control group being an estimate of the number of cases expected in the experimental group.  This assumption is inaccurate and could cause the vaccine to appear less effective than it is.
Finally, for the confidence interval, again the textbooks give differing requirements ($np(1-p) \ge 10$ vs $np \ge 5$ and $nq \ge 5$).  Only some of these requirements are met, suggesting we are on the hairy edge of validity with this analysis.  So a crude guess about the proportion who got covid is ($1.6\%$ to $8.2\%$).
What all of the means to me is that we can do some basic analysis, but this has to be taken with a grain of salt because the number of cases is still sufficiently small to not robustly meet the requirements.
Finally, thanks to the people who commented on this posting as they helped me to think more deeply about what was going on


Make aurelia-ui-virtualization work with tables

Question: I'm struggling to make Aurelia virtual-repeat.for work with table and tr elements. Here is my markup of the modified Users page of aurelia skeleton app:

      ${user.avatar_url}

When I scroll the table down, the items below aren't rendered and I see just the empty space all over. What am I missing?
Comment: There seems to be a bug with calculating the height table rows in UI-Virtualization. Does it work if you remove height: 50px? Can you please create an issue for this and I will take a look: http://github.com/aurelia/ui-virtualization/issues/new
Comment: @Gusten Created an issue https://github.com/aurelia/ui-virtualization/issues/43 . Removing "height: 50px" for tr seems to fix virtualization indeed, although I think it did not help when I tried with my original application...
Comment: This saved me a ton of work! thanks.
Answer: The issue was fixed in version 0.4.3, so the described scenario is working now


Iterating through 2 lists at a time using while

Question: var itemAmountList = _itemAmountvalue.Split(',');
var listItemDescriptionValues = _itemDescriptionvalue.Split(',');

IEnumerator enum1 = itemAmountList.GetEnumerator();
IEnumerator enum2 = listItemDescriptionValues.GetEnumerator();

DataTable dtCustomInvoiceDetail = new DataTable();
dtCustomInvoiceDetail.Columns.Add("purchaseOrderID");
dtCustomInvoiceDetail.Columns.Add("itemAmount");
dtCustomInvoiceDetail.Columns.Add("itemDescription");

while ((enum1.MoveNext()) && (enum2.MoveNext()))
{
            var description = enum1.Current;
            var amount = enum2.Current;

            DataRow dr = dtCustomInvoiceDetail.NewRow();
            dr["itemDescription"] = description;
            dr["itemAmount"] = amount;
            dr["purchaseOrderID"] = strPoid;
}

Above is the code that I am using to iterate through a list and populate a datatable.
I do not see any rows added to the datatable

Verified the data and looks good
it iterates through the loop the no of times as required

But still do not see any data in the datatable
Comment: You are not adding `DataRow` to `DataTable` inside `While` loop. Add this to the end of the loop:
`dtCustomInvoiceDetail.Rows.Add(dr )`
Comment: Thank You Farhad Jabiyev
Answer: In your code
DataRow dr = dtCustomInvoiceDetail.NewRow();

does not actually add a row to the table, it only creates a table row with the correct columns. In order to add the row you need to call
dtCustomInvoiceDetail.Rows.Add(dr)

at the end
Answer: the problem is that you are not adding the generated row to datatable.
Datatable.NewRow :

You must use the NewRow method to create new DataRow objects with
  the same schema as the DataTable. After creating a DataRow, you
  can add it to the DataRowCollection, through the DataTable
  object's Rows property

you should add the row to your datatable like this
dtCustomInvoiceDetail.Rows.Add(dr);

also note that you can convert the whole enumerator logic to a simple for loop, it has more readability and less overhead
for (int i = 0; i < Math.Min(itemAmountList.Count, listItemDescriptionValues.Count); i++)
{
    var description = itemAmountList[i];
    var amount = listItemDescriptionValues[i];

    DataRow dr = dtCustomInvoiceDetail.NewRow();
    dr["itemDescription"] = description;
    dr["itemAmount"] = amount;
    dr["purchaseOrderID"] = strPoid;
}

also it seems that there is an inconsistency between names and variable assignments. you assign a value from listItemDescriptionValues to amount variable and from itemAmountList to description. Please double check it.
Comment: You could also use LINQ's `Enumerable.Zip` to iterate both sequences simultaneously as well.


How to add ShareThis link field into View

Question: I have a view and I want to add ShareThis(module enabled) link to each result of the view.
How this can be achieve so that any of the result row of the View will be shared on social networking sites.
Comment: Pankaj, I'm adding this comment here to "help" you to get you going with suggesting edits: make sure that the edits you suggest are actually (way) more then just correcting 1 or 2 words, or changing the format a little bit. Instead add module links, correct typos, fix grammar, etc. Otherwise "we" (the reviewers) are assumed to "reject" such "insufficient" edit enhancements. And there is a rule that if you have a balance of "5 more rejected edits as compared to accepted edits", you'll be blocked from suggesting edits for 7 days (!!!). Just trying to help you to get going, ok?
Comment: @Pierre.Vriens, Ok I got it.

Thanks for suggestion.
Comment: OK, thank you for confirming, I look forward to reviewing your next series of suggestions. If they'll turn out to actually take my above considerations into account, know that pretty sure "I" will approve all such suggestions that I see showing up in the review queue. And if you want, contact @AdrianCidAlmaguer (+ say hello from me!) about how I made a similar comment to him about some of his wiki edit suggestions (after I "had" to reject like a dozen of them, which I hated but "had" to do). From my reviews of his wiki suggestions it seems like he indeed accepted my suggestion ...
Answer: Please check images for how to use sharethis filed within views

RESULT PAGE:


FileNotFound when tring to save file

Question: I have a strange problem. I'm trying to save a backup of my sqlite database. I've created this code
String DB_PATH;
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            DB_PATH = context.getFilesDir().getAbsolutePath().replace("files", "databases") + File.separator;
        }
        else {
            DB_PATH = context.getFilesDir().getPath() + context.getPackageName() + "/databases/";
        }

    String currentDBPath = "portals.db";
    String backupDBPath = "backupname.db";
    File sd = new File(Environment.DIRECTORY_DOWNLOADS + "/" + backupDBPath);
    File data = new File(DB_PATH + currentDBPath);
    try {
        InputStream in = new FileInputStream(data);
        OutputStream out = new FileOutputStream(sd);

        // Transfer bytes from in to out
        byte[] buf = new byte[1024];
        int len;
        while ((len = in.read(buf)) > 0) {
            out.write(buf, 0, len);
        }
        in.close();
        out.close();

    } catch(Exception e) {
        int test = 1;
        e.printStackTrace();
    }

When I run this I receive FileNotFound exception. Shouldn't it create the file ? I even tried to create a dummy file still same result.
Note that I want to save that on INTERNAL memory beucase I don't have SD card in my device. What is wrong with that code ?
Comment: What is the message of the `FileNotFoundException`? Access is denied? Too many open files? Cannot find the file specified?
Comment: Before I dig deeper, do you have the permission to write to external storage in your manifest? Know that on android, writing to `/sdcard/` is not actually writing to a physical external SDCard, but the user storage that is built into the device. A physical sd cards is located at `/storage/external_SD/` or the likes. To write to either of these, you still need the External Storage permission.
Comment: Rainbolt as I wrote. I get FileNotFound exception.
MattClark as I wrote. I want to save on INTERNAL memory. Not on external beucause I don't SD card in my device. And yes permission for SD are set
Comment: @Fixus That's the type of the exception, not the message. Inside of your catch, write `System.out.println(e.getMessage());` and add the output of that line of code to your question. There are many reasons why a `FileNotFoundException` might be thrown, and I want to know why yours was thrown.
Comment: @Rainbolt point taken :) message is: /Download/backupname.db: open failed: ENOENT (No such file or directory)
Comment: I just did a small test and `FileOutputStream.Write()` will create the file if it doesn't exist, but it won't create all of the directories leading up to it. So my next question is, does the "Download" directory exist? I see that Android devices have a /sdcard/download directory, but you said you don't have an SD card, so you probably can't write to it.
Comment: @Rainbolt Yes this directory exists on my device
Comment: Not an Android expert here, so I might be blowing smoke, but according to [this article](http://developer.android.com/guide/topics/data/data-storage.html#filesInternal) you can only see files that your application created. So even if that directory does exist, if your application didn't create it, then your application can't see it. If you just create it yourself, does that help? (The code to do that would be `data.mkdirs();`).
Answer: I've found the solution.
To solve problem with my code line
File sd = new File(Environment.DIRECTORY_DOWNLOADS + "/" + backupDBPath);

Should look like this
    File sd = new File(Environment.getExternalStorageDirectory() + "/" + Environment.DIRECTORY_DOCUMENTS + "/" + backupDBPath);

Also interesting note for future. The file is saved properly but I could see it only after unplugin and plugin device. So some kind of restart was needed in terms of connection to pc

FileNotFound when tring to save file


Merging multiple sheets into one master sheet

Question: I have one client form for 12 people. All 12 have the same exact form. I am creating a "Master" Client form.  All the sheets are EXACTLY alike.  I need to pull everyone's information onto the new Master sheet.  And the master has to be searchable.
It would be super helpful if it counted every duplicate, deleted the extra ones and kept the one with the oldest date. The date is in Column A. So I can see how many times a client called and the first date they called.
Example.
Answer: You could have a tab and do
<code> =IMPORTRANGE("abcd123abcd123", "sheet1!A1:C10")
 //IMPORTRANGE(spreadsheet_key, range_string)
</code>
and do that until you get all twelve sheet and the on your other sheet
just use
<code>  =ARRAYFORMULA({Sheet1!"A3:A19";Sheet2!"B3:B19"})
</code>
and add whatever filter you want with it., even create a pivot table.


Utilizando recursos de programação funcional para remover uma lista de palavras de outra lista

Question: Estou fazendo a análise de um texto literário e estou tendo o seguinte problema para remover um grupo de palavras do texto. Nâo são palavras repetidas. link_words são palavras que vão atrapalhar uma posterior análise do texto, por isso quero retirá-las. Os conjuntos abaixo são meramente ilustrativos!
link_words = ["for","on","this","will","do","did","have","has","my","the","as","be","he"]

words = ["for","on","this","will","do","did","have","has","my","the","as","be","he","virus"]

A ideia é aplicar map() e filter() de forma a que eu remova da lista words as palavras contidas em link_words. Peguei um conjunto menor para facilitar. A saída deveria ser
["virus"]

Aplica o filtro:
list(filter(lambda x: x in link_words,words))

cuja saída está correta:
["for","on","this","will","do","did","have","has","my","the","as","be","he"]

Tentei fazer o seguinte depois:
seja filtro = filter(lambda x: x in link_words,words)
Não estou conseguindo chegar ao resultado esperado com
 list(map(lambda  x: words.remove(x),filtro))

O que eu errei? Como corrigir o map?
Comment: Isto responde à sua pergunta? [Remover elementos repetidos usando duas listas](https://pt.stackoverflow.com/questions/443192/remover-elementos-repetidos-usando-duas-listas)
Comment: Não precisa de map e filter, a pergunta sugerida acima tem uma solução pro seu caso
Comment: @hkotsubo  eu gostaria de aprender a usá-los! poderia me ajudar a achar o erro?
Comment: @hkotsubo Na verdade nao sao palavras repetidas mas um grupo de palavras que quero retirar do texto para analisá-lo sem elas!
Comment: Não se prenda ao título da pergunta, leia as respostas que tem  lá, pois o problema é o mesmo (remover de uma lista os elementos que estão em outra)
Comment: De qq forma, `map` serve para transformar os elementos da lista em outra coisa, então o `lambda` deve retornar alguma coisa. Só que `remove` não retorna nada, então o resultado vai ser uma lista cheia de `None`. Eu acho que map/filter não são a melhor solução pra esse caso
Comment: @hkotsubo concordo mas eu gostaria de aplicar map e filter!
Comment: @hkotsubo O objetivo é mais aprender a usar map e filter, embora possa nao ser a melhor abordagem!
Answer: map serve para transformar os elementos da lista em outra coisa e no seu caso não tem o que transformar, pois você só quer filtrar.
Como você quer todos os elementos de uma lista que não estão em outra, bastaria fazer:
filtrados = list(filter(lambda x: x not in link_words,words))

Ou seja, os elementos de words que não estão em link_words.
Não tem porque usar map, pois os elementos não são alterados. map só faz sentido se você quer gerar outros valores diferentes da lista original. Por exemplo:
lista = [1, 2, 3]
# gera outra lista, contendo o dobro de cada elemento
dobros = list(map(lambda x: x * 2, lista))
print(dobros) # [2, 4, 6]

Usei map para gerar outra lista, contendo o dobro de cada número da lista original.
Mas no seu caso você não está modificando os valores da lista (no sentido de transformá-los em outros valores), você só está filtrando (escolhendo quais você quer e quais você não quer), então não faz sentido usar map nesse caso.
A outra resposta até dá um exemplo que usa map, mas ela faz coisas absolutamente desnecessárias, como por exemplo criar um array de booleanos, e depois usa index para cada elemento da lista, o que torna o algoritmo bem ineficiente, pois a lista será percorrida várias vezes, já que index faz uma busca linear para encontrar o índice. É uma volta desnecessária somente para "forçar" o uso de um recurso (tanto que ela reconhece no final que existe uma solução melhor, sem precisar de map).
Ou seja, só porque é possível usar map, não quer dizer que você deva fazê-lo. Você não deve forçar o uso de um recurso para resolver um problema, você deve avaliar o problema e ver qual recurso é o mais adequado. E usar map não é necessário nesse caso.
Veja mais sobre isso nesta pergunta, que inclusive dá alguns exemplos em que faz sentido  usar map e filter.
Veja também outras soluções para o seu caso (que nem precisaria de filter também) aqui.
Answer: O Erro
A linha list(map(lambda  x: words.remove(x),filtro)) está retornando
[None, None, None, None, None, None, None, None, None, None, None, None, None]

(que não é a resposta esperada) porque words.remove(x) retorna None. Por exemplo, se você define
words = ["for","on","this","will","do","did","have","has","my","the","as","be","he","virus"]

e usa o método remove, você terá
>> words.remove("for")
>> words
["on","this","will","do","did","have","has","my","the","as","be","he","virus"]

mas se você fizer
>> new_words = words.remove("for")
>> new_words
None

Uma possível resposta usando map e filter
Ao invés disso você pode fazer
words_in_link = list(map(lambda x: x not in link_words, words))
list(filter(lambda x: words_in_link[words.index(x)], words))

que retornará
['virus']

Explicação:
words_in_link será uma lista com Trues se a string da lista words correspondente não estiver em link_words e False se estiver.
>> print(words)
['for', 'on', 'this', 'will', 'do', 'did', 'have', 'has', 'my', 'the', 'as', 'be', 'he', 'virus']
>> print(words_in_link)
[False, False, False, False, False, False, False, False, False, False, False, False, False, True]

então você usa o filter como
list(filter(lambda x: words_in_link[words.index(x)], words))

que vai selecionar apenas as strings de words que não estão em link_words.
Uma resposta melhor, só usando filter
Não há necessidade de usar filter e map para resolver esse problema, você também pode fazer
list(filter(lambda x: x not in link_words, words))

que retornará
['virus']


Help with displaying views with EVA

Question: I am struggling with displaying a view in a node. I have installed EVA module and have created a view of products categorized for that node (Restaurant Type).
The entity displays in the node but I'm having trouble contextualizing the view. How do I dynamically provide the context when the page containing the entity is viewed.
I am normally a PHP programmer and a newbie recently thrown into the Drupal framework. I have been able to print out the criteria (in php template) and display the entity but I am failing in making them play together.
I presume that the "Criteria" I am able to pry out of the template is the long way round solution to my issue, how do I configure my "entity view attachment" to display a contextualized list. my file structure always contains the variable and resolves as "*.com/restaurants/steak_houses" where "restaurants" is the node and "steak_houses" is the context criteria.
I need help in feeding that data to the entity.
Comment: I am set up in the same way as your example to no success. I think I am getting my context wrong, let me try to explain the parameters.

I have an a product list "Broilers" (it is a node) that in categorized according to the restaurant type "Restaurants" (Also a node). I am listing categorized broilers in the restaurants node (Restaurant type examples steak_houses, Institutional, fast_food etc).

when accessing the restaurant types the URL cites *.com/restaurants/steak_houses (the last phrase specifies the context "steak_houses".
Comment: Fields
Content: Thumbnail (Thumbnail)
Content: Title
Content: Broiler Category (Broiler Category)
Content: Description (Description)

Entity content settings
Entity type: Node
Bundles: Restaurant
Arguments: id
Show title: No
Access: Permission | View published content

Contextual filters
Content: Broiler Category

-----------In the Context------------
I am able to set the context statically and get a proper list, unfortunately I am still unable to get the context to set dynamically.
Answer: == Updated. Rewrite the answer, provide step by step screenshot ==
P.S. you also need References, I apologize that didn't mention it.

Create a Content Type called Restaurant, everything by default, save.

Create a Content Type called Product

Add Node Reference field for Product Content Type.

Create an instance of Restaurant content.

Create an instance of Product content, set the node reference field to the resturant just created.

Next, create a new EVA view.

Add filter => Content type => Product

Set Entity to node, Bundle to Restaurant

Add Contextual filter, select the node reference field created.

Provide default value => Content ID from URL

So you overall EVA view set up should be something like this:

Next, an important step: clear the cache, then refresh the restaurant node page to see the change:

You can see the Product One has been attached to the Restaurant.
You will get it. :)
Comment: I am set up in the same way as your example to no success. I think I am getting my context wrong, let me try to explain the parameters.
Comment: Can you show your content types setup and views setup? Screenshot would be great.
Comment: I assume that you have two content types for this. One is Product. One is Resturant. You want to attach a list of products nodes that associated with a resturant node TO that resturant node.   Right?
Comment: Yes, "restaurants" and "broilers". A list of broilers matching the restaurant category (steak_houses, amusement_parks, etc.)
Comment: For some reason the link ref on the edit isn't changing but the images can be downloaded from the links listed above
Comment: http://glenbass.net/test/images/entity_config.gif
http://glenbass.net/test/images/context.gif
Comment: I reset the context. I am concerned that I am not applying the correct bundle and when I set up the context its just not displaying dynamically
Comment: Hi gilzero:
I copied my config images to the following links
http://glenbass.net/test/images/entity-02.gif
http://glenbass.net/test/images/entity-arg-02
Comment: I think you understand the flow of data, do I have the bundle set correctly?
Comment: do I have to set the context criteria and could you show me how you set up yours? Thanks for all your help
Comment: I am able to search contextually and get a result in my preview. Context, Config and Preview Below:
http://glenbass.net/test/images/entity-03.gif
http://glenbass.net/test/images/entity-con-03.gif
http://glenbass.net/test/images/entity-prev-03.gif
Comment: Still no dynamic context on node view
Comment: I updated my answer, attached the screenshot of default argument value. You should get it.
Comment: I made the changes seen in your example--
http://glenbass.net/test/images/entity-con-04.gif
Still the same result, do I need to change anything under "When the filter value IS available or a default is provided".
Up to this point I have done the same as you, what could I be doing wrong?
ARGH!
Comment: I will create step by step screenshot for you. bear with me
Comment: My answer updated, try it. good luck.
Comment: I am trying it now. Thank you for all your support

Help with displaying views with EVA


Finding the naive Bayes classifier

Question: Let $Y$ have a Bernoulli distribution with P(Y=0)=0.2 and $X$ have a Bernoulli distribution with:
$$f(X|Y) =
  \begin{cases}
    0.7       & \quad X=Y\\
    0.3  & \quad \text{ else}\\
  \end{cases}
$$
find the naive Bayes classifier of $Y$ based on two samples of $X$.
I want to find $\hat y=\arg \max_{i=1,\cdots,N} P(y=w_i|x)$ and I know I need to calculate it by:
$$P(Y|X)=\frac{P(X|Y)P(Y)}{P(X)}$$
Here is what I tried:
$$P(Y=1|X)=\frac{P(X|Y=1)P(1)}{P(X)}=\frac{P(X=?|Y=1)0.8}{P(X)=??}$$

Do I have to calculate all 4 $P(Y=1|X=1),P(Y=1|X=0),P(Y=0|X=1),P(Y=0|X=0)$?
How do I figure out $\hat y=\arg \max_{i=1,\cdots,N} P(y=w_i|x)$ from these? Is this the max of $P(Y=1|X=1),P(Y=1|X=0),P(Y=0|X=1),P(Y=0|X=0)$?

Is there a good reference or example in the subject?
Answer: for the case of one observation:
$$\hat y=\arg \max_{\omega=\Omega}\{P(Y|X)\}\doteq \arg \max_{\omega=0,1}\{P(X|Y=0)P(Y=0),P(X|Y=1)P(Y=1)\}= \arg \max_{\omega=0,1}\{P(X|Y=0)P(Y=0),P(X|Y=1)P(Y=1)\}$$
Let $m$ be the number of times x=1 hence:
$$\hat y=\arg \max_{\omega=0,1}\{P(Y=0)P(X=1|Y=0)^mP(X=0|Y=0)^{1-m},P(Y=1)P(X=1|Y=1)^mP(X=0|Y=1)^{1-m}\}=\arg \max_{\omega=0,1}\{0.2\cdot 0.3^m0.7^{1-m},0.8\cdot 0.7^m0.3
^{1-m}\}=\begin{cases}
    \arg \max_{\omega=0,1}\{0.2\cdot 0.3,0.8\cdot 0.7\}  & \quad m=1\\
    \arg \max_{\omega=0,1}\{0.2\cdot 0.7,0.8\cdot 0.3\}  & \quad m=0\\
  \end{cases}$$
$$\hat y=\begin{cases}
    \arg \max_{\omega=0,1}\{0.06,0.56\}  & \quad m=1\\
    \arg \max_{\omega=0,1}\{0.14,0.24\}  & \quad m=0\\
  \end{cases}=\begin{cases}
    1  & \quad m=1\\
    1  & \quad m=0\\
  \end{cases}$$
hence: $\hat y=1$ always with the error:
$$P_e(\hat y)=P(\hat y\neq y)=P(\hat y=0|y=1)P(y=1)+P(\hat y=1|y=0)P(y=0)$$
since  $\hat y=1$ always:
$$P_e(\hat y)=P(\hat y=1|y=0)P(y=0)=1\cdot 0.2=0.2$$
for the case of two variables:
$$\hat y=\arg \max_{\omega=\Omega}\{P(Y|X)\}\doteq \arg \max_{\omega=0,1}\{\prod_{k=1}^2P( X_k|Y=0)P(Y=0),\prod_{k=1}^2P( X_k|Y=1)P(Y=1)\}= \arg \max_{\omega=0,1}\{\prod_{k=1}^2 P(X_k|Y=0)P(Y=0),\prod_{k=1}^2 P(X_k|Y=1)P(Y=1)\}$$
Let $m$ be the number of times x_k=1 hence:
$$\hat y=\arg \max_{\omega=0,1}\{P(Y=0)P(X_k=1|Y=0)^mP(X_k=0|Y=0)^{2-m},P(Y=1)P(X_k=1|Y=1)^mP(X_k=0|Y=1)^{2-m}\}=\arg \max_{\omega=0,1}\{0.2\cdot 0.3^m0.7^{2-m},0.8\cdot 0.7^m0.3
^{2-m}\}=\begin{cases}
    \arg \max_{\omega=0,1}\{0.2\cdot 0.7^2,0.8\cdot 0.3^2\}  & \quad m=0\\
    \arg \max_{\omega=0,1}\{0.2\cdot 0.7\cdot 0.3,0.8\cdot 0.7\cdot 0.3\}  & \quad m=1\\
    \arg \max_{\omega=0,1}\{0.2\cdot 0.3^2,0.8\cdot 0.7^2\}  & \quad m=2\\
  \end{cases}$$
$$\hat y=\begin{cases}
    \arg \max_{\omega=0,1}\{0.098,0.072\}  & \quad m=0\\
    \arg \max_{\omega=0,1}\{0.042,0.168\}  & \quad m=1\\
    \arg \max_{\omega=0,1}\{0.018,0.392\}  & \quad m=2\\
  \end{cases}=\begin{cases}
    0  & \quad m=0\\
    1  & \quad m=1\\
    1  & \quad m=2\\
  \end{cases}$$
hence: $\hat y=\begin{cases}
    0  & \quad m=0\\
    1  & else\\
  \end{cases}$  with the error:
$$P_e(\hat y)=P(\hat y\neq y)=P(\hat y=0|y=1)P(y=1)+P(\hat y=1|y=0)P(y=0)$$
$$P_e(\hat y)=P(x_1=x_2=0|y=1)P(y=1)+P(x_1\neq 0 and x_2\neq 0|y=0)P(y=0)=P(x_1=x_2=0|y=1)P(y=1)+(1-P(x_1=x_2=0|y=0))|y=0)P(y=0)$$
$$P_e(\hat y)=0.8\cdot 0.3^2+(1-0.7^2)0.2=0.174$$

Finding the naive Bayes classifier


C# dictionary value is updating automatically

Question: In my recent project im trying to make a dictionary with key as a string and value as List of string(List) and adding value in dictionary using for loop ,
but the problem is that after first iteration when I Update the List for second iteration it is automatically changing in the first key value pair.
for example in first iteration it is saving key as apple and value as list {cucumber,chilli,tomato,apple} its fine but after first iteration when i update list to {cucumber,chilli,tomato,apple,mango} and saving it to second key mango it is also updating the first value to {cucumber,chilli,tomato,apple,mango}.
var mylist = new List<string>()
            {
                "cucumber",
                "chilli",
                "tomato"
            };
            var yourlist = new List<string>()
            {
                "apple",
                "mango",
                "banana"
            };
            var dict = new Dictionary<string, List<string>>();
            foreach (var i in yourlist)
            {
                mylist.Add(i);
                dict.Add(i,mylist);
            }
            foreach(var d in dict.Keys)
            {
                foreach(var l in dict[d])
                {
                    Console.WriteLine(l);
                }
            }

Comment: Issue is mentioned by Caius in his answer. But it's not only about solving that issue. But it looks very weird by looking at code and I didn't get that what you are trying to achieve here by this code? Could you please explain
Comment: You need to learn [the difference between a reference type and value type?](https://stackoverflow.com/questions/5057267/)
Comment: actually I want to add  keys which is username and value from another table which is list of access so for every iteration Im changing the value which is list of string(access) but it is giving last updated list of access to all values associated to keys
Comment: this is my acual code
Comment: for(int i = 0; i < user.Count(); i++)
                    {
                        var ll = accessRightForum.Where(x => x.UserRowId == user[i].RowID).ToList();
                        foreach (var j in ll)
                        {
                            var forumName = forum.Where(x => x.RowId == j.ForumRowId).FirstOrDefault().ForumName;
                            forumList.Add(forumName);
                        }
                        dict[user.ElementAt(i).FirstName] = forumList;
                        forumList.Clear();
                    }
Comment: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Answer: The dictionary entries' Value properties are always the same list, so anything you do to one, ends up showing in all of them (because there is only one)
Take a look at the code below; if you understand why a and b here both show the same change, then you should understand that your dictionary scenario is essentially the same
var list = new List<string>(){ "hello", "world" };
var a = list;
var b = list;

a.Add("there");

Console.Write(b.Count); //shows 3

If you don't understand why a, b and list above all refer to the same list, then drop a comment and I'll add some more explanation

As to what you should do about your "issue", it's not entirely clear to me what you're hoping to do but if you made sure that each key in the dictionary associated with a new list rather than the same one, then changes to one key's list would not show in other keys' lists:
dict.Add(i, new List<string>(mylist));

This makes a new list per key, and initializes the new list with the items present in mylist at the time (my list grows each pass of the loop)


Azure scheduled always on

Question: Is it possible with Azure to schedule the feature Always on on my Web App?.
I realized my invoice went up a lot because I had turn on the option to always have the server On, but I want to know if is possible to have the server from 7am-12m always on , and the remaining time just turn on the server when needed.
Thanks
Comment: Can you please edit your question to clarify what you're asking (as the question stands now, it runs the risk of being closed as 'unclear what you're asking')? That is: Are you referring to Virtual Machines? Web/worker role instances? Web app? And what exactly do you mean by "always on" ? (again, please edit your question; don't post clarifications in comments).
Comment: @DavidMakogon, I added web app on my question.
Comment: It's still not clear what you're attempting to do. I'm assuming you're running a web site with visitors. If you're doing something different (e.g. periodic scheduled tasks), that's something different.
Answer: For Web Apps, Always On means your app stays loaded in memory, vs being aged out (which prevents long delays when a user first visits after the app's cache has expired). It's not related to the allocated Web App resources being deployed. If you have an App Service Plan (and it's not at Free tier), you're paying for it.
With Web Apps, you can start and stop individual web apps under the app service plan, but you cannot just stop the app service plan. The ways to reduce cost-burn:

scale back number of instances
scale back size of instances
scale back service tier (you can go as low as Shared and still use custom domain name, with higher tiers required for SSL)

Azure scheduled always on


What can I do to increase the performance of a Lua program?

Question: I asked a question about Lua perfromance, and on of the responses asked:

Have you studied general tips for keeping Lua performance high? i.e. know table creation and rather reuse a table than create a new one, use of 'local print=print' and such to avoid global accesses.

This is a slightly different question from Lua Patterns,Tips and Tricks because I'd like answers that specifically impact performance and (if possible) an explanation of why performance is impacted.
One tip per answer would be ideal.
Comment: http://trac.caspring.org/wiki/LuaPerformance
Comment: Lua Optimization Tips
Answer: In response to some of the other answers and comments:
It is true that as a programmer you should generally avoid premature optimization. But. This is not so true for scripting languages where the compiler does not optimize much -- or at all.
So, whenever you write something in Lua, and that is executed very often, is run in a time-critical environment or could run for a while, it is a good thing to know things to avoid (and avoid them).
This is a collection of what I found out over time. Some of it I found out over the net, but being of a suspicious nature when the interwebs are concerned I tested all of it myself. Also, I have read the Lua performance paper at Lua.org.
Some reference:

Lua Performance Tips
Lua-users.org Optimisation Tips

Avoid globals
This is one of the most common hints, but stating it once more can't hurt.
Globals are stored in a hashtable by their name. Accessing them means you have to access a table index. While Lua has a pretty good hashtable implementation, it's still a lot slower than accessing a local variable. If you have to use globals, assign their value to a local variable, this is faster at the 2nd variable access.
do
  x = gFoo + gFoo;
end
do -- this actually performs better.
  local lFoo = gFoo;
  x = lFoo + lFoo;
end

(Not that simple testing may yield different results. eg. local x; for i=1, 1000 do x=i; end here the for loop header takes actually more time than the loop body, thus profiling results could be distorted.)
Avoid string creation
Lua hashes all strings on creation, this makes comparison and using them in tables very fast and reduces memory use since all strings are stored internally only once. But it makes string creation more expensive.
A popular option to avoid excessive string creation is using tables. For example, if you have to assemble a long string, create a table, put the individual strings in there and then use table.concat to join it once
-- do NOT do something like this
local ret = "";
for i=1, C do
  ret = ret..foo();
end

If foo() would return only the character A, this loop would create a series of strings like "", "A", "AA", "AAA", etc. Each string would be hashed and reside in memory until the application finishes -- see the problem here?
-- this is a lot faster
local ret = {};
for i=1, C do
  ret[#ret+1] = foo();
end
ret = table.concat(ret);

This method does not create strings at all during the loop, the string is created in the function foo and only references are copied into the table. Afterwards, concat creates a second string "AAAAAA..." (depending on how large C is). Note that you could use i instead of #ret+1 but often you don't have such a useful loop and you won't have an iterator variable you can use.
Another trick I found somewhere on lua-users.org is to use gsub if you have to parse a string
some_string:gsub(".", function(m)
  return "A";
end);

This looks odd at first, the benefit is that gsub creates a string "at once" in C which is only hashed after it is passed back to lua when gsub returns. This avoids table creation, but possibly has more function overhead (not if you call foo() anyway, but if foo() is actually an expression)
Avoid function overhead
Use language constructs instead of functions where possible
function ipairs
When iterating a table, the function overhead from ipairs does not justify it's use. To iterate a table, instead use
for k=1, #tbl do local v = tbl[k];

It does exactly the same without the function call overhead (pairs actually returns another function which is then called for every element in the table while #tbl is only evaluated once). It's a lot faster, even if you need the value. And if you don't...
Note for Lua 5.2: In 5.2 you can actually define a __ipairs field in the metatable, which does make ipairs useful in some cases. However, Lua 5.2 also makes the __len field work for tables, so you might still prefer the above code to ipairs as then the __len metamethod is only called once, while for ipairs you would get an additional function call per iteration.
functions table.insert, table.remove
Simple uses of table.insert and table.remove can be replaced by using the # operator instead. Basically this is for simple push and pop operations. Here are some examples:
table.insert(foo, bar);
-- does the same as
foo[#foo+1] = bar;

local x = table.remove(foo);
-- does the same as
local x = foo[#foo];
foo[#foo] = nil;

For shifts (eg. table.remove(foo, 1)), and if ending up with a sparse table is not desirable, it is of course still better to use the table functions.
Use tables for SQL-IN alike compares
You might - or might not - have decisions in your code like the following
if a == "C" or a == "D" or a == "E" or a == "F" then
   ...
end

Now this is a perfectly valid case, however (from my own testing) starting with 4 comparisons and excluding table generation, this is actually faster:
local compares = { C = true, D = true, E = true, F = true };
if compares[a] then
   ...
end

And since hash tables have constant look up time, the performance gain increases with every additional comparison. On the other hand if "most of the time" one or two comparisons match, you might be better off with the Boolean way or a combination.
Avoid frequent table creation
This is discussed thoroughly in Lua Performance Tips. Basically the problem is that Lua allocates your table on demand and doing it this way will actually take more time than cleaning it's content and filling it again.
However, this is a bit of a problem, since Lua itself does not provide a method for removing all elements from a table, and pairs() is not the performance beast itself. I have not done any performance testing on this problem myself yet.
If you can, define a C function that clears a table, this should be a good solution for table reuse.
Avoid doing the same over and over
This is the biggest problem, I think. While a compiler in a non-interpreted language can easily optimize away a lot of redundancies, Lua will not.
Memoize
Using tables this can be done quite easily in Lua. For single-argument functions you can even replace them with a table and __index metamethod. Even though this destroys transparancy, performance is better on cached values due to one less function call.
Here is an implementation of memoization for a single argument using a metatable. (Important: This variant does not support a nil value argument, but is pretty damn fast for existing values.)
function tmemoize(func)
    return setmetatable({}, {
        __index = function(self, k)
            local v = func(k);
            self[k] = v
            return v;
        end
    });
end
-- usage (does not support nil values!)
local mf = tmemoize(myfunc);
local v  = mf[x];

You could actually modify this pattern for multiple input values
Partial application
The idea is similar to memoization, which is to "cache" results. But here instead of caching the results of the function, you would cache intermediate values by putting their calculation in a constructor function that defines the calculation function in it's block. In reality I would just call it clever use of closures.
-- Normal function
function foo(a, b, x)
    return cheaper_expression(expensive_expression(a,b), x);
end
-- foo(a,b,x1);
-- foo(a,b,x2);
-- ...

-- Partial application
function foo(a, b)
    local C = expensive_expression(a,b);
    return function(x)
        return cheaper_expression(C, x);
    end
end
-- local f = foo(a,b);
-- f(x1);
-- f(x2);
-- ...

This way it is possible to easily create flexible functions that cache some of their work without too much impact on program flow.
An extreme variant of this would be Currying, but that is actually more a way to mimic functional programming than anything else.
Here is a more extensive ("real world") example with some code omissions, otherwise it would easily take up the whole page here (namely get_color_values actually does a lot of value checking and recognizes accepts mixed values)
function LinearColorBlender(col_from, col_to)
    local cfr, cfg, cfb, cfa = get_color_values(col_from);
    local ctr, ctg, ctb, cta = get_color_values(col_to);
    local cdr, cdg, cdb, cda = ctr-cfr, ctg-cfg, ctb-cfb, cta-cfa;
    if not cfr or not ctr then
        error("One of given arguments is not a color.");
    end

    return function(pos)
        if type(pos) ~= "number" then
            error("arg1 (pos) must be in range 0..1");
        end
        if pos < 0 then pos = 0; end;
        if pos > 1 then pos = 1; end;
        return cfr + cdr*pos, cfg + cdg*pos, cfb + cdb*pos, cfa + cda*pos;
    end
end
-- Call
local blender = LinearColorBlender({1,1,1,1},{0,0,0,1});
object:SetColor(blender(0.1));
object:SetColor(blender(0.3));
object:SetColor(blender(0.7));

You can see that once the blender was created, the function only has to sanity-check a single value instead of up to eight. I even extracted the difference calculation, though it probably does not improve a lot, I hope it shows what this pattern tries to achieve.
Comment: Nicely done.  I'm not 100% sure I understand the "Function constructors" section.  It might be because the math doesn't seem to have any real meaning.  Can you provide an actual example?  But otherwise, this is an outstanding list and well worth noodling over.
Comment: @JonEricson thank you for the feedback, I renamed the last section to "Parametrized functions" which I believe is a better name, as I wasn't happy with it either.
Comment: "Use tables for SQL-IN alike compares" -- In my opinion, the opposite should be suggested, use conditionals as often as possible to avoid table creation unless the lookup table is persistent or if it's just a one-time thing.
Comment: @Alundaio Obviously a static `if` construct with many OR's should be replaced by a persistent table, creating the table on each function call would defeat the purpose. I guess my example could be misinterpreted to mean it should be constructed each time. I'll try to find the time to fix it, thank you!
Comment: Also there is `local function ...`.
Comment: @Cloudiness true, but it is implicit in the very first section "avoid globals" since functions are just another type of value in Lua.
Comment: @dualed Though you used global/environment fields to put functions in the examples ;)
Comment: @Cloudiness If you mean that I have not put a `local` in front of every named function definition in my examples: I remember going back and forth on that decision, but there are valid use-cases for using globals and those examples are not about variable scope. (holy... I wrote this 5 years ago :))
Answer: If your lua program is really too slow, use the Lua profiler and clean up expensive stuff or migrate to C.  But if you're not sitting there waiting, your time is wasted.
The first law of optimization: Don't.
I'd love to see a problem where you have a choice between ipairs and pairs and can measure the effect of the difference.
The one easy piece of low-hanging fruit is to remember to use local variables within each module.  It's general not worth doing stuff like

local strfind = string.find

unless you can find a measurement telling you otherwise.
Comment: yes I know, and it is abused to create slow software to hell and back. Maybe it works for Don himself who can always assert what needs optimizing and what not. For 90% programmers out there it reads as "regardless how slow your stupid algorithm is, don't optimize it ever". That's why I don't like when it is propagated.
Comment: Are you working for MS? It seems they took your "law" to heart, considering that Notepad noticeably stutters on hardware more powerful that entire military and space control complex of roughly 30 years ago.
Comment: This is legitimately the most useless answer I have ever seen on this website and yet perfectly fine questions get downvoted to hell.
Answer:
Making the most used functions locals
Making good use of tables as HashSets
Lowering table creation by reutilization
Using luajit!
Answer: It must be also pointed that using array fields from tables is much faster than using tables with any kind of key. It happens (almost) all Lua implementations (including LuaJ) store a called "array part" inside tables, which is accessed by the table array fields, and doesn't store the field key, nor lookup for it ;).
You can even also imitate static aspects of other languages like struct, C++/Java class, etc.. Locals and arrays are enough.
Answer: Keep tables short, the larger the table the longer the search time.
And in the same line iterating over numerically indexed tables (=arrays) is faster than key based tables (thus ipairs is faster than pairs)
Comment: @Stomp if k is constant, the size of the table does not matter. This is what I am talking about.
Comment: O(1) hides a constant k. It k they are talking about here.
Comment: Lua lua tables are a combination of arrays and hastables, accessing either part by index has constant time complexity (`O(1)`).


header("Location: /") redirect in template file only works if user is logged in. Why?

Question: I was developing a new feature for a client, and deployed it live today.  However, due to circumstances out of my control, they decided to implement an alternative solution and asked me to implement a temporary redirect.
This feature was a standalone full screen flash app that did not fit the site theme at all.  So, I implemented the page as a custom html--pagename.tpl.php file and loaded the page-specific JS and CSS.  It is a custom template for an existing content type.
Now, since the client wants a temporary redirect, I added
header("Location: /");

as the first line of PHP code in the template to redirect any access to that URL to the index page.  However, this only works if the user is logged in.  Anyone
else can view this page and the header code seems to be ignored for some reason.
This URL loads regardless of the Published status set to Published or Unpublished.  In the morning, I will modify the .htaccess to execute a 302 redirect, but would like an explanation for this behavior for future reference.
Comment: try this **drupal_goto('');**
Answer: Output of the tpl files is cached for anonymous users. But only their text output is, that's how Output Control Functions in PHP works. That's why Drupal API functions will not work as intended, either. They may do their job, all right, by serving as cache breakers, but you do not want to break cache on a busy website, as it may be a server killer.
Most stable way of doing temporary redirections is, as far as I found, an .htaccess way. mod_rewrite is already used by Drupal, so it cost you nothing in additional binaries loaded to memory, and it acts before PHP is even started. As described in this answer on SO, you can do this like that:
RewriteRule ^example/my-stuff/$ /example/home/ [L,R=301]

Put it before Drupal redirects, but in the same section.
If you need a PHP logic to decide if redirect is needed, you can do it using hook_menu_alter() instead:

Substitute original callback with yours
Perform any logic needed
Log to watchdog if you want to

Call original callback and return it's result if you are not redirecting, or
Redirect with drupal_goto('<front>')
Comment: I dont attention this note, yes , your answer is better
Comment: @zhilevan I had an opportunity to fail at this issue already. On a busy site. I just don't want others to have the problems I gave myself then :D
Comment: You are awesome molot ;)
Comment: Thanks for the answer, as it does make sense to why different page load functionality exists in regard to login status.  Fortunately, this was a URL that had not gone public as of yet, so the damage is minimal.
Comment: @Jason actually there are modules that cache some pages for logged-in users, too. When you use them, and have more "levels" of caching than default one, things may really get "fun". Been there, done that, got a massive server down issue ;)
Comment: You are right that pages will be cached and the headers won't be sent again.
If you did a `header(); die()`, Drupal won't have a chance to cache the page, right? That's a horrible thing to do yes but it is still possible?
Comment: @AyeshK yes, yes it is. But again, it will be a cache breaker. [`drupal_goto`](https://api.drupal.org/api/drupal/includes!common.inc/function/drupal_goto/7) is doing basically the same thing, just more gracefully, and it takes care for messages etc. Also, some contrib cache solutions may still cache if you do that this way. Not turning PHP on at all was safest I tried.
Answer: If you work drupal try to use drupal API,
Drupal has simple api to redirect Drupal_goto
For redirecting to another url just need drupal_goto('url');
Comment: And it still may fail in tpl files, you know.


CustomValidator ServerValidate dropdownlist

Question: Im trying to validate a dropdownlist and if the selectedValue = "0" then the Isvalid = false but it doesn't seem to be working does any one know what todo here
protected void valCountry_ServerValidate(object sender, ServerValidateEventArgs e)
{
    if ((e.Value == "0"))
    {
        e.IsValid = false;
        MasterPage master = Page.Master;
        AjaxControlToolkit.ModalPopupExtender popupExtender = (AjaxControlToolkit.ModalPopupExtender)master.FindControl("ModalPopupExtender1");
        popupExtender.Show();
    }
}

<asp:DropDownList ID="ddlCountries" runat="server">
    <asp:ListItem Value="0" Text="Please Choose" />
    <asp:ListItem Value="New Zealand" Text="New Zealand" />
</asp:DropDownList>
<asp:CustomValidator ID="valCountry" runat="server"
    ControlToValidate="ddlCountries"
    Display="Dynamic"
    ErrorMessage="You must select a Country."
    SetFocusOnError="true"
    ValidationGroup="UserInfo"
    OnServerValidate="valCountry_ServerValidate">*</asp:CustomValidator>

Answer: Try using ddlCountry.options[ddlCountry.selectedIndex].value instead of e.Value.


Why is this not a valid moment generating function?

Question: Explain why there can be no random variable for which $M_x(t) = \frac{t}{1-t}$, where M is the moment generating function.
Attempt: I tried writing $\frac{t}{1-t}$ as the sum of an infinite series so $\sum t^n$ from $n=1$ to $\infty $. We know the formula for a moment generating function is $\sum e^{tx}f(x)$. So I compared the two and tried to show this leads to a density that does not integrate to 1, but I get that: $$ f(n) = \sum (\frac{t}{e^t})^n $$, which in general is convergent since $t<e^t$. How else would I show this?
Thanks!
Answer: There's no playing about with series needed, no worrying about integrals (well not directly). This is "think about what MGFs are" kind of question.
Consider the definition of an MGF -- $\quad M_X(t)=E(e^{tX})$.
Given that definition, what should $M_X(0)$ be?
Do you notice the problem?
Comment: Got it! Yes of course - this just struck me as a minute after I posted the question. Thanks :)


Laplace transform in ODE

Question: Use any method to find the laplace transform of coshbt
Looking to get help with this example for my exam review
Comment: …on a more serious note, you could use the definition of cosh: $cosh(x)=\frac{e^x+e^{-x}}{2}$, take Laplace transforms of the exponentials (that should be doable by hand), and sum them up.
Comment: …"any method" eh? I choose table :) http://tutorial.math.lamar.edu/pdf/Laplace_Table.pdf
Comment: Lol yes I wish so
Answer: The laplace transformations of $coshbt$ is the following
$$\int_0^\infty cosh({bt})e^{-st} dt$$
$$= \int_0^\infty \frac{(e^bt + e^{-bt})e^{-st}}{2} dt$$
$$= \frac{1}{2} \int_0^\infty e^{-st + bt} + e^{-st - bt} dt $$
$$= \frac{1}{2} \int_0^\infty e^{(-s + b)t} + e^{(-s - b)t} dt$$
$$=\frac{1}{2} \begin{bmatrix} \frac {e^{(-s+ b)t}}{-s+b} + \frac{e^{-st - bt}}{-s-b} \end{bmatrix}^\infty_0$$
You can probably take it from here.
Comment: You should've probably been taught that if you're doing Laplace Transformations but your welcome anyways. If it is any help using the definition of sin and cos as complex exponentials is helpful when deriving their Laplace transformation.
Comment: Okay awesome so I just need to integrate that now?
Comment: And then plug in infinity and then minus it from plugging in 0(both for t)
Comment: Thanks for the edit, I was unaware the definition of cosh but now it seems straight forward  thanks
Comment: I got a final answer of s/s^2-b^2 is that CORRECT?
Comment: And yes I was taught it I just missed a class on the intro of laplace transform and I learned inverses first
Comment: @javahelper123 If your final answer is $s/(s^2-b^2)$, then yes, you are correct (you can compare to the table I linked to above).

Laplace transform in ODE


On solving ode/pde with Neural Networks

Question: Recently, I watched this video on YouTube on the solution of ode/pde with neural network and it motivated me to write a short code in Keras. Also, I believe the video is referencing this paper found here.
I selected an example ode
$$
\frac{\partial^2 x(t)}{\partial t^2} + 14  \frac{\partial x(t)}{\partial t} + 49x(t) = 0
$$
with initial conditions
$$
x(0) = 0, \ \frac{\partial x(t)}{\partial t}\rvert_{t=0} = -3
$$
According to the video, if I understand correctly, we let the neural network $\hat{x}(t)$, be the solution of our ode, so $x(t) \approx \hat{x}(t)$
Then, we minimize the ode which is our custom cost function per say. Since, we have initial conditions, I created a step function for individual data point loss:
At, $t=0$:
$$
loss_i = \left( \frac{\partial^2 \hat{x}(t_i)}{\partial t^2} + 14  \frac{\partial \hat{x}(t_i)}{\partial t} + 49\hat{x}(t_i) \right)^2 +
\left( \frac{\partial \hat{x}(t_i)}{\partial t} + 3 \right)^2 +
\left( \hat{x}(t_i) \right)^2
$$
else
$$
loss_i = \left( \frac{\partial^2 \hat{x}(t_i)}{\partial t^2} + 14  \frac{\partial \hat{x}(t_i)}{\partial t} + 49\hat{x}(t_i) \right)^2
$$
Then, minimize batch loss
$$
\min  \frac{1}{b}  \sum_{i}^{b} loss_i
$$
where $b$ is the batch size in training.
Unfortunately, the network always learns zero. On good evidence, the first and second derivatives are very small - and the $x$ coefficient is very large i.e.: $49$, so the network learns that zero output is a good minimization.

Now there is a chance that I misinterpret the video because I think my code is correct. If someone can shed some light I will truly appreciate it.
Is my cost function correct? Do I need some other transformation?
Update:
I managed to improve the training by removing the conditional cost function. What was happening was that the the conditions were very infrequent - so the network was not adjusting enough for the initial conditions.
By changing the cost function to the following, now the network has to satisfy the initial condition on every step:
$$
loss_i = \left( \frac{\partial^2 \hat{x}(t_i)}{\partial t^2} + 14  \frac{\partial \hat{x}(t_i)}{\partial t} + 49\hat{x}(t_i) \right)^2 +
\left( \frac{\partial \hat{x}(t=0)}{\partial t}\rvert_{t=0} + 3 \right)^2 +
\left( \hat{x}(t=0)\rvert_{t=0} \right)^2
$$
The results are not perfect but better. I have not managed to get the loss almost zero. Deep networks have not worked at all, only shallow one with sigmoid and lots of epochs.
Highlight:
I am surprised this works at all since the cost function depends on derivatives of non-trainable parameters. This is interesting to me. I would love to hear some insight.

I would appreciate any input on improving the solution. I have seen a lot of fancy methods but this is the most straight forward. For example, in the referenced paper above - the author uses a trial solution. I do not understand how that works at all.
Results:

Method A = method described above

Method B = method described in the accepted answer

Shallow = One layer, 1024 nodes, gaussian activation with $b=2$

Deep    = Three layer, 10 nodes each, sigmoid activation in all

The transform method B appears to work better. The other method A, may come in handy as a control method or when boundaries are very difficult to model with a trial function, or when not solving on a rectangular domain.
I think both methods can be improved with better domain sampling instead of random shuffle, for example different sampling for the boundaries and different sampling for points inside the domain where the points are collocated.
Comment: Taking a quick look at the paper the method presented there seems different than what you are doing. In the paper they make an ansatz that explicitely fulfills the initial conditions. In your case this would be $\Psi(t)=-3t+t^2\hat{x}(t)$, where $\hat{x}(t)$ is the neural net. The cost function you need to minimize then is $\sum_i(\Psi''(t_i)+14\Psi'(t_i)+49\Psi(t_i))^2$.
Comment: When the target is not all zeros, predicting all zeros is strongly suggestive of a programming error. We have some suggestions on how to go about debugging a neural network in the linked thread; in particular, I recommend constructing unit tests for each method of your class. As a starting point, I suggest beginning with a dirt-simple ODE/PDE and building up complexity from there.
Comment: @Sycorax I removed all the code. Thank you. I only added the code to support the interpretation.
Comment: @Sycorax Can you please consider reopening. This is not a duplicate - you can see the loss function I added. This is a specific question about duplicating the results of the video.
Comment: Even if it's not a duplicate, it's not on-topic per the [help] because this question is primarily concerned with debugging code. In particular, the loss going to  zero but the model making poor & constant predictions is strongly suggestive of a bug in your code. If you could edit your post to focus on an on-topic statistical question, not debugging code, I will re-open.
Comment: Thanks! I think this is an interesting question (+1).
Comment: I'm impressed by the improvement that you achieved using this new loss function. It looks like you're on the right track. In terms of getting the loss to zero, you've noted that *deeper* networks are not effective. Have you explored using *wider* networks? For instance, what happens if you double the number of hidden units?
Comment: @Sycorax I did that and was able to improve but up to a certain point. Not much roi after a certain number.
Comment: @sebhofer Thank you. I still have trouble with this. I do not understand how to come up with that form of the equation. Also, is there something after the minus sign in your cost function? I was following the video - but in the video he references the paper.
Comment: @sebhofer Explaining how the paper constructs their function - would qualify as improving the solution so you can get the point. You can post it as an answer. Thank you/
Comment: ohh i see - because when you take the derivative - you want a $t$ left there to get rid of that part of the equation
Comment: I added an answer. Let me know if there's still something unclear. I would be interested to see plots of your final solution, pls post some!
Comment: @EdvBeq Could you please provide the working code? I have been trying to reproduce it but I failed.
Comment: @MatheusManzatto Sorry for the late reply - I have the inverse version of this (meaning i'm also solving for the parameters) but you can bring it back to the above. Here is my github: https://github.com/uninstallit/CS584/tree/master/Final_Project_Beqari_and_Williamson/src
Answer: The procedure presented in the paper seems to be slightly different from the one above. In the paper the authors make an ansatz that explicitely fulfills the initial conditions. For a second order differential equation of the form
$$
\Psi''(t)=f(t,\Psi(t),\Psi'(t))
$$
with $\Psi(0)=A$ and $\Psi'(0)=B$ they suggest to use (see section 3.1 and specifically equation (13) in the preprint) $$\Psi(t)=A+Bt+t^2N(t),$$ where $N(t)$ is the neural net. Note that this form is not unique, but it will have the correct initial values no matter what $N(0)$. The cost function to optimize on the other hand is
$$
C=\sum_i(\Psi''(t_i)-f(t_i,\Psi(t_i),\Psi'(t_i)))^2,
$$
where $\{t_i\}_i$ is a set of collocation points that are sampled from the domain of $\Psi$. So for your example problem you have $A=0$, $B=-3$, and $C=\sum_i(\Psi''(t_i)+14\Psi'(t_i)+49\Psi(t_i))^2$.
Comment: @EdvBeq It seems that you have additional questions about pde/ode models. This is great, but the best way to ask a question is to use the Ask Question button, not write a comment.
Comment: Added result graphs. The trial solution works well.

On solving ode/pde with Neural Networks


Obter os valore de uma nova linha num gráfico

Question: Tenho uma folha Excel em que todos os dias acrescento uma nova linha com os valores da cotação Brent e outros indicadores. Esses mesmo valores depois são atualizados num gráfico de linhas.
Acontece que sempre que adiciono uma linha nova, os valores da mesma não são inseridos no gráfico por não fazerem parte do range da formula no gráfico.
Há alguma forma de ir buscar a informação da nova linha automaticamente sem ter que reconfigurar o gráfico para o novo range?
Esta fórmula está inserida no gráfico
<code>=Energias!$A$1:$C$417;Energias!$J$1:$Q$417;Energias!$T$1:$V$417
</code>
Quando adiciono uma linha nova a fórmula do gráfico fica assim:
<code>=Energias!$A$1:$C$1;Energias!$A$3:$C$418;Energias!$J$1:$Q$1;Energias!$J$3:$Q$418;Energias!$T$1:$V$1;Energias!$T$3:$V$418
</code>
Comment: Não compreendi muito bem. Favor criar um [mcve]
Answer: Olá
Você pode criar um Range com desloc.
Suponhamos que você tenha a situação abaixo

Você pode criar um campo através do gerenciador de nomes

Utilizando a formula
=DESLOC(Planilha1!$A$2;0;0;CONT.NÚM(Planilha1!$A:$A);1)
Aí é só colocar o nome pelo qual optou no meu caso usei "Teste" na serie que você quer

O gráfico a seguir ficou dinâmico. Cada valor digitado na coluna A será automaticamente replicado para o gráfico.


Java JPanel.add(...) not working

Question: Here is my code I have written thus far, along with the errors I'm receiving. I'm sure I've done similar code before with no errors. I'm pretty sure I'm missing something stupid but I can't figure it out or find anything on the web.
import javax.swing.*;
import java.awt.*;

public class MyFrame extends JFrame {

    JPanel mainPanel = new JPanel();
    JButton editButton = new JButton("Edit");
    JPanel.add(editButton);
}

Error:
Syntax error on token(s), misplaced construct(s) - for the underlined '.'
on the last line
Syntax error on token "editButton", VariableDeclaratorId expected after this
token - for the underlined parameter within the brackets on the last line.

Answer: You're trying to use it as if it's a static method - which panel would you expect to add the edit button to? You need to call it on the mainPanel:
mainPanel.add(editButton);

However, you can't do that in a class declaration - a statement like that has to be in a method or constructor. So you probably want:
import javax.swing.*;
import java.awt.*;

public class MyFrame extends JFrame {
    JPanel mainPanel = new JPanel();
    JButton editButton = new JButton("Edit");

    public MyFrame() {
        mainPanel.add(editButton);
    }
}

Or possibly put all the initialization into the constructor, and also make the variables final and private:
import javax.swing.*;
import java.awt.*;

public class MyFrame extends JFrame {
    private final JPanel mainPanel;
    private final JButton editButton;

    public MyFrame() {
        mainPanel = new JPanel();
        editButton = new JButton("Edit");
        mainPanel.add(editButton);
    }
}

Comment: Thanks, i knew it was something stupid, this is because i'm trying to rush code, I even typed my question wrong thanks to rushing. I meant to put mainPanel.add() which also wasn't working, but you've answered that anyway. Thanks.
Answer: Use it as follows.
mainPanel.add(editButton);

You must call it through it's object and not with associating it with its class name as it is not a static method.

Java JPanel.add(...) not working


Mixture of Two Normals

Question: Suppose we have a data which consists of two normals,
x = rnorm(50,mean=1,sd=2)
y = rnorm(50,mean=2,sd=3)
z = sample( c(x,y) , size = 100, replace=FALSE )

The goal is to estimate the two normal parameters and then classify which of the two normal families each of the data points belong to.
Here is an approach to this problem that I was thinking of. Let $p = (\mu_1,\sigma_1,\mu_2,\sigma_2)$ denote the 4-dimensional vector of real numbers, and $I$ a binary string of length $100$, i.e. for example, $I = (1,1,...,1,0,0,..0)$, here $I$ is categorical and indicates which normal family each data point belongs to. We can define the following likelihood function,
L = function(p,I){
likelihood = 0
         for(i in 1:100){
         likelihood = likelihood + dnorm(z[i],p[1],p[2])*I[i] + dnorm(z[i],p[3],p[4])*(1 - I[i])
} }

Now that this function has been defined it remains to sample from it. It is possible to write  a custom Metropolis-algorithm here, however, I am not sure how efficient it would be, and wondered if there is a way to do this using Stan? According to the Stan manual, the software does not allow discrete parameters, so I would be unable to define a binary parameter $I$ as a vector of $100$ components. Is there a workaround to this that will allow Stan to sample from this high-dimensional distribution?
Comment: Using directly the observed likelihood in an MCMC scheme has been proposed a while ago. E.g., in our [Bayesian Core book.](https://amzn.to/3Vd6Y4J)
Answer: If I remember correctly, in cases where Stan does not allow you to use discrete parameters, the most straightforward solution is to marginalize over the latent discrete variable. I am quoting the pymc3 documentation (from: https://pymc3-testing.readthedocs.io/en/rtd-docs/notebooks/marginalized_gaussian_mixture_model.html)

A natural parameterization of the Gaussian mixture model is as the latent variable model:
$$
\begin{split}\begin{align*}
\mu_1, \ldots, \mu_K
    & \sim N(0, \sigma^2) \\
\tau_1, \ldots, \tau_K
    & \sim \textrm{Gamma}(a, b) \\
\boldsymbol{w}
    & \sim \textrm{Dir}(\boldsymbol{\alpha}) \\
z\ |\ \boldsymbol{w}
    & \sim \textrm{Cat}(\boldsymbol{w}) \\
x\ |\ z
    & \sim N(\mu_z, \tau^{-1}_z).
\end{align*}\end{split}
$$
An alternative, equivalent parameterization that addresses these problems is to marginalize over z. The marginalized model is:
$$
\begin{split}\begin{align*}
\mu_1, \ldots, \mu_K
    & \sim N(0, \sigma^2) \\
\tau_1, \ldots, \tau_K
    & \sim \textrm{Gamma}(a, b) \\
\boldsymbol{w}
    & \sim \textrm{Dir}(\boldsymbol{\alpha}) \\
f(x\ |\ \boldsymbol{w})
    & = \sum_{i = 1}^K w_i\ N(x\ |\ \mu_i, \tau^{-1}_z),
\end{align*}\end{split}
$$
Marginalizing $z$ out of the model generally leads to faster mixing and better exploration of the tails of the posterior distribution. Marginalization over discrete parameters is a common trick in the Stan community, since Stan does not support sampling from discrete distributions.

I think this post might be useful: https://jeremy9959.net/Blog/StanMixture/

Mixture of Two Normals


Is there any benefit to making a Makefile a dependency of one of its own rules?

Question: I came across this in a Makefile the other day:
%.elf: $(OBJS) $(LDSCRIPT) Makefile
    $(Q)$(LD) -o $(*).elf $(OBJS) $(LDFLAGS)

What is the use of this?
Answer: Assuming that this rule is the "parent" of all your other rules (i.e. all your targets require this rule), having the Makefile itself as a dependency ensures that if $(OBJS) is ever modified, your targets will be relinked no matter the state of your .elf output, because the Makefile's last modified date will have changed.
It's a bit like a self-repair mechanism.
Comment: It is also a bit if a nuisance if you need to edit the Makefile for others reasons.


Possible arrangments Letters?

Question: How many arrangements are possible of the letters in EZPZ I CAN DO IT, which has five vowels (A, E, I, I, O) and seven consonants (C, D, N, P, T, Z, Z).
a)  if there are no restrictions,
b)  if the letters must be in alphabetical order?
c)  if the vowels must be together?
d)  6 letter words if the vowels must be together.
Comment: I edited your tags. More tags does not mean more help.
Comment: first time users. Thanks!
Comment: Welcome to math.SE. Please share your thoughts on the problem so far. Knowing what you know will help us provide the most useful hints and solutions!
Answer: (a) If there are no restrictions, we permute. However, since $I$ and $Z$ are repeated twice, we have to divide out according to the multinomial distribution:
$$\dfrac{12!}{2! * 2!}$$
(b) If you count the two $I$ and two $Z$ characters as the same, there is only one string with the letters in alphabetical order. If you count the two $I$ and two $Z$ characters as distinct, we can swap them, so we get $4$ ways. I would personally treat the $I$ and $Z$ characters as the same.
(c) Start by grouping the vowels. There are $\dfrac{5!}{2!}$ ways to permute the vowels. We then permute the consonants. There are $\dfrac{7!}{2!}$ ways to do this. You then place the vowel cluster in the consonants string. There are $8$ ways to do this. So by rule of product, we get:
$$\dfrac{5! * 7!}{2! * 2!} * 8 $$
(d) We again permute the vowels, which gives us $\dfrac{5!}{2!}$. We then choose a consonant. The $z$ is counted twice, so we have $6$ ways to choose a consonant. It must go at the beginning or end. There are two ways to do this. So by rule of product we get:
$$ \dfrac{5!}{2!} * 6 * 2 = 6! $$
Comment: Note that for (d), I am only describing a single six letter word. If you want two six letter words, I will be happy to update my answer.

Possible arrangments Letters?


Custom User Login Template Error

Question: I am using Drupal 7 and I have added some code to my template.php file in my theme (which I found on a Drupal tutorial) so I can use a custom template for the user login form. The form works and it is using the custom template but it is generating these errors:
Notice: Undefined index: classes_array in template_process() (line 2527 of /includes/theme.inc).
Warning: implode(): Invalid arguments passed in template_process() (line 2527 of /includes/theme.inc).
Notice: Undefined index: attributes_array in template_process() (line 2533 of /includes/theme.inc).
Notice: Undefined index: title_attributes_array in template_process() (line 2534 of /includes/theme.inc).
Notice: Undefined index: content_attributes_array in template_process() (line 2535 of /includes/theme.inc).

The code I have in my template.php file is:
function cr_theme() {
  $items = array();

  $items['user_login'] = array(
    'render element' => 'form',
    'path' => drupal_get_path('theme', 'cr') . '/templates',
    'template' => 'user-login',
    'preprocess functions' => array(
       'cr_preprocess_user_login'
    ),
  );
  return $items;
}

function cr_preprocess_user_login(&$vars) {
  $vars['forgot_link'] = l(t('Forgot password?'),'user/password');
}

and my user-login.tpl.php file contains:

  <?php print drupal_render_children($form) ?>

<?php print render($forgot_link); ?>

I have been trying to find the cause of the errors which I think is probably something I have not added to my cr_theme() function but now need some help.
Answer: By playing around with the preprocess function I seem to have fixed the errors. I changed the preprocess function to:
function cr_preprocess_user_login(&$vars) {
  $vars['content_attributes_array']['class'][] = 'content';
  $vars['title_attributes_array']['class'][] = 'content';
  $vars['attributes_array']['class'][] = 'content';
  $vars['classes_array'] = array('content');
  $vars['forgot_link'] = l(t('Forgot password?'),'user/password');
}

I just added in the undefined indexes that were coming up in the errors. I'm really not completely sure what I am doing but it does seem to be removing the errors so far.
It would be great if someone could comment how this is working.
Answer: I would try add back in all the default preprocess functions to the array, given that the system is whining about getting undefined indexes.

preprocess functions: A list of functions used to preprocess this
  data. Ordinarily this won't be used; it's automatically filled in. By
  default, for a module this will be filled in as
  template_preprocess_HOOK. For a theme this will be filled in as
  phptemplate_preprocess and phptemplate_preprocess_HOOK as well as
  themename_preprocess and themename_preprocess_HOOK.
Comment: Thank you cchen for taking the time to answer my query. I seem to have fixed it using the answer I have given, though I'm not completely sure how I managed it.
Comment: You're welcome @Ben glad you figured it out :)
Answer: I had exactly the same issue (guessing I found & used the same tutorial referenced in the question... :)
Based on the info provided by cchen, I simply removed the preprocess array info:
   'preprocess functions' => array(
   'cr_preprocess_user_login'
),

Cleared cache, and that was it.
Also like you, I don't know enough about this stuff to understand why this worked.  All I can do is confirm it did.

Custom User Login Template Error


OSSEC Treat Multiple Files as One

Question: A while back I posted about using OSSEC as a sudo SIEM as far as sending logs from various servers to one OSSEC server and using the correlation to trip alerts. Overall that solution worked very well but I've recently had to divide out the logs for load balancing reasons so now I have something along the lines of:
OSSEC Server

/mnt/logs/unix1.log
/mnt/logs/unix2.log
/mnt/logs/unix3.log
/mnt/logs/unix4.log

Each log file is being written by a different syslog server for load balancing reasons and it works pretty good overall. However the issue I'm running into is that OSSEC can run correlation rules against each log file individually but if there are 4 failed logins and they happen to be spread out over the 4 log files, OSSEC would only see 1 on each instance and not alert of 4 failed logins from 1 user.
Is there anyway to make OSSEC treat the files as one? I'm looking at other solutions such as a gluster/cluster file system where I can write from multiple servers to one file which would fix the issue.
Answer: I was misinformed on how OSSEC processes logs. Per this forum, Jesus Linares clarified that OSSEC groups logs on a "decoded as" basis and treats multiple log formats the same.
I also confirmed this by verifying a 6 failed login correlation actually had individual logs over 4 different log files.

OSSEC Treat Multiple Files as One


Troubleshooting an NFS server hanging after authenticated mount request

Question: I need some advice on troubleshooting an NFS server problem on Scientific Linux (RHEL) 6.1. The log on the server shows that an authenticated mount request was made:
Jan 13 16:30:02 ??? rpc.mountd[3996]: authenticated mount request from ????:784 for /shared-storage/cm/shared (/shared-storage/cm/shared)

But after that, it does not continue. On the client, it is also hanging. The interesting thing now is that I have two NFS servers, which should be identical, and the one is working perfectly, but the other exhibits the above mentioned behaviour. The problem is also not completely persistent, i. e. sometimes the mount request succeeds.
I assume that the problem must be related to the server rather than to the client, because it is working perfectly on the other server. My question is where I should search the problem. I have already re-created the exports using exportfs -r, I have restarted the NFS server, I have compared the rpcinfo outputs of both server - no success. The problem even survives a reboot. Any other ideas are appreciated.
As answer to Tim's question: I have sporadically the following in dmesg, but do not know whether it is related
e1000e 0000:0c:00.0: eth4: Detected Hardware Unit Hang:
  TDH                  <24>
  TDT                  <25>
  next_to_use          <25>
  next_to_clean        <24>
buffer_info[next_to_clean]:
  time_stamp           <1c3d12940>
  next_to_watch        <24>
  jiffies              <1c3d12940>
  next_to_watch.status <0>
MAC Status             <80383>
PHY Status             <792d>
PHY 1000BASE-T Status  <7800>
PHY Extended Status    <3000>
PCI Status             <10>

Further edit: The problem above does not occur on the machine that is working, so it probably is related.
Again an edit: The error is not on the (software) device that is used for NFS, but on another one. The NFS mount also does not trigger the message.
Comment: anything in syslog or dmesg that seems suspicious? wonder if there is hardware trouble on the misbehaving machine
Comment: That's an idea. Possibly in dmesg: "e1000e 0000:0c:00.0: eth4: Detected Hardware Unit Hang:" - "e1000e" is the Intel NIC.
Comment: @Tim: Post an answer that I can vote for it if your are right...
Comment: Created an answer, good luck!
Comment: Also added a hopefully relevant link, from our good friends at ServerFault ;)
Answer: Make sure portmap is running on both the server AND the client.
Answer: Anything in syslog or dmesg that seems suspicious? I am curious if there is hardware trouble on the misbehaving system.
Edit, curious about your error you saw in dmesg, and found the same error mentioned here: Linux e1000e (Intel networking driver) problems galore, where do I start?
From all of the debugging output the OP posted, I was SURE that his hardware was about dead, apparently there was a kernel parameter to fix the issue: pcie_aspm=off
You can try booting with that parameter and see if it fixes things up!
Comment: I was able to further investigate the effect this morning. The error message is on another device than the one serving NFS and the message also cannot be provoked vis NFS use (actually, it works this morning). I will nevertheless try `pcie_aspm=off` asap, but I cannot reboot the machine in question without notifying the users some time in advance...
Comment: I now have added `pcie_aspm=off`, which fixes the issue in dmesg, but unfortunately not the actual problem...
Comment: As I did not have another idea, I finally replaced the NIC in the machine for testing purposes. Now it is working again, so I think the hardware was actually broken although the other NIC produced the output in dmesg...
Comment: Interesting... Good rule to follow though, if it is acting up, and there is no logical explanation, assume the hardware is failing / defective.


Display a field 'Category' on the home page

Question: How do I display a field 'category' (so as 'tags') in the node.tpl.php?
Comment: Every field that is associated to a node is shown in the node view page, if the field is not set to be hidden. Then, the home page (a.k.a. the front page) doesn't use node.tpl.php.
Answer: Every field that is associated to a node is shown in the node view page, if the field is not set to be hidden.
You don't need to change the template file used for nodes, as it shows all the node fields by default. If you are not seeing a field is probably because that field settings are set to not show that field.
As you are referring to the home page (a.k.a. the front page), which normally shows the teaser of nodes promoted to front page, check the field settings for the teaser view mode.


Dividir uma query

Question: Preciso de uma ajuda de vocês,
eu tenho uma query em meu sistema, que tem 4 cases
SELECT TOP 3 * FROM (
    SELECT Title, SUM(Count) AS Count
        FROM (
            SELECT
                CASE WHEN EstadoId = EstadoIdAutor AND EstadoId = Meddoc1 AND EstadoId = Meddoc2 AND EstadoId = Meddoc3 THEN
                    'Sem deslocamento'
                ELSE
                    CASE WHEN EstadoId = EstadoIdAutor AND (EstadoId = Meddoc1 OR EstadoId = Meddoc2 OR EstadoId = Meddoc3) THEN
                        'Fora de domicílio do autor'
                    ELSE
                        CASE WHEN EstadoId <> EstadoIdAutor OR (EstadoId = EstadoIdAutor AND (EstadoId <> Meddoc1 AND EstadoId <> Meddoc2 AND EstadoId <> Meddoc3)) THEN
                            'Mais de um deslocamento'
                        ELSE 'Outros'
                        END
                    END
                END AS Title,
            COUNT(*) AS Count
            FROM jud_Processos
            WHERE Status = 1 AND Trancado = 1 {dados}
            GROUP BY EstadoId, EstadoIdAutor, Meddoc1, Meddoc2, Meddoc3
        ) Sub
    GROUP BY Title
) TB_AUX

Queria saber se é possivel dividir esta query que no caso seria em 4 partes
Sem deslocamento - Fora de domicílio do autor - Mais de um deslocamento - Outros
É possivel?
Para ser mais específico, cada case em um select.
Comment: Qual é o gerenciador de banco de dados: MySQL? Oracle Database? SQL Server? outro?
Comment: SQL Server Amigo
Answer: É possível, através de UNION. Mas não é recomendável - a performance vai ser pior.
A recomendação é simplificar seus CASE. Você abre um CASE completo como "caso contrário" de cada case. Você pode alcançar o mesmo objetivo da seguinte forma:
SELECT TOP 3 * FROM (
    SELECT Title, SUM(Count) AS Count
        FROM (
            SELECT
                CASE
                    WHEN EstadoId = EstadoIdAutor AND EstadoId = Meddoc1 AND EstadoId = Meddoc2 AND EstadoId = Meddoc3 THEN
                        'Sem deslocamento'
                    WHEN EstadoId = EstadoIdAutor AND (EstadoId = Meddoc1 OR EstadoId = Meddoc2 OR EstadoId = Meddoc3) THEN
                        'Fora de domicílio do autor'
                    WHEN EstadoId <> EstadoIdAutor OR (EstadoId = EstadoIdAutor AND (EstadoId <> Meddoc1 AND EstadoId <> Meddoc2 AND EstadoId <> Meddoc3)) THEN
                        'Mais de um deslocamento'
                    ELSE
                        'Outros'
                END AS Title,
                COUNT(*) AS Count
            FROM jud_Processos
            WHERE Status = 1 AND Trancado = 1
            GROUP BY EstadoId, EstadoIdAutor, Meddoc1, Meddoc2, Meddoc3
        ) Sub
    GROUP BY Title
) TB_AUX

Fica inclusive mais legível.
Recomendo dar uma olhada nesse artigo da Microsoft sobre a sintaxe do CASE. De repente você até otimiza outras queries do seu sistema ;)
Comment: Entendi, obrigado pela ajuda, mais eu realmente precisava dividir, porque esta query esta puxando 4 valores, e acho que dividindo eu consigo deixar só os 3 evitando que apareça i 4 valor que seria aquele "Outros", se eu só apagar o else 'Outros' ele vem como NULL então queria dividir para ver se consigo puxar apenas um dado por select, por ser uma query pequena aparentemente, não vai mudar muito a performa do meu sistema, mais eu agradeço a sua ajuda obrigado!
Answer: Se o intuito é separar os 3 primeiros CASE, você pode fazer da seguinte forma:
-- Sem deslocamento
SELECT 'Sem deslocamento' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid = jp.estadoidautor
   AND jp.estadoid = jp.meddoc1
   AND jp.estadoid = jp.meddoc2
   AND jp.estadoid = jp.meddoc3

-- Fora do domicílio do autor
SELECT 'Fora de domicílio do autor' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid = jp.estadoidautor
   AND (jp.estadoid = jp.meddoc1
    OR  jp.estadoid = jp.meddoc2
    OR  jp.estadoid = jp.meddoc3)
-- Pelo menos 1 diferente
   AND (jp.estadoid <> jp.meddoc1
    OR  jp.estadoid <> jp.meddoc2
    OR  jp.estadoid <> jp.meddoc3)

-- Mais de um deslocamento
SELECT 'Mais de um deslocamento' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid <> jp.estadoidautor
    OR (jp.estadoid <> jp.meddoc1
   AND  jp.estadoid <> jp.meddoc2
   AND  jp.estadoid <> jp.meddoc3)

E se quiser trazer em apenas um resultado:
-- Sem deslocamento
SELECT 'Sem deslocamento' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid = jp.estadoidautor
   AND jp.estadoid = jp.meddoc1
   AND jp.estadoid = jp.meddoc2
   AND jp.estadoid = jp.meddoc3
UNION
-- Fora do domicílio do autor
SELECT 'Fora de domicílio do autor' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid = jp.estadoidautor
   AND (jp.estadoid = jp.meddoc1
    OR  jp.estadoid = jp.meddoc2
    OR  jp.estadoid = jp.meddoc3)
-- Pelo menos 1 diferente
   AND (jp.estadoid <> jp.meddoc1
    OR  jp.estadoid <> jp.meddoc2
    OR  jp.estadoid <> jp.meddoc3)
UNION
-- Mais de um deslocamento
SELECT 'Mais de um deslocamento' AS Title,
       COUNT(1) AS Count
  FROM jud_processos jp
 WHERE jp.status = 1
   AND jp.trancado = 1
   AND jp.estadoid <> jp.estadoidautor
    OR (jp.estadoid <> jp.meddoc1
   AND  jp.estadoid <> jp.meddoc2
   AND  jp.estadoid <> jp.meddoc3)

Comment: Isso mesmo amigo!, funcionou perfeitamente, agradeço sua ajuda!

Dividir uma query


Does Converting a byte array to object increases the size..?

Question: my code is as follows..
private Object populateObj(InputStream inputStream) {
        int i=0;
        Object resObject = null;
        try {
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            int next = inputStream.read();
            while (next > -1) {
                i++;
                bos.write(next);
                next = inputStream.read();
            }
            bos.flush();
            byte[] result = bos.toByteArray();
            System.out.println("Size of byte array 1: "+result.length);
            System.out.println("Result is []");
            for(int j=0;j<result.length;j++){
                System.out.println(result[j]);
            }
            resObject = result;
            byte[]  result2=toByteArray(resObject);
            System.out.println("Size of byte array 2 : "+result2.length);
            System.out.println("Result2 is []");
            for(int j=0;j<result.length;j++){
                System.out.println(result2[j]);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }

        System.out.println("size of inputStream in bytes : "+i);
        return resObject;
    }

code for converting object to byte array is:-
public static byte[] toByteArray(Object obj) throws IOException {
        byte[] bytes = null;
        ByteArrayOutputStream bos = null;
        ObjectOutputStream oos = null;
        try {
            bos = new ByteArrayOutputStream();
            oos = new ObjectOutputStream(bos);
            oos.writeObject(obj);
            oos.flush();
            bytes = bos.toByteArray();
        } finally {
            if (oos != null) {
                oos.close();
            }
            if (bos != null) {
                bos.close();
            }
        }
        return bytes;
    }

when i compare the length of the both the byte arrays its different.some extra byte are added but at which point of conversion..?(while assigning the byte array to object or when)
Size of input Stream comes same as first byte array.
I want to convert the Inputstream of http request into a resObject which can be then given to outputstream.Using ObjectInputStream.readObject() gives StreamCorruptedException.
Answer: You're serializing a byte array using Java object serialization. This has an overhead over the raw byte array, since the serialized stream contains a serialization header, the name of the class of the object being serialized, and probably some additional bytes to be able to know where the object starts and ends in the stream.
Remember that, when deserializing, the receiver knows nothing about the contents of the ObjectInputStream. So it must be able to know that what has been serialized is a single byte array (and not a String, an Integer or any other serializable object). So this information has to be stored somehow in the stream, and thus needs some additional bytes.
Comment: so how should i retrieve the original input Stream.If i directly retrieve it from object by converting it into byte Array,some garbage are added at the start of file..
Comment: I want to convert the Inputstream of http request into Object.Using ObjectInputStream.readObject() gives StreamCorruptedException.Is there any way to remove the extra bytes.
Comment: Assigning an object to a variable will never change the value of an object. Of course it's the toByteArray() method. That's where you're serializing your array using an ObjectOutputStream. You seem to really not understand what you're doing. Why are you transforming a byte array into another byte array using serialization. What do you want to achieve?
Comment: I'm not sure why you're using object serialization in the first place if all you want to do is write one byte array to a file. But you do it using `ObjectInputStream.readObject()`.
Comment: It's extremely unclear what you want to achieve. Edit your question, and post the actual code, and the complete stack trace of the exception. You keep talking about converting an InputStream to Object. But InputStream is already an Object. And byte[] is also an Object.
Comment: Can you plz tell me where the addition bytes are added.During `resObject = result` or while reconstructiong the byte array by calling `toByteArray(Object)`
Answer:     byte[] array = new byte[] { 1, 2, 3, 4 };
    System.out.println(array.length);

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(baos);
    oos.writeObject(array);
    oos.close();

    ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
    ObjectInputStream ois = new ObjectInputStream(bais);
    Object obj = ois.readObject();
    ois.close();

    System.out.println(((byte[]) obj).length);


I want to add input box and name attribute in input tag using javascript while inside table row while creating using javascript on the run time

Question: I was trying to add the new table row using javascript when the user clicks on the add button. I have successfully created the logic for it. Further I am adding input box within the  tag and name attribute inside the <input> tag, but my code is only adding input box either in the first td of tr tag or at the end.
My code follows :-

srno = 1

function CreateTables() {
  var inputbox = document.createElement('input')
  inputbox.classList.add('textbox')

  const tables = document.querySelector('table')

  const NewRow = tables.insertRow(-1)

  let cell1 = NewRow.insertCell(0)
  cell1.innerHTML = (srno += 1)

  let cell2 = NewRow.insertCell(1)
  cell2.appendChild(inputbox)

  let cell3 = NewRow.insertCell(2)
  cell3.appendChild(inputbox)

  let cell4 = NewRow.insertCell(3)
  cell4.appendChild(inputbox)

  let cell5 = NewRow.insertCell(4)
  cell5.appendChild(inputbox)
}

let add = document.querySelector('.Addbutton')
add.addEventListener('click', function() {
  CreateTables()
});
<form method='post' action='/analyzer'>

            Sr no.
            Name of Appliances
            Watts (w)
            Number
            No. of Hours

            1
            <input class='textbox' type='text' name='nameofapp'>
            <input class='textbox' type='number' name='watts'>
            <input class='textbox' type='number' name='no'>
            <input class='textbox' type='number' name='hours'>

    <button class='Submitbutton' type='submit'>Submit</button>

</form>

  <button type='submit' class='Addbutton'>Add</button>

Pls help me adding <input> tag in all  tags of new table row generated. Also how can I add the name attribute within the <input> tag
Answer: Appending an element doesn't make a copy of it. Every time you do cellN.appendChild(inputbox) you move the input box from the previous cell to the current cell.
You can use inputbox.cloneNode() to make a copy of it.

srno = 1

function CreateTables() {
  var inputbox = document.createElement('input')
  inputbox.classList.add('textbox')

  const tables = document.querySelector('table')

  const NewRow = tables.insertRow(-1)

  let cell1 = NewRow.insertCell(0)
  cell1.innerHTML = (srno += 1)

  let cell2 = NewRow.insertCell(1)
  cell2.appendChild(inputbox)

  let cell3 = NewRow.insertCell(2)
  cell3.appendChild(inputbox.cloneNode())

  let cell4 = NewRow.insertCell(3)
  cell4.appendChild(inputbox.cloneNode())

  let cell5 = NewRow.insertCell(4)
  cell5.appendChild(inputbox.cloneNode())
}

let add = document.querySelector('.Addbutton')
add.addEventListener('click', function() {
  CreateTables()
});
<form method='post' action='/analyzer'>

            Sr no.
            Name of Appliances
            Watts (w)
            Number
            No. of Hours

            1
            <input class='textbox' type='text' name='nameofapp'>
            <input class='textbox' type='number' name='watts'>
            <input class='textbox' type='number' name='no'>
            <input class='textbox' type='number' name='hours'>

    <button class='Submitbutton' type='submit'>Submit</button>

</form>

  <button type='submit' class='Addbutton'>Add</button>

Answer: You may make use of innerHTML property on cells.
    const NewRow = tables.insertRow(-1)

    let cell1 = NewRow.insertCell(0)
    cell1.innerHTML = (srno += 1)

    let cell2 = NewRow.insertCell(1)
    cell2.innerHTML = "<input name='nameOfApp'>";

    let cell3 = NewRow.insertCell(2)
    cell3.innerHTML = "<input name='watts'>";

    let cell4 = NewRow.insertCell(3)
    cell4.innerHTML = "<input name='number'>";

    let cell5 = NewRow.insertCell(4)
    cell5.innerHTML = "<input name='noOfHours'>";

Comment: I recommend @Barmar's Solution as the Appropriate One. Thanks.
Answer: function CreateTables() {
        const tables = document.querySelector('table');
        let tbody = tables.getElementsByTagName("tbody")[0];
        let tr_clone = tbody.children[0].cloneNode(true);
        tr.children[0].innerHTML = (srno += 1);
        const NewRow = tables.insertRow(-1);
        NewRow.innerHTML = tr.innerHTML;
    }