For a convex surface, why does the second fundamental form have to be positive semidefinite, instead of negative semidefinite?

Question: My confusion comes from the answer of this question: Is there a way to relate convexity to Gaussian curvature?
In the first paragraph of Willie's answer, he wrote

The answer is yes. Convexity implies that for every point $p$ on the boundary $\partial \Omega$, we can find a plane $\Pi$ through $p$ such that $\partial \Omega$ lies "on the one side" of $\Pi$. This implies that the second fundamental form is signed (either positive-semi-definite or negative-semi-definite). For surfaces in $\mathbb{R}^3$, the Gaussian curvature is the determinant of the second fundamental form, and hence is the product of the eigenvalues (the principal curvatures), and hence is always positive definite.

However, if the second fundamental form is negative-semidefinite, then because the matrix representation of the second fundamental form for a surface in $\mathbb{R}^3$ is $2 \times 2$, won't the Gaussian curvature also be non-negative?
Thanks for all the help; I am new to differential geometry and my apologies if this question is too elementary.
Answer: That's precisely the point: For a surface, Gaussian curvature does not depend on the choice of the unit normal. However, changing that choice changes the sign of the second fundamental form — hence changing a positive-definite symmetric matrix to negative-definite and vice-versa.
Comment: By changing the "choice", are you referring to choosing the additive inverse of the originally chosen normal vector?
Comment: Precisely, yes.


Random walk: police catching the thief

Question: This is a problem about the meeting time of several independent random walks on the lattice $\mathbb{Z}^1$:
Suppose there is a thief at the origin 0 and $N$ policemen at the point 2.
The thief and the policemen began their random walks independently at the same time following the same rule: move left or right both with probability 1/2.
Let $\tau_N$ denote the first time that some policeman meets the thief.
It's not hard to prove $E\tau_1=\infty$.
so what is the smallest $N$ such that $E\tau_N<\infty$?
I was shown this problem on my undergraduate course on Markov chains, but my teacher did not tell me the solution. Does anyone know the solution or references to the problem?
Comment: You can probably consider a Markov chain on $\mathbb{Z}^{N+1}$ starting at $(0,2, ..., 2)$ and evolving under a probability transition kernel correctly chosen. Then you're looking at the entrance time to some subset. Just a few ideas, though, I don't know the answer.
Comment: Crossposted to MO: http://mathoverflow.net/questions/106133/random-walk-police-catching-the-thief
Answer: First some simplifications. Pretend that the origin moves with the thief; then we can say that the thief always stands still, while each policeman (rather, each police officer) moves 2 spots left with probability $1/4$, moves 2 spots right with probability $1/4$, and stands still with probability $1/2$. Now we might as well scale down by a factor of 2, so that the moves to the left and right are 1 spot each (and the officers start at 1 instead of 2, though that won't affect whether the expectation is infinite). Finally, if you like, ignore the possibility of standing still and just let each officer random walk on $\mathbb Z$ with probabilities $1/2$ to the left and right: the standing still just multiplies the expectation by $2$, the expected time it takes to move a spot.
So we've reduced to the following problem: $N$ police officers do a standard random walk on $\mathbb Z$, starting at 1. Is the expectation of the time it takes for one of them to hit the origin finite or infinite?
A general identity will be useful here. Let $X_k$ be a sequence of independent coin flips (not necessarily fair or identical coins) with possible outcomes S (Success) or F (Failure). Let $FS(n)$ be the probability that the $n$th flip is the First Success (that is, $X_1=\cdots=X_{n-1}={}$F and $X_n={}$S). Let $NSY(n)$ be the probability that there have been No Successes Yet by the $n$th step (that is, $X_1=\cdots=X_{n}={}$F). Then the expectation of the first success is
$$
\sum_{n=0}^\infty n FS(n) = \sum_{n=0}^\infty n \big( NSY(n-1)-NSY(n) \big) = \sum_{m=0}^\infty NSY(m)
$$
after telescoping the sum.
Now I believe (but someone should verify) that in a standard random walk on $\mathbb Z$ starting at 1, the probability that 0 has not been hit yet by the $n$th step is asymptotically proportional to $1/\sqrt n$. (This probably has to do with Catalan numbers.) This is one way of seeing that $E\tau_1$ is infinite: it's summing the infinite series $NSY(m)$ where each term is essentially a constant times $1/\sqrt m$.
In the $N$-officer case, the No Success Yet probability is just the $N$th power of the No Success Yet probability when $N=1$. Therefore for $N=2$, the series will still (barely) diverge, but it will converge for $N=3$; in other words, $E\tau_2 = \infty$ but $E\tau_3<\infty$.
Comment: The problem that I have with this solution is that the $N$ police-thief systems are **not** independent. So I'm not sure that the first simplification is valid.
Comment: I should add that this is a correct solution when the thief is stationary, and I'll be willing to bet that the answer is the same for the moving thief. It's just that the argument seems incomplete.
Comment: Aha. For example, it's impossible (in the post-first-simplification description) for one officer to move 2 spots left while another moves 2 spots right. Not independent - good point.
Answer: (I do not have enough reputation to leave comments, so I am writing in a new post)  Using Greg Martin's argument and an additional coupling argument (using the fact that at any time $n$, with probability $>1/2$, the random walk corresponding to the thief is at a nonnegative point at time $n$) you can show that $\mathbb{E}\tau_2 = +\infty$ as well.
His comment about the Catalan numbers is also correct; the only way to drop below $0$ for the first time on step $2j+1$ is to have a Dyck path of length $2j$ followed by a downward jump; this means that the corresponding probability is about $Cj^{-3/2}$, and summing over $j$ from $1$ to $n$ gives $n^{-1/2}$.
I don't yet know how to show that $\mathbb{E}\tau_3<+\infty$, though.

Random walk: police catching the thief


Ler algo depois de uma palavra-chave C#

Question: Eu tenho a seguinte situação, tenho um documento(txt) que tem algumas coisas 'preenchidas' tipo:
Nome:"João"; Idade:"20"; Estado:"SC";

O resultado que eu queria é que as variáveis tivessem somente o que está escrito entre as aspas depois de cada palavra chave.
string Nome; //Receberia João

int Idade; //Receberia 20

string Estado; //Receberia SC

OBS: Esse documento é gerado em uma única linha por outro programa, gostaria de usar isso para preencher alguns campos automaticamente.
Comment: Existe a possibilidade de deixar esse TXT em formato JSON? Se sim, ficaria muito mais fácil com o `Newtonsoft`
Comment: Ronaldo Araújo Alves, a extensão do arquivo é .info mas todas as formas de abrir mostrar como se fosse um txt
Answer: Use a expressão regular [^\\s;\\\"]+\\\"[^\\\"]+\\\" para pegar as entradas do tipo <identificador>:<valor>.
Separe <identificador> de <valor> com String.Split(':'); e coloque-os em um dicionário onde a chave é <identificador> e o valor é <valor>.
Do dicionário fica mais fácil. Você pode popular variáveis, classes, vetores,... do jeito que quiser.
No Repl.it
using System;
using System.Collections.Generic;
using System.IO;
using System.Text.RegularExpressions;

public class Analisador
{
   public static Dictionary<string,string> Parse(string entrada)
   {
     Dictionary<string,string> resultado =new Dictionary<string,string>();
      MatchCollection matches = Regex.Matches(entrada, "[^\\s;\\\"]+\\\"[^\\\"]+\\\"");
      foreach (Match match in matches)
      {
        string[] arr = match.Value.Split(':');
        resultado.Add(arr[0], arr[1].Replace("\"", ""));
      }
      return resultado;
   }

   public static void Main()
   {
      Dictionary<string,string> Valores = Parse("Nome:\"João\"; Idade:\"20\"; Estado:\"SC\";");

      foreach(var item in Valores)
      {
        Console.WriteLine($"{item.Key} = {item.Value}");
      }
   }
}

Comment: Obrigado Augusto Vasques, criei um projeto separado para testar seu código e funcionou perfeitamente, vou tentar implementar ele no meu projeto mas valeu mesmo pela sua ajuda:)
Comment: Consegui implementar e funcionou mas ainda não entendi muito bem essa coisa dos dicionários para esse fim, você poderia me indicar um lugar para entender melhor?
Comment: https://docs.microsoft.com/pt-br/dotnet/api/system.collections.generic.dictionary-2?view=netframework-4.8
Comment: Exemplo bem simples usando o dicionário Valores : `String nome = Valores["Nome"];  Console.WriteLine("O nome é " + nome);`

Ler algo depois de uma palavra-chave C#


Standard Error of the cumulative value for time series

Question: I have two time series, as in the picture below. The data was gathered experimentally. A practical example could be a measured mass flow rate, where I measure the mass flow rate over a certain time period with changing boundary conditions, and I am interested in the total mass of fluid consumed during this period.

The depicted intervals represent the Standard error of the mean and were determined through ten repeated measurements of the entire cycle. For each time step, the  interval equals:
$ SEM= \frac{\sigma}{\sqrt{10}}$
The ten replications naturally constitute ten identifiable time series.
I now have three questions:

How can I assess, at a specific time step, if the difference between the means of the time series is statistically significant? t-test?

How can I determine the SEM of the cumulated value for each time series?

How can I assess if the difference between the cumulated values of both time series is statistically significant?

Does it make sense to create an "interval" of the cumulative value of both time series by cumulating the upper bound values and lower bound values?
Answer: 1) One could estimate a model for each of the ten separately and then estimate the parameters globally across all 10 items leading to an F test. Do this for each time step.
2) You can use Monte Carlo techniques (boostrapping) to obtain density functions for the next k periods and then simply sum the pseudo observations over time to get the probability density function of the sum and then mark of probability limits. This is a backdoor way to create the distribition of the sum or correlated values. I have implemented this in AUTOBOX, a piece of software that I helped to develop
3) By examining the coefficients in the identified models for each NOT in the sum of the forecasts.
Comment: As you said each of the 10 realizations would/must have their own thread or history and would be ten identifiable time series.
Comment: @whuber : I edited the question to stress that the replications naturally constitute ten identifiable time series and not independant measurements at each time step.
Comment: Could you indicate how the methods you mention are able to take advantage of the 10 replicates of each measurement?
Comment: Tks .. modified my first answer .
Comment: Wouldn't estimation of ten separate models involve arbitrary assignments of each replicate to a model, thereby increasing the variability and reducing the quality of the fit?  I guess the issue comes down to whether the replications naturally constitute ten identifiable time series or are genuinely independent replications at each step.
Comment: @IrishStat : Thank you for your answer. What type of model do you mean ? Something like ARIMA ?
Comment: 4 sure . possibly. leading to a form of an ARMAX model see
 https://stats.stackexchange.com/questions/397240/forecasting-methodology/397267#397267
Answer: 1) You can not use a bunch of pair-wise t-tests because this will massively increase the likelihood of a type 1 error. You need to perform a 2-step procedure to avoid this:
Step 1. If your null hypothesis is that all the means are equal, and the alternative is that the means are not equal, first use a 1-Way ANOVA Test (similar to an F-test).
Step 2. If you reject the null, then you need to know which means are different and by how much. For this, you can use a Post-Hoc procedure for pair-wise comparisons. I would suggest Tukey's HSD, Bonferroni, or Mann-Whitney u test. All of these essentially to a something like a t-test but against a different distribution which avoids inflating the Type 1 errors. The Mann-Whitney u test is a non-parametric test so maybe more robust.
2) As suggested by IrishStat, Bootstrap is the way to go here as the sample size is small. If you looking for CI on the errors, you may need bias-corrected and accelerated bootstrap interval as the distribution may be skewed.
3) Create another test statistics called the difference in cumulative values, and bootstrap this statistic for inference.
Comment: Thanks for your input, I will definitely look into bootstrap methods.
What is your opinion about the last question where I create an interval for the cumulated mean value, by cumulating all the lower bound values and al the upper bound values ?
Comment: It seems that this would be a way to create a confidence interval for the cumulated mean but I doubt it is an efficient estimator for the interval due to the use of min/max statistics. Bootstrap intervals will be easier to interpret.


Sum of $k$ consecutive integers

Question: I'm trying to prove or disprove:
Characterize $k \in\Bbb N$ that makes the following statement true: "$a$ is
divisible by $k$ if and only if $a$ can be written as a sum of $k$ consecutive integers".
I tried the converse:

Proof. If $k\mid a$ and $k=2q+1$, $q \in\Bbb Z $, then $a=c(2q+1)$ for $c \in\Bbb Z$.
Notice: $$\begin{align*}ck &= (c - q) + \big(c - (q-1)\big) + \ldots+ (c -1) + c\\&+\, (c +1) +\ldots+ \big(c + (q-1)\big) + (c + q)\;,\end{align*}$$ so $ck$ is the sum of the $2q+1$ consecutive integers.
Hence $a$ is expressable as a sum of $k$ consecutive integers

Is this okay? Other methods?
Comment: I guess you meant "I tried the forward direction."
Comment: For the first part where you say $k|a \implies k=2q+1...$ is wrong. It should be $kq=a$ for some $q\in\mathbb{Z}$. And maybe have two cases one for when $k$ is odd and then when $k$ is even.
Answer: Since the statement you are trying to prove contains "if and only if", you also need to prove
"if $a$ can be written as the sum of $k$ consecutive integers, $a$ is divisible by $k$."
Also, I'm not sure if it's just me being tired, but I'm not following your 'Notice:' line of reasoning.
Comment: The reasoning is correct; the OP omitted a lot of parentheses that would have made it much clearer. I've taken the liberty of inserting them.
Answer: You were asked to find a characterization of those positive integers $k$ that have a certain property; this means that you're to find and prove a statement of the form

a positive integer $k$ has the specified property if and only if $k$ is something.

What you've proved is that $k$ has the desired property if $k$ is odd; to complete the characterization you must show that if $k$ is even, it does not have the specified property. That means that you must show that if $k$ is an even positive integer, then there is some integer $a$ such that either

$k\mid a$, but $a$ cannot be written as the sum of $k$ consecutive integers, or
$k\nmid a$, but $a$ can be written as the sum of $k$ consecutive integers.

HINT: What if $a=k$?

Sum of $k$ consecutive integers


Playing with the torus and semisimplicial sets (prove that $\phi$ and $\psi$ are not homotopic)

Question: Recall that we can express the torus $|X.| \cong T$ as a square with edges $e$ and $f$, diagonal $g$, faces $T_1$ and $T_2$, and a single vertex $v$, and appropriate identifications.
Let $Y.$ be the semisimplicial set where $Y_1 = \{a, b\}$, $Y_0 = \{u\}$, and $Y_n = \emptyset$ for all $n \geq 2$. We can consider two maps as follows:
$\phi: Y. \rightarrow X.$ defined by $a \mapsto e$, $b \mapsto f$.
$\psi: Y. \rightarrow X.$ defined by $a \mapsto e$, $b \mapsto g$.
Which map from the "figure-eight" to the torus. The first map seems to take the figure-eight to the traditional two non-trivial loops (going inside the hole then back out, and going around the equator). The second map, on the other hand, seems to take the first loop to the same loop on the torus, but the second loop to a combination of the first and second (a kind of diagonal loop composed of one of each of the non-trivial loops from before).
My goal is to compute the kernel and cokernel of $\phi_*$ and $\psi_*$ for each of $H_0, H_1, H_2$, and also to prove that $\psi$ and $\phi$ are not homotopic (which seems intuitively obvious, but for which the proof is escaping me).
Comment: Oh, okay. Computing the kernel and cokernel (at least, the isomorphism types) will not suffice to answer the latter part of your question but computing the *image* of the map on $H_1$ does.
Comment: Could you let me know what you are using for a reference for this material?  For example, have you learned a way to compute the induced map on homology from a simplicial map? If you have that, you can compute $\phi_*$ and $\psi_*$.
Comment: Could you let me know what reference you are using for your definitions?
Comment: I'm familiar with these kinds of arguments, but it'd be helpful to be walked through it again (it's been a long time). I've thought about it some more, and I'm pretty sure that $\phi_*$ is surjective but not injective...
Comment: Hatcher, for the most part.
Answer: For the figure eight $\large\infty$ the chain complex is
$$0\to \langle a,b \rangle\to \langle u \rangle\to 0$$
where all maps are $0$, so the these cycles also generate the homology groups
$$H^Δ_2=0,\quad H^Δ_1=⟨a,b⟩=\Bbb Z^2, \quad H^Δ_0=⟨u⟩=\Bbb Z$$
For the torus $T^2$ the chain complex is
$$0 →⟨T_1,T_2⟩ →⟨e,f,g⟩ →⟨v⟩ →0$$
where only the first map is non-trivial and sends $T_i$ to $e+f-g$. The simplicial homology groups are
$$H^\Delta_2= ⟨T_1-T_2⟩=\Bbb Z,\quad H^Δ_1=\frac{⟨e,f,e+f-g⟩}{⟨e+f-g⟩}=⟨e,f⟩=\Bbb Z^2,
\quad H^Δ_0=\Bbb Z$$
To show that $\phi_*,\psi_*$ are not homotopic, consider the commutative square
$$\begin{array}{ccc}
H^\Delta_n(\infty) & \to & H_n(\infty)\\
\downarrow & & \downarrow\\
H^\Delta_n(T^2) & \to & H_n(T^2)
\end{array}$$
The horizontal maps are isomorphisms between the simplicial and the singular homology groups, and these morphisms respect the maps $\phi_*$ and $\psi_*$. But homotopic maps induce the same maps on singular homology groups, thus also on simplicial groups. However, $\phi_*$ and $\psi_*$ are not equal. The first sends $a$ to $[e]$ and $b$ to $[f]$, and is an isomorphisms. The second one sends $a$ to $[e]$ and $b$ to $[g]$ which is equal to $[e+f]=[e]+[f]$, and is an isomorphisms, too, but a different one.


Are those users bots?

Question: It's not the first time I noticed that.
Users with the consecutive visit badges but no activity at all on the website
See this user: https://magento.stackexchange.com/users/36497/raju?tab=activity
Are those bots ?
Comment: @QaisarSatti http://meta.stackexchange.com/questions/66482/consecutive-days-problem-97/66485#66485 'login pages, along with a number of other homepage style pages, don't count as "access" '
Comment: http://stackoverflow.com/users/5832518/raju -- that's his profile on Stack Overflow
Comment: He earned "Fanatic" Badge, its really need lot & lotttt of dedication to earn that badge :-) As he earned 101 reputation, i guess he may be a regular member of other stackexchnage site.
Comment: I think for `Fanatic` badge activity is not necessary.
Comment: Im pretty sure you have to browse questions and be a little bit active to get the badge I'm pretty sure opening the website and not doing anything won't get you the badge
Comment: @RaphaelatDigitalPianism I'm pretty sure opening the website will do that.
Comment: @QaisarSatti nope it doesn't, I'm 90% sure you have to browse questions to be considered as active. Simply opening the website would not work
Comment: @RaphaelatDigitalPianism  Visit the site each day for 100 consecutive days.
Comment: @RaphaelatDigitalPianism already opening and visiting counts for fanatic. There is no additional activity needed. i already had 75+ days but "lost" a day, so i had to "restart". Hopefully i won't miss a day in the next 6-7 days :)
Answer: The user has 101 points and a fanatic badge on 143 SE websites (has a few more points on some of them).
I guess he has a script that opens automatically 2 or 3 pages each day on each website. Or maybe he does it manually? but I doubt it.
I can definitely say he's not a bot.
Here is one of his answers from meta.stackexchange.com
https://meta.stackexchange.com/a/277658/224252
Comment: You say, "I guess he has a script that opens automatically 2 or 3 pages", but then say "I can definitely say he's not a bot". What is your definition of a bot? At the very least you suggest he's partly using the account as a bot, correct?
Comment: You can use scripts and still be a person. This could be as simple as an auto-refresh browser plugin.
Comment: He may be a person, but doesn't having a script running for badge considered fraud and thus make him liable to a ban?
Comment: @JulienLachal. It's not moral, but it's not illegal either. I will not ban a user just for this. He is not banned on the other 143 websites either.
Comment: Would he have earned vote-related badge using the same method, would you still not ban him?
Comment: because he earned a badge, no...because of fraudulent points, yes.

Are those users bots?


SQL Self Join Update

Question: I need to procedurally update a table by self-joining itself. Using SQL Server 2019.
CREATE TABLE Sect
(
     Section_Id INT,
     Locale VARCHAR(10),
     Record_Id INT,
     Section_Id_1 INT
);

INSERT INTO Sect (Section_Id, Locale, Record_Id, Section_Id_1)
VALUES
(100, 'US', 1, Null),
(101, 'CA', Null, 100),
(101, 'MD', Null, 100)

The goal is to update the null values of Record_Id where Section_Id_1 equals Section_ID with the matching Record_ID.
This is the intended result:
100|US|1|Null
101|CA|1|100
101|MD|1|100

I think I am close with:
UPDATE t1
SET Record_Id = t2.Record_Id
FROM Sect t1
INNER JOIN Sect t2 ON t1.Section_Id_1 = t2.Section_Id
WHERE t1.Record_Id IS NULL

Appreciate your all's help.
Comment: We need a [mre] because nothing in your question mentions `crossRecord_Id` - one might guess you are creating a stored procedure, and that you have attempted to create it a second time when it already exists. But without seeing exactly what you are trying to do its all a guess. Is that what you meant by "procedurally update"? Because normally procedural is the opposite of set-based which is the opposite of a relational database.
Comment: then show us on Fiddle
Comment: Your query looks correct.
Comment: What is your question actually ?
Comment: How do I get the "intended result"?
Comment: isn't your query already doing this ?
Comment: It's not on SQL Fiddle.
Comment: I just don't get it; still not working. I tested on SQL Server with the following error:
Msg 2714, Level 16, State 3, Procedure crossRecord_Id, Line 6 [Batch Start Line 15]
There is already an object named 'crossRecord_Id' in the database.
Comment: Dale- Thank you. You made me look more closely and most of the error was (sleep deprivation) related to the SQL Server stored procedure that I had built. I kept looking at the live data, when I had made a test copy to play with... Such is life.
Answer: I'm not familiar with SQL Server 2019. But I think you don't need to use INNER JOIN, try this.
update
  Sect t1
set
  Record_Id = (select t2.Record_Id
               from Sect t2
               where t2.Section_ID = t1.Section_Id_1
                 and t2.Record_Id is not null
               limit 1)
where Record_Id is null
  and Section_Id_1 is not null

Comment: Thank you for the valuable feedback. I just needed sleep.
The final procedure is outlined as below.
...
UPDATE t1
SET Record_Id = t2.Record_Id
FROM Test_Section t1
INNER JOIN Test_Section t2 ON t1.Section_Id_1 = t2.Section_Id
WHERE t1.Record_Id IS NULL
...

Enjoy your day!

SQL Self Join Update


Subgroups of small groups

Question: Is there a book/website where I can find the subgroups of small groups (maybe up to order 20)? I'm working on finding the subgroups of some finite groups for Galois theory exercises and would like to check whether I found all of them. Thank you in advance!
Comment: Cf. [this](https://groupprops.subwiki.org/wiki/Main_Page)
Answer: Tim Dokchitser's website GroupNames has amazing amounts of information about groups of order up to 500 (except those of order 256 or 384), including diagrams of subgroup structure where you can hover your cursor to get basic information about each subgroup.

Subgroups of small groups


Compiler errors for GCC (via CUDA) intrinsic functions, but I'm not using any

Question: Problem
I wrote a bunch of small CUDA programs.  Most of them compile fine in debug and release builds.  However, a few fail when compiled in Release mode because various GCC intrinsics are being given the wrong types of pointers.  But I'm not actually using intrinsics.  This program partially reproduces my problem:
#include <iostream>
#include <cuda_runtime.h> // To pacify the syntax highlighter
#include <immintrin.h> // NOTE: I don't ever include this header in my real code

__global__ void kernel() {
  // Do nothing
}

using namespace std;

int main() {
  kernel<<<1, 1>>>();
  cout << "Hello, world!" << endl;

  return 0;
}

The problem, however, is that in my actual code I do not include <immintrin.h> or use GCC intrinsics of any kind.  It's possible that  some library code I use does, but I don't know for sure.  If I remove <immintrin.h> from this example, the program compiles and runs fine.
The actual offenders are here and here, if you want to see them.
Relevant Facts

I am using the following software:

Ubuntu 17.04
nvcc version 8.0.44
gcc version 5.4.1
cmake version 3.8.20170418

The projects build and run perfectly fine in Debug mode, including the sample program above.
Most of my small CUDA programs compile fine in Release builds, but I can't identify any patterns among the ones that fail.
The Release build compilation command is as follows:

/usr/bin/g++-5  -std=c++11 -fopenmp -O3 -DNDEBUG  -rdynamic CMakeFiles/DotProduct.dir/DotProduct_generated_main.cu.o CMakeFiles/DotProduct.dir/DotProduct_intermediate_link.o  -o DotProduct -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -ldl -lrt ../../Common/libCommon.a -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -ldl -lrt -Wl,-Bstatic -lcudadevrt -Wl,-Bdynamic -L/usr/lib/x86_64-linux-gnu  -lSDL2 -lSDL2_ttf -lSDL2 -lGLEW -lGLU -lGL

The full build log is too large to attach, but can be found here.  Here's a relevant sample of the errors:

/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9533): error: argument of type "void *" is incompatible with parameter of type "long long *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h(9542): error: argument of type "void *" is incompatible with parameter of type "long long *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(54): error: argument of type "const void *" is incompatible with parameter of type "const long long *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(62): error: argument of type "const void *" is incompatible with parameter of type "const int *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(70): error: argument of type "const void *" is incompatible with parameter of type "const long long *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(78): error: argument of type "const void *" is incompatible with parameter of type "const int *"
/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h(86): error: argument of type "void *" is incompatible with parameter of type "const long long *"

The error log does not say who is actually using these intrinsics.
As it turns out, the sequence of headers is as follows: <random> -> <opt_random.h> -> <x86intrin.h> -> <immintrin.h> -> (every other header mentioned in the error log).  My new goal is now to enable all the usual optimizations except those which intrinsics.
Comment: I will repeat myself more explicitly. No version of CUDA 8 supports Ubuntu 17, nor does it support the canonical gcc 5.4-1 build you are using
Comment: you need to add explicit casts to the incompatible types, i.e. myLongLongPtr = (long long *)myVoidPtr;
Comment: @Serge Where?  I'm not actually using those functions and I don't know who is.
Comment: CUDA doesn't support your compiler version or operating system. http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#axzz4lB5unFj4
Comment: Upgrading to CUDA 8.0.61 doesn't help...
Comment: ...So I have to downgrade my Ubuntu installation?
Comment: Everything is guaranteed to work if you use a supported operating system and host compiler. It isn't guaranteed to work if you don't.
Comment: I'm trying something with docker.  If it works out I'll answer my own question with my solution.
Answer: It turns out that this is likely an nvcc bug stemming from CUDA's stated lack of support for my particular system configuration.  I filed a report here (you need to be logged in to see it).
For now, I worked around it by not using anything that requires intrinsics.  In my case I used Thrust's random number generators instead of the standard library's.  Someone I talked to suggested that I could also separate my host and device code more carefully such that the source files processed by nvcc don't ever include <immintrin.h>.  Haven't tried it but for those in the future who see this, it's worth a shot.
Answer: The problem isn't with GCC, it is simply nvcc being not compatible with gcc.
The following is a simple walkaround without messing with system file.
It should be safe to do it if your CPU doesn't have avx512 support.

copy immintrin.h from /usr/lib/gcc/x86_64-linux-gnu/5/include/ to wherever you want.
comment out all includes of avx512*.h.
supply the include directory to host compiler (do it with nvcc and then propagate it to gcc) and override the system one.

You can use the following cmake code to safely do so.
# Otherwise, avx512*intrin.h error: argument of type "const void *" is incompatible with parameter of type "const float *", etc...
if(CUDA_VERSION STREQUAL "8.0" AND CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
  execute_process(COMMAND ${CUDA_HOST_COMPILER} -dumpversion OUTPUT_VARIABLE
                  TEMP_CUDA_HOST_COMPILER_VERSION)

  if(${TEMP_CUDA_HOST_COMPILER_VERSION} VERSION_EQUAL "5.5" OR
     ${TEMP_CUDA_HOST_COMPILER_VERSION} VERSION_GREATER "5.5")
    # maybe simply downgrade the host compiler
    # set(CUDA_HOST_COMPILER "/usr/bin/gcc-4.9")
    set(CUDA_NVCC_FLAGS "-Xcompiler -I${CMAKE_CURRENT_SOURCE_DIR}/include/fix-cu80-gcc5;${CUDA_NVCC_FLAGS}")
    message(WARNING " CUDA 8.0 does not works well with GCC == 5.5.0 due to AVX512 intrinsics.\n"
                    " Will use\n"
                    "   include/fix-cu80-gcc5/immintrin.h\n"
                    " as a replacement of\n"
                    "   /usr/lib/gcc/x86_64-linux-gnu/5/include/immintrin.h\n")
  endif()
  unset(TEMP_CUDA_HOST_COMPILER_VERSION)
endif()

If your CPU does support avx512, you might need supply a bunch of -mno-avx512f -mno-avx512pf ... flags to explicitly disable them.
Comment: `-march=native -mno-avx512f` should disable all other AVX512 extensions; they others all depend on the AVX512-Foundation extension.  Just like `-mno-avx` also disables AVX2 and FMA.  Unless nvcc works very differently from gcc/clang options...


tried to update angular to 13 broke a bunch of stuff now trying to go back to angular 11 how to?

Question: First I am the only one working on this project so we can completely nuke everything but the code of course.
I have tried things like this with no effect
npm uninstall -g @angular/cli
npm cache clean
npm install -g @angular/cli@11

when I try to run ng serve the current situation is like this
bullshit@ChristohersMBP2 decksite % ng serve
This version of CLI is only compatible with Angular versions ^13.0.0,
but Angular version 12.2.16 was found instead.

Please visit the link below to find instructions on how to update Angular.
https://update.angular.io/

What I did do that worked for a little is revert back to a old commit in my project but now that is not an option any more.
so my thoughts are to clear my package.json and delete package-lock.json  and delete node_modules
and try to install the angular 11 cli again (as well as all other npm libraries i need since I no longer have my package.json populated.
Or even better go grab the package.json from a older commit since that is still an option and then do whatever I have to do on my local to completely wipe everything that is not source code. Delete current package.json , package-lock.json, node_modules, and run npm uninstall on everything? IDK whatever I have to do to wipe.
Place the new package.json in my angular project and then hit the npm install command.
that seems like a really good solution, am I hitting in the right place?
Answer: I typically do the following via (ng update):

Make sure you are fully committed and functioning.  Make a branch 'angular_13_update'
Run ng update, it will come back with a report of things to update.
I commonly npm install @angular/x, y, z per the list.
See how it goes.  Using 3rd party stuff sometimes leads to some complaints. --force can some times get that part done.
If that bit succeeds test a build and see what broke as a result.

I believe from 11 to 13 you are going by Webpack 5 updates and other breaking changes. So depending on how much 3rd party i.e. commonjs style packages you have you may need to tweak your angular.json file with architect->build->options:
"allowedCommonJsDependencies": [
                            "uuid",
                            "ulid",
                            "url",
                            "lodash/get",
                            "lodash/isEmpty",
                            "lodash/isEqual",
                            "@aws-amplify/core",
..etc


Trying to center 3 div blocks alongside each other with gaps between, they are aligned next to each another horizontally

Question: #block1 {
    background-color: blue;
    font-family:"Arial Black", Gadget, sans-serif;
    width: 25%;
    padding: 4px;
    border-radius: 1em;
    margin-left: auto;
    margin-right: auto;
    float: left;
    display: block;
}

#block1 p {
            color: white;
}

#block2 {
        background-color: blue;
        font-family:"Arial Black", Gadget, sans-serif;
        width: 25%;
        padding: 4px;
        border-radius: 1em;
        margin-left: auto;
        margin-right: auto;
        float: left;
        display: block;
}

#block2 p {
            color: white;
}

#block3 {
    background-color: blue;
    font-family:"Arial Black", Gadget, sans-serif;
    width: 25%;
    padding: 4px;
    border-radius: 1em;
    margin-left: auto;
    margin-right: auto;
    float: left;
    display: block;
}

#block3 p {
        color: white;
}

As you can see I've set up the three blocks to align next to each other but I'm just struggling to centre them with spaces between.
I've set it so that the blocks all float alongside each other but they simply go to the left as designated with the float left, is there a way I can make them centred?
Comment: Don't use `float`, use `display:inline-block` and on the parent container add `text-align:center` - http://jsfiddle.net/fku0h4b1/
Answer: Don't use float, use display:inline-block and on the parent container add text-align:center

.container {
  text-align: center;
}
.block {
  background-color: blue;
  font-family: "Arial Black", Gadget, sans-serif;
  width: 25%;
  padding: 4px;
  border-radius: 1em;
  display: inline-block;
  height: 250px;
  margin: 0 10px;
}


Implementing Stochastic Gradient Descent with both Weight Decay and Momentum

Question: So I'm trying to implement a neural network using only numpy module in Python. The problem I'm facing is related to the correct implementation of the regularization through weight decay, and also the momentum in stochastic gradient descent.
When I only perform sgd without any weight decay and momentum my neural net performs well on the application scenario I'm facing. If I try to add either of these two terms, my model performs really badly. So I suppose that I'm not understanding them well.
Since my model works fine with sgd, let us assume that my model do it's job in forward and back propagation.
The problematic aspect arise in the implementation of the parameters update for a layer of my net.
The method for the parameters update is the following:
def update_params(self, n_samples):
        weight_decay_term = (self.learning_rate * self.weight_decay / n_samples) * self.W
        weight_velocity = self.momentum * self.weight_velocity + self.dW
        self.W = self.W - weight_decay_term - self.learning_rate * weight_velocity
        self.weight_velocity = np.copy(weight_velocity)
        bias_velocity = self.momentum * self.bias_velocity + self.db
        self.b = self.b - self.learning_rate * bias_velocity
        self.bias_velocity = np.copy(bias_velocity)

where 'n_samples' is the size of the mini-batch.
For the weight decay formula I followed the guideline described in:
https://github.com/limberc/deeplearning.ai/blob/master/COURSE%202%20Improving%20Deep%20Neural%20Networks%20Hyperparameter%20tuning%2C%20Regularization%20and%20Optimization/week%2001/week%2001%20Setting%20up%20your%20Machine%20Learning%20Application.pptx
For the momentum formula I followed the guideline described in:
https://cs230.stanford.edu/files/C2M2.pdf
I also tried the formula described in:
Neural Networks: weight change momentum and weight decay
without any success.
None of these solutions worked, meaning that setting for example:
self.learning_rate = 0.01
self.momentum = 0.9
self.weight_decay = 0.1

my model performs really badly. I suppose it is related to my understanding of the implementation details of weight decay and momentum, but I really can't wrap my head around this problem. If I set the momentum and weight_decay parameters to zero I get acceptable results with my model, which are similar to those that I obtain with a pytorch model with the same architecture.
I tried to instantiate a pytorch multy layer perceptron with the same architecture that I tried with my model, and used as optimizer:
torch_optimizer = torch.optim.SGD(torch_model.parameters(), lr=0.01, momentum=0.9, weight_decay=0.1)

and the torch net performs greatly on my application scenario.
So, can you spot the problem on my implementation of the update parameters method?
Thank you for reading
Answer: In short, the n_samples in OP's weight_decay_term equation should be the size of the whole training set instead of mini batch. It was the same error I made.
Below are the explanation of my mistakes and corrections:
I had the same problem, restudy the equations, and found that in my coding, I mixed up the n, size of the whole training set, with the m, size of the mini batch.
Based on my understanding, for the weight decay term calculation, n, size of the whole training set should be used.
Whereas, m, size of the mini batch is used in the approximation of gradient of cost function w.r.t weights or biases using mini batch, which are $\frac{1}{m}\sum_x\frac{\partial C_x}{\partial w}$ and $\frac{1}{m}\sum_x\frac{\partial C_x}{\partial b}$, and I assumed that you have covered in your self.dw and self.db calculations as it's not shown in your codes.
I am following Michael Nielsen's convention so the terms might be different from yours but I think they are the same concept. With some algebra, my equation derivations are as follows:
For weights:
$$
v\rightarrow v'=\mu v-\eta\nabla C
\\\therefore v\rightarrow v'=\mu v-\frac{\eta}{m}\sum_x\frac{\partial C_x}{\partial w}-\frac{\eta\lambda}{n}w
\\w\rightarrow w'=w+v'
$$
For biases:
$$
v\rightarrow v'=\mu v-\eta\nabla C
\\\therefore v\rightarrow v'=\mu v-\frac{\eta}{m}\sum_x\frac{\partial C_x}{\partial b}
\\b\rightarrow b'=b+v'
$$
where,
$\nabla C$ = gradient of cost function w.r.t weights or biases,
$\mu$ = momentum coefficient, $\eta$ = learning rate, $\lambda$ = regularization parameter in weight decay term,
$m$ = size of mini batch, $n$ = size of the whole training set
After the correction, with $\mu=0.5$, $\eta=0.1$, $\lambda=5$, $m=10$, the model managed to get around 97% of final evaluation accuracy with 100 epochs in the MNIST problem. $n$ will be size of the MNIST training set, which is $50000$.
Please let me know if I made mistakes in the explanation above, coz I am new to neural networks.
Comment: This might have helped you, but can you be specific about how the exposition here fixes the problems in OP's implementation? As it stands, it's not obvious how this addresses the question.
Comment: Sorry that my explanation was a bit confusing. In short, the **n_sample** in OP's **weight_decay_term** equation should be the size of the whole training set instead of mini batch. It was the same error i made. The rest of the explanation was where i learned those implementation.
Comment: Perhaps you could [edit] to plainly & clearly state in the answer what the mistake is and how to fix it.
Comment: In OP's code, `n_samples` is only used in `self.weight_decay / n_samples`. Suppose you have $C = \frac{ w}{ n}$. Then we know there must be some scalar $k$ so that $C=\frac{ w}{ n} = \frac{wk}{m}$. In other words, the difference between using $m$ and using $n$ is that you've adjusted $w$ by a factor of $k$. If this is OP's problem, then solving for $k$ and plugging it in should solve it. But if it doesn't, then we know that $m$ vs $n$ is not the issue.


How can we visualize multiple regression with 3 or more continuous variables or with categorical variables

Question: I'm trying to understand multiple regression visually.
So far I understand that when you have two continuous variables explaining y, this will take the shape of a plane (see the image).

What happens if you have 3 continuous variables? Does this take the shape of a cube or rectangular prism? What about 4 continuous variables? I don't think we can visualize 4 dimensions? As a size note, I am aware that I can use Level plots. But I still wanted to build from the plane example.
Also, how can we imagine adding a categorical variable? How would that effect the plane.
Using the example from the image bellow, we could add the "make of the car" to the model.
"Make" could be a vector contaning ("Toyota", "Subaru", "Chevrolet")
So we have MPG~Weight+Horsepower+Make.
Would "Make" be a new axis? Or would it instead be a group of points. For example Toyota could be red points, Subaru = blue points, and Chevrolet being black points?
Thank you!
Answer: 3 continuous variables: make a row of such plots at 3-5 defined levels (or 'slices') of the third variable.
4 continuous variables: make a column of such plots at 3-5 defined levels of the fourth variable.
I would also consider using colour for the third dimension, though it makes it  much harder to show the points in addition to the fitted surface. This can be partially ameliorated by showing an additional plot of observed vs. fitted values.
Here's an example from a recently published paper that shows the output of a theoretical model, but the same approach can be used for a statistical model.
(https://aslopubs.onlinelibrary.wiley.com/doi/abs/10.1002/lno.12040):

In principle this plot matrix approach can be used with the 3D plots too, but it will likely become too crowded.
Categorical variable: same as above, it's just 2 panels (for a binary variable, more otherwise) instead of 3-5 levels as in the case of the continuous variable. Or if you use the 3D plots instead of colour, you may be able to squeeze 2-3 categories into the same panel using different colours.
What about 5 continuous variables? Either make several sets of these plot matrices at 3-5 defined levels of the fifth variable, or animate each panel in the single plot matrix i.e. use time as the 5th dimension. These options will almost never work well, but there will be a few datasets for which it's possible.
Answer: You usually use so-called matrix plots, a collection of 2-dimensional scatter plots. Categories are usually represented by different colors of the data points.
Answer: I have used matrix plots before, but I also used another way. I made a regression plot between the measured variable and the main input variable. Then I took the residuals, and plotted those as a function of the second most important variable. And so on.
I feel this is a nice graphical representation of how much "value" you get from each step in making the model more complicated. I personally find 3D plots of any kind difficult to read and interpret, so my choice is very much dependent on my personal aesthetic preferences.
I obviously do present the regression results for the "all at once": THAT is the analysis, everything else is pretty pictures for illustrative purposes.
Main Trouble with this approach is that a lot of people pretty much ignore entire sections as soon as the word "residual" comes up. It's up to me to educate these people ;)
Comment: When there are not too many interactions, [nomograms](https://en.wikipedia.org/wiki/Nomogram) can be useful.  See also [this](https://stats.stackexchange.com/questions/155430).


Paypal PHP SDK implementation

Question: My paypal sdk version is ^1.13
I'm having errors in my paypal implementation. I followed this tutorial
https://www.youtube.com/watch?v=BD1dOWIABe0
<code><?php
require 'vendor/autoload.php';
use PayPal\Api\Payer;
use PayPal\Api\Item;
use PayPal\Api\ItemList;
use PayPal\Api\Amount;
use PayPal\Api\Transaction;
use PayPal\Api\RedirectUrls;
use PayPal\Api\Payment;
use PayPal\Api\Details;
use PayPal\Rest\ApiContext;
Use PayPal\Auth\OAuthTokenCredential;

if(!isset($_POST['product'], $_POST['price'])){
die();
}

$clientId = 'client-id';
$clientSecret = 'client-secret';

$apiContext = new ApiContext(
new OAuthTokenCredential(
$clientId,
$clientSecret
)
);

$product = $_POST['product'];
$price = $_POST['price'];
$tax = 2.00;

$total = $price + $tax;

$payer = new Payer();
$payer->setPaymentMethod('paypal');

$item = new Item();
$item->setName($product)
->setCurrency('USD')
->setQuantity(1)
->setPrice($price);

$itemList = new ItemList();
$itemList->setItems([$item]);

$details = new Details();
$details->setShipping($tax)
->setSubtotal($price)
->setTax(2.00);

$amount = new Amount();
$amount->setCurrency('USD')
->setTotal($total)
->setDetails($product);

$transaction = new Transaction();
$transaction->setAmount($amount)
    ->setItemList($itemList)
    ->setDescription('dsds')
    ->setInvoiceNumber(uniqid());

$baseUrl = "http://www.localhost/xc";
$redirectUrls = new RedirectUrls();
$redirectUrls->setReturnUrl("$baseUrl/pay.php?success=true")
->setCancelUrl("$baseUrl/pay.php?success=false");

$payment = new Payment();
$payment->setIntent('sale')
->setPayer($payer)
->setRedirectUrls($redirectUrls)
->setTransactions([$transaction]);

try{

$payment->create($apiContext);

}
catch(PayPal\Exception\PayPalConnectionException $ex){
echo $ex;
die($ex);
}

echo $approvalUral = $payment->getApprovalLink();
</code>
I've tried to search about the error but I can't seem to find an answer so I need some help. What have I done wrong ? Do I need to downgrade ? because the video in tutorial made 2 years ago so I think I need to downgrade but is it possible ?

exception 'PayPal\Exception\PayPalConnectionException' with message 'Got Http response code 400 when accessing  https://api.sandbox.paypal.com/v1/payments/payment.' in C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Core\PayPalHttpConnection.php:202
  Stack trace:
  #0 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Transport\PayPalRestCall.php(78): PayPal\Core\PayPalHttpConnection->execute('{"intent":"sale...')
  #1 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Common\PayPalResourceModel.php(104): PayPal\Transport\PayPalRestCall->execute(Array, '/v1/payments/pa...', 'POST', '{"intent":"sale...', NULL)
  #2 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Api\Payment.php(577): PayPal\Common\PayPalResourceModel::executeCall('/v1/payments/pa...', 'POST', '{"intent":"sale...', NULL, Object(PayPal\Rest\ApiContext), NULL)
  #3 C:\xampp\htdocs\ladea\checkout.php(78): PayPal\Api\Payment->create(Object(PayPal\Rest\ApiContext))
  #4 {main}exception 'PayPal\Exception\PayPalConnectionException' with message 'Got Http response code 400 when accessing https://api.sandbox.paypal.com/v1/payments/payment.' in C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Core\PayPalHttpConnection.php:202 Stack trace: #0 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Transport\PayPalRestCall.php(78): PayPal\Core\PayPalHttpConnection->execute('{"intent":"sale...') #1 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Common\PayPalResourceModel.php(104): PayPal\Transport\PayPalRestCall->execute(Array, '/v1/payments/pa...', 'POST', '{"intent":"sale...', NULL) #2 C:\xampp\htdocs\ladea\vendor\paypal\rest-api-sdk-php\lib\PayPal\Api\Payment.php(577): PayPal\Common\PayPalResourceModel::executeCall('/v1/payments/pa...', 'POST', '{"intent":"sale...', NULL, Object(PayPal\Rest\ApiContext), NULL) #3 C:\xampp\htdocs\ladea\checkout.php(78): PayPal\Api\Payment->create(Object(PayPal\Rest\ApiContext)) #4 {main}

Thank you and Regards.
Comment: Rather than following a 2 year old video tutorial, why not refer to the actual documentation?
Comment: Well I actually refer to it. But still It didn't work. I also copied the actual example documentation codes.
Answer: Remove your *.paypal.com cookies.
This an unfortunate bug in the PayPal system, where the cookie gets too large and the server errors out on it.
You can avoid the problem by using two different browsers, e.g. IE for PayPal Live and FF for the PayPal Sandbox.
Check with incognito mode also.
Answer: I have provided this information in your GitHub issue but will provide an answer here to help.
I updated the exception output of your code sample with the information in the wiki which gave some additional information, the 400 was a Malformed Request https://github.com/paypal/PayPal-PHP-SDK/wiki/exception-%27PayPal%5CException%5CPayPalConnectionException%27-with-message-%27Got-Http-response-code-400-when-accessing.
<code>$details = new Details();
$details->setShipping($tax)
->setSubtotal($price)
->setTax(2.00);

$amount = new Amount();
$amount->setCurrency('USD')
->setTotal($total)
->setDetails($product);
</code>
If your total is tax and price of the item, then don't set shipping. Also amount->setDetails should be details.
<code>$details = new Details();
$details->setSubtotal($price)
->setTax(2.00);

$amount = new Amount();
$amount->setCurrency('USD')
->setTotal($total)
->setDetails($details);
</code>

Paypal PHP SDK implementation


Como limpar o console no Python?

Question: Sou usuário do Sistema Operacional Ubuntu e, quando quero limpar o Terminal, eu utilizo o comando clear.
>>> clear

Porém, no Python, como eu poderia fazer para poder limpar o terminal, quando eu o utilizo na linha de comando?
Existe alguma maneira simples de fazer isso?
Answer: Simplesmente pressione ctrl + L   (funciona no shell também - você nem precisa escrever  clear).
Isso no terminal. Em um programa, você pode simplesmente imprimir um número de linhas em branco maior que o tamanho do terminal: print("\n" * 130), ou, se estiver no Linux ou Mac, imprimir a sequência ANSI que apaga a tela:
print("\x1b[2J"). Veja mais detalhes abaixo.
O CTRL + L limpa a tela se você estiver no terminal de algum Linux - e talvez um ou outro dos ambientes interativos do Windows - mas não no Idle, e outros prompts do Python integrados a IDE's - e certamente não é apropriado se você desejar apagar a tela de dentro de um programa (você não pode pedir ao usuário para digitar "ctrl + L" para apagar a tela :-)  )
A razão de não existir uma chamada de função simples,na biblioteca padrão do Python, é que as entradas e saídas padrão não são pensadas, numa linguagem de programação genérica, sempre como um "terminal" - elas representam um fluxo de dados de entrada, e dois de saída (stdout e stderr). Mas a linguagem em si "não sabe" que está rodando no terminal quando algum programa é executado. Para programas feitos específicamente para o terminal que queiram uma interação mais sofisticada (inclusive com o mouse), a biblioteca padrão do Python inclui o módulo curses (veja abaixo).
A solução chamando o comando externo - seja o cls no windows ou clear no Linux/Mac OS, com o os.system funciona -mas tem um problema básico: é um programa externo - todo um processo externo no sistema operacional, achar o arquivo no disco, criar o processo, para fazer uma chamada de API ao terminal. Em termos de usos de recursos é o mesmo que chamar um chaveiro para abrir e trancar a porta de novo cada vez que for sair de casa.
Bom - esses programas para "apagar a tela" - tanto o clear do Unix quanto o "cls" do DOS, quanto o "clear" e o próprio o próprio CTRL + L, não fazem muito mais do que simplesmente imprimir várias linhas em branco, de forma que o conteúdo da tela atual role pra cima.
Para fazer isso em Python, basta imprimir uma única string com um número suficiente de "linefeeds" (o caractere representado por '\n' - cujo código em decimal é "10").
Um pouco mais chato é saber o número exato de linefeeds necessário para apagar a tela exatamente e sem linefeeds a mais. Se isso não for uma preocupação, um número como 130 linhas em branco é mais que suficiente para rolar um terminal em full HD com fonte em tamanho 4 - então simplesmente:
print ("\n" * 130)

Garantidamente vai te apagar qualquer terminal, independente do tipo de S.O. sem precisar executar um outro processo fora do interpretador Python só para isso.
Claro que é mais elegante imprimir um número de linhas em branco só igual ao número de linhas do terminal - e não mais que isso. Entre outras coisas, um usuário acostumado ao uso do terminal pode esperar poder rolar a tela para cima para ver a tela anterior nas linhas imediatamente precedentes à primeira linha da tela em branco (embora o comando clear resete esse histórico também).
Para isso, a partir do Python 3.3, há a função get_terminal_size no módulo OS do Python. Para quem está escrevendo um programa que vai funcionar em várias versões do Python, a funcionalidade pode ficar encapsulada numa função do tipo:
def clear():
    try:
        import os
        lines = os.get_terminal_size().lines
    except AttributeError:
        lines = 130
    print("\n" * lines)

Mas no Python 3.3 ou acima, basta fazer:
import os
print("\n" * os.get_terminal_size().lines)

Além disso, nos Unixes, e talvez em alguns programas de terminal/prompt do Windows, são reconhecidas as "sequências ANSI" - isso é, o próprio terminal reconhece sequências especiais de caracteres que representam comandos tais como limpar a tela, alterar a cor da fonte, posicionar o cursor -etc - São um "brinquedo" bem interessante - e bem mais simples que o módulo curses do Python(do qual falo abaixo). Para apagar a tela em qualquer terminal Linux ou na maior parte dos Unixes (não sei se no Mac OS X funciona direto) - basta imprimir a sequência "\x1b[2J" - (o "\x1b" é o caractére <ESC> o mesmo código que é gerado pela tecla com esse nome). A sequência <ESC>[ inicia várias sequências ANSI. Para ver as sequências exatas suportadas pelo seu terminal, execute o comando infocmp no shell.
Por essa técnica, apagar a tela, e posicionar o cursor na primeira linha e primeira coluna pode ser feito assim:
print("\x1b[2J\x1b[1;1H")

(O comando print imprime por padrão um "\n" - se estiver no Python 3 use print("\x1b[2J\x1b[1;1H", end="") para evitar ir parar na segunda linha - em Python2, vale a pena colocar um from __future__ import print_function no começo do arquivo e fazer o mesmo.)
Lembrando que - e pelo que entendi do contexto dessa pergunta - se  a idéia é só apagar a tela enquando se está no modo interativo do Python - nesse caso, o "ctrl + L" vai servir bem.
Para quem quiser fazer um programa complexo com interação pelo terminal, minha dica sempre é a seguinte: se voê está nesse ponto, é hora de pensar em fugir do terminal. Python permite interação com várias bibliotecas gráficas para criação de um aplicativo em uma janela própria - que é muito mais confortável para o usuário final.  Usuários acostumados com interface de linha de comando estarão felizes com chamar o programa da linha de comando do sistema, passando opções diretamente na mesma  - para isso consulte o módulo argparse da biblioteca padrão.
Se mesmo assim se o autor preferir criar uma interface interativa no terminal, a dica é usar o módulo curses - também da biblioteca padrão - com ele você obtem um objeto do tipo "janela" (de texto) no terminal, onde pode posicionar o cursor em lugares exatos, apagar a tela, mudar cores, etc... E com algum trabalho (não pouco) poderá fazer um aplicativo com uma carinha realmente profissional - desde os usuários desse programa estejam congelados no tempo desde a década de 1990. Se não for esse o objetivo, apagar a tela com o print("\n" * 130) é o suficiente.
Com curses, você pode apagar a tela fazendo:
import curses
window = curses.initscr()
window.clear()
window.refresh()

No entanto o terminal estará nesse modo de "aplicação" em que "print" e "input" podem ter resultados diferentes do que acontece num terminal normal. Em particular, é importante posicionar o cursor na posição desejada da tela antes de cada impressão chamando a função  window.move(line, col) . Ao encerrar o programa, é vital retornar o terminal para o estado normal, chamando a função curses.endwin().
Comment: Não - o autor pergunta como limpar o console no terminal interativo do Python - e essa é a forma mais prática. Mas sim, a resposta está curta, -  e eu posso colocar mais tarde como fazer o Python limpar a tela. Note que nenhuma das duas outras respostas faz isso no entanto: elas chamam programas externos uqe limpam a  tela (então não é "o python").
Comment: Na minha leitura da pergunta há duas dicas de que o problema do autor não é escrever um porgrama - mas sim, apagar o console no modo interativo. Mas concordo que é ncessário ensa resposta para ela ser completa - vamos lá.
Comment: O autor quer que o python limpe a tela, poderia editar a sua resposta?
Comment: Justamente as outras resposta são programas, pois provavelmente ele está escrevendo um programa em python que trabalhe no terminal e provavelmente ele quer que em determinada ação do programa limpe a tela e este me parece o motivo das demais respostas serem códigos python. Talvez o autor da pergunta não tenha conseguido transmitir a necessidade e isto tornaria a sua resposta válida sim. No entanto nestes casos, vale mais você comentar na pergunta qual solicitando detalhes. Leve em consideração também o escopo: http://pt.stackoverflow.com/help/on-topic
Answer: Tente isso:
import os
os.system('cls' if os.name == 'nt' else 'clear')

Segue o link do SOen, aqui, tem outras alternativas também.
Comment: Pela sua resposta, acabo de aprender a detectar o Sistema operacional :)
Answer: Faça:
>>> import os
>>> clear = lambda: os.system('clear')
>>> clear()

Dessa forma é só tu chamar o clear() quando quiser limpar, se tiver no Windows só trocar clear por cls.
Answer: Como você é usuário do Sistema Operacional Ubuntu e quer uma maneira simples de fazer isso. Pode fazer assim:
import os
os.system('clear')

Answer: import os
if os.name == "nt":
    os.system("clear")
else:
    os.system("cls")

Comment: Isso já tinha sido respondido.
Comment: Para formatar o texto como código fonte, use o botão `{ }`

Como limpar o console no Python?


If I delete openenterprise module from drupal module folder site is collapse

Question: I am wrongly paste the open enterprise module in drupal module folder. If I try to delete that folder total site is collapsed. How to fix this issue.
Answer: You will need to rebuild the registry. The simplest solution is to use Drush and Registry Rebuild. There's also instructions on that page in case you can't use Drush or the command line.


Prove that if $ b> a $

Question: Let $ a $ be a positive integer and $ b $ a divisor d $ a ^ 2 + 1 $. Prove that if $ b> a $. So $ b> a + \sqrt{a}$
What did I do:
Let b = a + k for positive integer k.
We have to prove that $ k > \sqrt{a}$
$(a^2+1)/(a+k)=a-k+(k^2+1)/(a+k)$. Thus,
$(k^2+1)/(a+k)$ has to be whole number and it is obvious it is positive.
That means $ k^2+1 > a+k$
$k^2-k+1 > a$
But if $k <= \sqrt{a} $ then $k^2 <= a$ and $k^2 - k + 1 <= a$, contradiction.
Are there other methods to prove it?
Answer: The statement you're asking to prove is not always true. For example, if $a = 1$, then $a^2 + 1 = 2$, so $b = 2$ is a divisor. However, you then have $a + \sqrt{a} = 1 + 1 = 2 = b$. However, changing the $\gt$ to $\ge$ will work, i.e., $b \ge a + \sqrt{a}$.
Your proof is basically correct, but you made a small mistake. You stated that $\frac{k^2 + 1}{a + k}$ is a positive whole number. As such, it could be $1$, so the proper inequality to use is
$$k^2 + 1 \ge a + k \implies k^2 - k + 1 \ge a \tag{1}\label{eq1}$$
Thus, if $k \lt \sqrt{a}$, you have $k^2 \lt a \implies k^2 - k + 1 \lt a$, which is now a contradiction.
Note the only edge case occurs when $k = 1$, giving $a = 1$ and $b = 2$, which I stated originally. Thus, to keep the original condition of $b \gt a + \sqrt{a}$, you can just simply add the extra condition that $a \gt 1$.
As for other methods to prove it, there are a few, but the technique you used is the one I would've also used, and I believe it's at least one of the simplest & most direct ways to prove this.

Prove that if $ b> a $


Não consigo diminuir o tamanho do arquivo APK

Question: Preciso gerar o APK do meu app em Xamarin fForms, mas quando marco a opção de vinculação "Somente Assenblies de SDK" para diminuir o APK, eu tomo o erro "Falha Inesperada da Tarefa 'LinkAssemblies'". Desmarcando essa opção eu consigo gerar normalmente, mas ele fica com 40 mb.
Alguém sabe o motivo desta mensagem ou se não é isso que fará com que o tamanho do meu APK diminua?
Answer: Tive esse problema.
Resolvi criando um projeto novo e "copiando" sua configuração e depois mudando pra "SDK Assemblies Only".
Comment: Isso é quase 1 dia de trabalho, eu já fiz isso que você falou por outros motivos similares, o Xamarin dá bug de mais... até desanima.
Se nada mais der certo vou fazer isso de novo.
Comment: Estranho, cara. Aqui Xamarin roda bem tranquilo. Mas não trabalho com Forms, apenas com Android.
Comment: Só ressaltando, que depois que você faz upload pra Google Play, a própria
Google diminui o tamanho do apk. Ela faz esse Assemblies Only.
Comment: Bom saber Diego, eu vou tentar subir direto pra Play Store para ver.
Detalhe, acho que a zica esta no meu VS... criei um projeto novo como você falou, só que antes de perder tempo montando minha solução inteira eu ja tentei gerar o APK logo de cara. e tomei o mesmo erro  sem ter adicionado absolutamente nenhuma referencia.
Falha inesperada de tarefa "LinkAssemblies" Deixando a vinculação com "Nenhum" , a aplicação compila.
Complicado viu...


Find CDF from an estimated PDF (estimated by KDE)

Question: I would like to find the CDF from an estimated PDF.
This PDF was estimated from Kernel Density Estimation (with a Gaussian kernel using a 0.6 width window).
I know, in theory, that the CDF can be estimated as:
$F_{X}(x) = \int_{-\infty}^{x}f(t)dt$
Is it possible to apply this integral directly on the estimated pdf? In this case, I am using python.
thanks in advance.
Comment: @Michael Isn't the whole point that the OP has an estimate of $f$?  The question therefore concerns the properties of a procedure that estimates a CDF by integrating an estimate of the PDF.
Comment: What you know is the height of the density at each point. So I think you need something like a Riemann -Stieltjes integral through a rectangular approximation.
Comment: You don't know what f(t) is and the CDF is a integral. What else do you think you could do?
Comment: Isn't there a more straightforward way to do it instead of using an integral? Just asking.
Comment: The estimate is not in closed form so I am just saying that it has to be calculated numerically.  The normal distribution is also not in closed form but has been numerically integrated already.  I was just addressing the idea that it could be computed "directtly".
Comment: Closely related: https://stats.stackexchange.com/questions/175580
Answer: There's no need to integrate anything if you know the cdf of the kernel itself. I believe this is straightforward for all the common kernels.
Note that

a KDE is a mixture density
the cdf of a mixture is the mixture of the cdfs.

that is, if $\hat{f}(x)=\frac{1}{n}\sum_i f_i(x)$ is your KDE at $x$, then
$\hat{F}(x)=\frac{1}{n}\sum_i F_i(x)$.
Take a Gaussian kernel for example. If $x_i$ are your observations, $f_i$ is $\frac{1}{\sigma} \phi(\frac{x- x_i}{\sigma})$ and $F_i=\Phi(\frac{x-x_i}{\sigma})$, where commonly $\sigma$ is defined as the bandwidth (in some implementations the bandwidth may be some multiple of $\sigma$).
Indeed, R does that (defines bandwidth = $\sigma$) for all its kernels, not just the Gaussian one. But it's easy as long as you can convert a bandwidth to the parameters of the kernel so you can call a function for the cdf.
So you can evaluate the cdf of your mixture at any $x$ in linear time. If you need it to be able to calculate $\hat{F}$ fast, you could evaluate it over a grid (fine enough to get sufficient accuracy), and use interpolation in between (e.g. in R this is easily done with approxfun; no doubt Python has a convenient way to do something similar)
Here's an example of a plot of a kde and cdf for a Gaussian kernel.

Here's the code I used (it was done in R - this is a quick kludge to show the idea, a proper function would be checking arguments, providing better info, labelling axes, letting you specify the kernel and so on). The workhorse is the third line, which defines the function that does all the actual calculation of the cdf, everything else is details of data or plotting.
x <- c(11,12,16) #data
xx <- seq(7,20,.1) # plot values for the cdf
kdecdfnorm <- function(x,xdat,bw) rowMeans(pnorm(outer(x,xdat,"-"),0,bw)) #cdf of KDE
opar <- par() # save graphics parameter settings
par(mfrow=c(1,2)) # 1 x 2 plot grid
kde <- density(x)
plot(kde)
bw <- kde$bw
plot(xx,kdecdfnorm(xx,x,bw),type="l")
abline(h=c(0,1),col=rgb(.5,.5,.5,.5),lty=3)
par(opar) # restore graphics parameters

How does that rowMeans(pnorm(outer(x,xdat,"-"),0,bw)) work?

rowMeans is just doing $\frac{1}{n}\sum_{i=1}^{n}$ of its argument
pnorm is computing the cdf of the Gaussian kernel terms, with bandwidth at its last argument
the first argument to pnorm is just $x-x_i$ over the data values ($x_i$) and the various x's we want to find the curve at

which is to say we're just computing $\frac{1}{n}\sum_i \Phi(\frac{x-x_i}{\sigma})$ in a quite direct way, across whatever values for $x$ we want to calculate it at.


Exposing state props from functional component to navigationOptions function

Question: I have a component that renders the input field, I want to pass the data to the next page when user clicks on "next" button in the header. What is the best practice for doing so? How do I expose this into Page.navigationOptions?
Or is it best to just set up redux for these types of things?
const Page = () => {
  const [desc, getDesc] = useState('');

  return (
    <View style={styles.inputFieldDescContainer}>
      <TextInput
        multiline
        placeholder='Write a description...'
        onChangeText={(text) => getDesc(text)}
        value={desc}
      />
    </View>
  );
};

// How do I pass desc properties down into navigationOptions?

Page.navigationOptions = (navData) => {
  return {
    headerTitle: 'Page,
    headerRight: (
      <HeaderButtons HeaderButtonComponent={HeaderButton}>
        <Item
          title='Next'
          onPress={() => {
            navData.navigation.navigate('NextPage', {data: navData});
          }}
        />
      </HeaderButtons>
    ),
    headerBackTitle: null
  };
};

/* NextPage.js */

const NextPage = (props) => {
  console.log('enter props data', props.navigation.getParam('data'));
  console.log('enter props navigation', props.navigation);
  const [valueText, setValueText] = useState();
  return (
    <View>
      <TextInput onChangeText={(text) => setValueText(text)} value={valueText}/>
      <TouchableOpacity><Text>Create your workout</Text></TouchableOpacity>
    </View>
  );
;}

Answer: Sharing state and props between component and options is possible in React Navigation 5 https://blog.expo.io/announcing-react-navigation-5-0-bd9e5d45569e
In React Navigation 4, you can use params to store the value to be able to share it:
const Page = ({ navigation }) => {
  const desc = navigation.getParam('description', '');

  return (
    <View style={styles.inputFieldDescContainer}>
      <TextInput
        multiline
        placeholder='Write a description...'
        onChangeText={(text) => navigation.setParams({ description: text )}
        value={desc}
      />
    </View>
  );
}


Longest distance to goal for top 5 European league?

Question: Stoke City goalkeeper Asmir Begovic's 91.9 metres (301ft 6in) goal against Southampton is the longest goal scored in the Premier League.
So, what is the longest goal scored in other top European leagues?
Comment: @gdrt That is right but back when I suggested edit for similar type of question, my edit got rejected. Stating: This edit introduces tags that do not help to define the topic of the question. Tags should help to describe what the question is about, not just what it contains. I don't mind though just letting you know.
Comment: IMO, your question is about records of top 5 leagues. And if there are different subjective opinions about that, I would encourage you to start a meta topic about tagging the series of questions you've already asked about top 5 leagues. I would contribute to that meta with my 5 cents.
Comment: @gdrt Not sure about those three added tag though.
Comment: Why? Isn't the question explicitly asking for the records achieved in La Liga, Bundesliga, Serie A and Ligue 1?
Answer: German Bundesliga: Moritz Stoppelkamp for SC Paderborn against Hannover 96 in 2014/15 season from 83 meters. Video, source: bundesliga.com - German.
Italian Serie A: Enzo Ferrari for Palermo against Roma in 1968/69 season from 77 meters. Source: forzapalermo.it - Italian, gazetta.it - Italian.
Spanish La Liga: Antonio José González Santos for Numancia against Sevilla in 2004/05 season from 68 meters. Videoǂ, source: elmundo - Spanish, marca - Spanish.

ǂ *Video has a very poor quality, and player's team information label is wrong, i.e Levante instead of Numancia, probably because they have similar outfits.
Comment: Couldn't find an information for French Ligue 1.
Comment: Accepting your answer for now and will let you know if I find about Ligue 1.


Proof Regarding Effective Annual Rate Inequality

Question:
I tried to use the $$\lim_{m \to \infty}(1 + \frac{i}{m})^{m} = e^i$$ as a starting point but couldn't work it out quite right.
I am using the definition that the EAR is $$(1 + \frac{i}{m})^{m} - 1$$
Comment: Can you prove that the effective rate is an increasing function of $m$?
Answer: We want to show that $\left(1+\frac im\right)^m$ is an increasing function of $m$.  This is intuitively clear to finance folk (as compounding more often seems as though it should generate more money) but it turns out to be surprisingly hard to prove.    It is possible to do that via calculus but in this situation we know that $m\in \mathbb N$ and that allows us to find an elementary approach (noting that "elementary" is not a synonym for "easy").  I'll show both methods.  In both we suppose that $i>0$.
Method I:  Calculus
We define $f(x)=\left(1+\frac ix\right)^x$ and let $g(x)=\ln\,f(x)$.  We want to prove that $f'(x)>0$.  Since $\ln$ is increasing, it suffices to show that $g'(x)>0$.  We note that $$g'(x)=\ln\left(1+\frac ix\right) -\frac i{i+x}$$
Letting $G(x)=\left(1+\frac ix\right)e^{-\frac i{i+x}}$ we see that $g'(x)=\ln\,G(x)$.  Since the Taylor expansion for $e^{-x}$ is alternating (for $x>0$) we have $$ G(x)> \left(1+\frac ix\right)\left(1-\frac i{i+x}\right)=1$$  It follows that $$g'(x)>\ln\,1=0$$ and we are done.
Method II:  Elementary
Specifically, then, we want to show that $$\left(1+\frac im\right)^m<\left(1+\frac i{m+1}\right)^{m+1}$$  We remark that, for  $x>y>0$ we have $$\frac {x^{m+1}-y^{m+1}}{x-y}=\sum_{j=0}^mx^jy^{m-j}<(m+1)x^m$$
Cross-multiplying and rearranging yields $$x^m\left((m+1)y-mx\right)<y^{m+1}$$  Now let $x=1+\frac im$ and $y=1+\frac i{m+1}$.  We see that for these values  $\left((m+1)y-mx\right)=1$ and we are done.


How to include WAR file in EAR file using pom.xml in maven

Question: I have a project where I need that my WAR file for the project should get build inside my EAR using maven build tool. I need to configure maven-ear-plugin for that. I configured the maven-war-plugin and it is working fine.
Can anybody please help me out in configuring maven-ear-plugin or do in need to do anything else.
I also do not know the significance of application.xml that get generated with maven-ear-plugin
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.Avizva.FBVision</groupId>
  <artifactId>Enrollment_GUID</artifactId>
  <packaging>ear</packaging>
  <name>Enrollment_GUID</name>
  <build>
    <sourceDirectory>${basedir}/src</sourceDirectory>
<resources>
      <resource>
        <directory>${basedir}/src</directory>
        <includes>
        <include>**/*.properties</include>
         <include>**/*.java</include>
        </includes>

      </resource>
    </resources>
    <plugins>
    <plugin>
        <artifactId>maven-ear-plugin</artifactId>
        <version>2.9.1</version>
        <configuration>
        <generateApplicationXml>false</generateApplicationXml>
        <finalName>Enrollment_GUID.war</finalName>
        <modules>
        <webModule>
                <groupId>${project.groupId}</groupId>
                <artifactId>Enrollment_GUID-war</artifactId>
                <bundleFileName>Enrollment_GUID.war</bundleFileName>
                <contextRoot>/Enrollment_GUID</contextRoot>
        </webModule>

        </modules>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.1</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>

      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.3</version>
        <configuration>

          <warSourceDirectory>${basedir}/WebContent</warSourceDirectory>
          <warName>Enrollment_GUID</warName>
          <packagingIncludes>WEB-INF/lib/*.jar,WEB-INF/wsdl/*.xsd,WEB-INF/wsdl/*.wsdl,WEB-INF/properties/*.properties,**/*.xml</packagingIncludes>
          <failOnMissingWebXml>false</failOnMissingWebXml>
        </configuration>
      </plugin>
      <plugin>
      <artifactId>maven-resources-plugin</artifactId>
      <version>2.4</version>
      <configuration>
        <encoding>UTF-8</encoding>
      </configuration>
    </plugin>
    </plugins>

  </build>
  <dependencies>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-1.2-api</artifactId>
        <version>2.0-rc1</version>
    </dependency>

    <dependency>
        <groupId>org.kie.modules</groupId>
        <artifactId>org-apache-commons-lang3</artifactId>
        <version>6.1.0.CR1</version>
        <type>pom</type>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-jdk14</artifactId>
        <version>1.7.7</version>
    </dependency>
<dependency>
    <groupId>joda-time</groupId>
    <artifactId>joda-time</artifactId>
    <version>2.3</version>
</dependency>
    <dependency>
        <groupId>org.apache.directory.studio</groupId>
        <artifactId>org.apache.commons.lang</artifactId>
        <version>2.6</version>
    </dependency>
    <dependency>
        <groupId>aspectj</groupId>
        <artifactId>aspectjweaver</artifactId>
        <version>1.5.4</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-aop</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-asm</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-beans</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-context</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-core</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-expression</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-jdbc</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-orm</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-web</artifactId>
        <version>3.0.1.RELEASE</version>
    </dependency>
    <dependency>
    <groupId>net.sf.dozer</groupId>
    <artifactId>dozer</artifactId>
    <version>5.3.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-adb</artifactId>
        <version>1.6.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.axis2</groupId>
        <artifactId>axis2-kernel</artifactId>
        <version>1.6.2</version>
    </dependency>
    <dependency>
    <groupId>org.apache.axis2</groupId>
    <artifactId>axis2-transport-local</artifactId>
    <version>1.6.2</version>
    </dependency>
    <dependency>
    <groupId>org.apache.axis2</groupId>
    <artifactId>axis2-transport-http</artifactId>
    <version>1.6.2</version>
    </dependency>
    <dependency>
    <groupId>org.apache.ws.commons.axiom</groupId>
    <artifactId>axiom-api</artifactId>
    <version>1.2.13</version>
    </dependency>
    <dependency>
    <groupId>org.apache.ws.commons.axiom</groupId>
    <artifactId>axiom-impl</artifactId>
    <version>1.2.13</version>
    </dependency>
    <dependency>
        <groupId>antlr</groupId>
        <artifactId>antlr</artifactId>
        <version>2.7.5</version>
    </dependency>
    <dependency>
        <groupId>aspectj</groupId>
        <artifactId>aspectjrt</artifactId>
        <version>1.5.4</version>
    </dependency>
    <dependency>
    <groupId>org.apache.ws.schema</groupId>
    <artifactId>XmlSchema</artifactId>
    <version>1.4.5</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient-cache</artifactId>
        <version>4.2.3</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpcore</artifactId>
        <version>4.2.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpmime</artifactId>
        <version>4.2.3</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.2.3</version>
    </dependency>
  </dependencies>

  <version>1</version>
</project>

Comment: Check http://stackoverflow.com/questions/5167024/how-to-add-war-inside-ear-with-maven
Comment: I tried it but it is not working for me I am getting the following error when i Used it in my pom.xml
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-ear-plugin:2.9.1:generate-application-xml (default-generate-application-xml) on project Enrollment_GUID: Artifact[war:com.Avizva.FBVision:Enrollment_GUID-war] is not a dependency of the project. -> [Help 1]  What is the significance of application.xml here and how to configure it.
Answer: First you should create a folder structure like this:
+-- root (pom.xml)
     +--- mod-war (pom.xml)
     +--- mod-ear (pom.xml)

Add the mod-war as a dependency to the mod-ear. Make the packging of mod-war war whereas the packaging of mod-ear ear.
Apart from that follow the default folder layout structure instead of changing it cause it's not neccessary.
Comment: Can you please modify the current above provided pom.xml in this fashion.


Calculate the Kullback-Leibler Divergence in practice?

Question: I am using KL Divergence as a measure of dissimilarity between 2 $p.m.f.$ $P$ and $Q$.
$$D_{KL}(P||Q) = \sum_{i=1}^N \ln \left( \frac{P_i}{Q_i} \right) P_i$$
$$=-\sum P(X_i)ln\left(Q(X_i)\right) + \sum P(X_i)ln\left(P(X_i)\right)$$
If $$P(X_i)=0$$
then we can easily calculate that  $$P(X_i)ln\left(Q(X_i)\right)=0$$ $$P(X_i)ln\left(P(X_i)\right)=0$$
But if $$P(X_i)\ne0$$ and $$Q(X_i)=0$$
how to calculate the $$P(X_i)ln\left(Q(X_i)\right)$$
Comment: @Matthew $Q(X_i)=0$ for some $X_i$, not all. I am considering a workaround to base $P$ and $Q$ on *the same set* of outcomes and add a small *pseudo count*, say 0.001, for the not-show-up outcomes. It can avoid the zero-valued probabilities. But I am not sure if there's any side-effects.
Comment: To save everyone else some time staring at what you meant you might want to change $P(X_i)!=0$ to $P(X_i) \ne 0$ with the "\ne" token
Comment: In addition, do you mean that $Q(X_i) = 0 $ for all $X_i$? In this case the KL divergence is not defined, since $Q$ is not a probability function (those must sum to 1 over the support).
Comment: @Matthew Thanks, corrected. I followed my coding habit accidentally.
Answer: You can't and you don't.
Imagine that you have an random variable of probability distribution Q. But your friend Bob thinks that the outcome comes from the probability distribution P. He has constructed an optimal encoding, that minimizes the number of expected bits he will need to use to tell you the outcome. But, since he constructed the encoding from P and not from Q, his codes will be longer than necessary. KL-divergence measure how much longer the codes will be.
Now lets say he has a coin and he wants to tell you the sequence of outcomes he gets. Because head and tail are equally likely he gives them both 1-bit codes. 0 for head, 1 for tail. If he gets tail tail head tail, he can send 1 1 0 1.
Now, if his coin lands on the edge he cannot possibly tell you! No code he sends you would work. At this point KL-divergence breaks down.
Since KL-divergence breaks down you will either have to use another measure or other probability distributions. What you should do really depends on what you want. Why are you comparing probability distributions? Where do your probability distributions come from, are they estimated from data?
You say your probability distributions come from natural language documents somehow, and you want to compare pairs of categories.
First, I'd recommend a symmetric relatedness measure. For this application it sounds like A to be as similar to B as B is similar to A.
Have you tried the cosine similarity measure? It is quite common in NLP.
If you want to stick with KL, one thing you could do is estimate a probability function from both documents and then see how how many extra bits you'd need on average for either document. That is (P||(P+Q)/2 + Q||(P+Q)/2)/2
Comment: Great explanation but slightly confusing: the way you describe the first paragraph, isn't that KL(Q||P)?
Comment: using the mixture distribution to get the union of the supports and act as a mediator is the idea I've been missing for 20 years \o/
Answer: In practice, I have run into this issue as well. In this case, I've found that substituting the value of 0 for some very small number can cause problems. Depending upon the value that you use, you will introduce a "bias" in the KL value. If you are using the KL value for hypothesis testing or some other use that involves a threshold, then this small value can bias your results. I have found that the most effective way to deal with this is to only consider computing the KL over a consistent hypothesis space X_i where BOTH P and Q are non-zero. Essentially, this limits the domain of the KL to a domain where both are defined and keeps you out of trouble when using the KL to perform hypothesis tests.
Comment: Thanks. It's an interesting suggestion. Basically, it is also trying to base P and Q on the same set of outcomes. I will try that.
Comment: If I calculate KL over the data subset where both P and Q are non zero, do I need to re-normalize P and Q over that subset? Or just use the original probability value? I think I should. Otherwise, P and Q are still not on the same base.
Comment: I just tried with your suggestion. P distributes over 10K outcomes, and Q distributes over 10K outcomes, too. But P and Q only have 3K outcomes in common. If I only use the common 3K outcomes to estimate the difference between P and Q, I don't think it's reasonable. Because we are ignoring to many things. And btw, the result with this approach is quite different from what I get by adding a small number (or pseudo count).
Comment: Add some context, I am working on a NLP experiment. I have several categories of documents and I want to tell how close each pair of categories are related to each other.
Answer: Having a probability distribution where $Q_i=0$ for any $i$ means that you are certain that $Q_i$ can not occur. Therefore if a $Q_i$ were ever obeserved it would represent infinite surprise/information, which is what Shannon information represents. KL diveregence represents the amount of additional surprise (ie information lost) per observation if the distribution $Q$ is used  as an approximation for the distribution $P$. If the approximation predicts 0 probability for an event that has a postive probability in reality, then you will experience infinite surprise some percentage of the time and thus infinite surprise on average.
The solution is to never allow 0 or 1 probabilities in estimated distributions. This is usually achieved by some form of smoothing like Good-Turing smoothing, Dirichlet smoothing or Laplace smoothing.