c# How to merge two data rows & columns into one in datatable by using ItemArray?

Question: I would like to produce this output as shown on the image below with ItemArray function, please help :D
Assume im using like dr.ItemArray[0] with index to allocate the row data and merge with respective ItemArray index to get the combined data
Answer: Assuming dt is your datatable
<code>    foreach(DataRow row in dt.Rows)
    {
        row[0] = row[0].ToString() + row[1].ToString();
    }
    dt.Columns.RemoveAt(1);
</code>
Comment: Thanks man the solution works pretty well as I thought operand + cannot apply to both object


Javascript - Dynamically add a value from input to the GET url

Question: I have 3 simple input filed's
I need do add dynamically the value of the input to the GET request, I need to use only vanilla js for this.
My Html

        <input class="input" id="player1" type="text">

Submit

        <input class="input" id="player2" type="text">

Submit

        <input class="input" id="player3" type="text">

Submit

My JS
function submit(){
    var http = new XMLHttpRequest();

    var guessPlayer1 = document.getElementById('player1').value
    var guessPlayer2 = document.getElementById('player2').value
    var guessPlayer3 = document.getElementById('player3').value

    http.onreadystatechange = function(){
        if(http.readyState == 4 && http.status == 200){
            console.log(http.response)
        }
    }

    http.open("GET", "https://Dummy-End-Point.com/" + guessPlayer1, true);
    http.send();

}

Comment: Looks like you have done it.  Whats the problem?
Comment: @Programnik, hello, i need that guessPlayer1 on the end point to be dynamic, and not hardcoded . If im thinking straight
Comment: It is dynamic.  Its a variable you are setting higher up in the code.
Comment: You mean if they click on player1, it should append player 1 value, player2, player 2 value etc?
Comment: @Programnik yes, thats is
Answer: Change submit() so it takes the ID as a parameter.

function submit(id) {
  var http = new XMLHttpRequest();

  var guess = document.getElementById(id).value

  http.onreadystatechange = function() {
    if (http.readyState == 4 && http.status == 200) {
      console.log(http.response)
    }
  }

  http.open("GET", "https://Dummy-End-Point.com/" + guess, true);
  http.send();

}

    <input class="input" id="player1" type="text">

Submit

    <input class="input" id="player2" type="text">

Submit

    <input class="input" id="player3" type="text">

Submit

Answer:

        <input class="input" id="player1" type="text">

Submit

        <input class="input" id="player2" type="text">

Submit

        <input class="input" id="player3" type="text">

Submit

function submit(elem) {
  var http = new XMLHttpRequest();

  var guess = elem.parentElement.getElementsByTagName("input")[0].id;

  http.onreadystatechange = function() {
    if (http.readyState == 4 && http.status == 200) {
      console.log(http.response)
    }
  }

  http.open("GET", "https://Dummy-End-Point.com/" + guess, true);
  http.send();

}


Como fazer um ngRepeat sem um elemento pai?

Question: Estou usando o Angular com Admin LTE. Preciso fazer um ng-repeat onde as li dentro de uma ul vão se repetir. Não posso repetir a ul, mas apenas as li dentro desse ul. Também não posso englobar essas li com um elemento pai para fazer o laço de repetição com ng-repeat, pois isso está quebrando o estilo, uma vez que o seletor do  css está capturando ul.timeline > li.
O meu código está assim, e as linhas comentadas indicam onde preciso iniciar e terminal o ngRepeat :

                Responsável:

                 realizou  em .

Não posso usar um elemento englobando as li que quero repetir, mas preciso que elas se repitam dentro do trecho comentando. Como fazer isso?
Comment: Não sei se entendi direito. Você quer utilizar o mesmo `ng-repeat` com os dois elementos `li`?
Comment: @Randrade preciso que o `ng-repeat` comece onde está comentado `[ngRepeat status in os.status]` e termine onde está comentado `[endNgRepeat]`. Porém não posso "englobar" com um elemento, para usar ng-repeat, pois os elementos tirariam a formatação do css.
Answer: No Angular você tem o ng-repeat-start onde você pode fazer o que deseja. Seu código ficaria mais ou menos assim:

                Responsável:

                 realizou  em .

Dessa forma você está mostrando onde irá começar o laço com o ng-repeat-start e onde terminará, com o ng-repeat-end.
Tomei a liberdade de criar uma demo no Plnkr.
Comment: Interessante +1. No momento que eu tive o problema, pensei que o Angular teria uma maneira de fazer isso através de comentários html, mas foi mais simples do que eu pensava;
Comment: @WallaceMaxters Existem várias coisas que o Angular possui e que muitos desconhecem.
Answer: Faz tipo isso:

      I get repeated

     I also get repeated

Comment: +1 é isso mesmo. Valeu :D


How does Canonical Time Warping help in time alignment?

Question: Canonical Time Warping is a state-of-the-art technique for time alignment. According to the original paper, it helps account for individual varieties when aligning sequences derived from different subjects performing similar event (e.g., aligning sensor data referred to two subjects' walking), by maximizing sequences correlation in a projected subspace.
Consider the iterative procedure of CTW that combines Canonical Coef. Analysis (CCA) and Dynamic Time Warping (DTW): Given a pair of temporally misaligned time-series that are in the same dimension, DTW is performed to initialize the warping matrices.

Then, CCA is performed on the warped time-series (not necessarily well aligned) to give a pair of projection matrices.
Multiply the original input (unwarped sequences) by the corresponding projection matrix obtained in 1, on which DTW is then performed. The warping matrices will be used in the next iteration.
Before next iteration, warp (obtained in 2) the input and multiply it with projection matrices (obtained in 1), and assess the Euclidean distance of two sequences. The iteration stops as the Euclidean loss reaches a threshold.

My question is, how does CCA help time-alignmnet in first step of iteration? Since warping matrices given by DTW is not well tuned, it might not make sense to force maximal correlation between two not-well-aligned or misaligned sequences.
For example, consider an extreme case: two same, but out-of-phase time-sequences are firstly analyzed by CCA, followed by DTW. The resulting warping path can be diagonal (45 degree) if the projected sequences are flattened to zeros by CCA, which is totally wrong. Even if not zeros, making similar the two temporally misaligned time-series can harm the performance of DTW, so I see no benefit of CCA in CTW.
Could some give a straight forward explanation about it? Or correct my understanding of CTW, please.
Comment: Thanks for the great reference. But traditionally CCA is applied to simultaneous pair of data. E.g., a video recording of a subject speaking (visual data) and its simultaneously recorded audio stream (audio data). In such case, visual and audio data are well aligned, on which CCA performs and makes sense; no temporal misalignment. On the other hand, it doesn't make sense to me if one try to perform CCA on a pair of temporally misaligned data.
Comment: A related question/answer this site with an excellent explanation of CCA is [How to visualize what canonical correlation analysis does (in comparison to what principal component analysis does)?](https://stats.stackexchange.com/questions/65692/how-to-visualize-what-canonical-correlation-analysis-does-in-comparison-to-what?rq=1)
Answer: I think you raise a valid question, which is that "without an initial time-aligned pair of datasets, how can CCA perform well?"
My guess is that the answer is that "CCA doesn't perform well in the first step". The reason I say this is because the initial estimates "CCA" projectors don't require time-alignment, because PCA is used for each data matrix ($X,Y$) separately to estimate their initial "spatial" projections, respectively ($V_x, V_y$).
According the (Zhou and de la Torre 2009):

The algorithm starts by initializing $V_x$ and $V_y$ with identity matrices. Alternatively, PCA can be applied independently to each set, and
  used as initial estimation of $V_x$ and $V_y$ if $d_x\ne d_y$. In the case of high-dimensional data, the
  generalized eigenvalue problem is solved by regularizing the covariance matrices adding a scaled
  identity matrix. The dimension $b$ [referring to the reduced dimension, i.e., number of canonical/principal vectors] is selected to preserve 90% of the total correlation.

In other words, the first DTW is applied to the "high variance" portion of each dataset. From there onward, the first DTW estimate is used to align the matrices.
Also, note that according the paper, when the data dimensions match ($d_x=d_y$), identity matrices are used for the initial CCA projections, meaning that in effect DTW is applied first. So, whether PCA is used or not, the CTW algorithm always applies CCA to an initially time-aligned sequence.


How to get full control of umask/PAM/permissions?

Question: // Updated Feb 8 - Outstanding issues in brief:

How to umask directories differently from files?
How to umask on Nautilus copy/paste?
How to set umask for SSHFS?

OUR SITUATION
Several people from our company log in to a server and upload files. They all need to be able
to upload and overwrite the same files. They have different usernames, but are all part of the
same group. However, this is an internet server, so the "other" users should have (in general) just
read-only access. So what I want to have is these standard permissions:
files: 664
directories: 771
My goal is that all users do not need to worry about permissions. The server should be configured in such
a way that these permissions apply to all files and directories, newly created, copied, or over-written. Only
when we need some special permissions we'd manually change this.
We upload files to the server by SFTP-ing in Nautilus, by mounting the server using sshfs and accessing it in Nautilus
as if it were a local folder, and by SCP-ing in the command line. That basically covers our situation and what we aim
to do.
Now, I have read many things about the beautiful umask functionality. From what I understand umask (together with PAM) should allow me
to do exactly what I want: set standard permissions for new files and directories. However, after many many hours of reading and trial-and-error,
I still do not get this to work. I get many unexpected results. I really like to get a solid grasp of umask and have many question unanswered. I will post these questions below,
together with my findings and an explanation of my trials that led to these questions. Given that many things appear to go wrong, I think that
I am doing several things wrong. So therefore, there are many questions.
NOTE: I am using Ubuntu 9.10 and therefore can not change the sshd_config to set the umask for the SFTP server. Installed SSH OpenSSH_5.1p1 Debian-6ubuntu2 < required OpenSSH 5.4p1. So here go the questions.
1. DO I NEED TO RESTART FOR PAM CHANGS TO TAKE EFFECT?
Let's start with this. There were so many files involved and I was unable to figure out what does and what does not affect things, also because I did not
know whether or not I have to restart the whole system for PAM changes to take effect. I did do so after not seeing the expected results, but is this really
necessary? Or can I just log out from the server and log back in, and should new PAM policies be effective? Or is there some 'PAM' program to reload?
2. IS THERE ONE SINGLE FILE TO CHANGE THAT AFFECTS ALL USERS FOR ALL SESSIONS?
So I ended up changing MANY files, as I read MANY different things. I ended up setting the umask in the following files:
~/.profile -> umask=0002
~/.bashrc -> umask=0002
/etc/profile -> umask=0002
/etc/pam.d/common-session -> umask=0002
/etc/pam.d/sshd -> umask=0002
/etc/pam.d/login -> umask=0002

I want this change to apply to all users, so some sort of system-wide change would be best. Can it be achieved?
3. AFTER ALL, THIS UMASK THING, DOES IT WORK?
So after changing umask to 0002 at every possible place, I run tests.
------------SCP-----------
TEST 1:
scp testfile (which has 777 permissions for testing purposes) server:/home/
testfile                                      100%    4     0.0KB/s   00:00

Let's check permissions:
user@server:/home$ ls -l
total 4
-rwx--x--x 1 user uploaders 4 2011-02-05 17:59 testfile (711)

UPDATE: fixed by ONLY setting umask in pam.d/common-sessions (see comments)
---------SSH------------
TEST 2:
ssh server
user@server:/home$ touch anotherfile
user@server:/home$ ls -l
total 4
-rw-rw-r-- 1 user uploaders 0 2011-02-05 18:03 anotherfile (664)

--------SFTP-----------
Nautilus: sftp://server/home/
Copy and paste newfile from client to server (777 on client)
TEST 3:
user@server:/home$ ls -l
total 4
-rwxrwxrwx 1 user uploaders 3 2011-02-05 18:05 newfile (777)

Create a new file through Nautilus. Check file permissions in terminal:
TEST 4:
user@server:/home$ ls -l
total 4
-rw------- 1 user uploaders    0 2011-02-05 18:06 newfile (600)

I mean... WHAT just happened here?! We should get 644 every single time. Instead I get 711, 777, 600, and then once 644. And the 644 is only achieved
when creating a new, blank file through SSH, which is the least probable scenario.
So I am asking, does umask/pam work after all?
UPDATE: fixed test 4 by ONLY setting umask in pam.d/common-sessions (see comments)
4. SO WHAT DOES IT MEAN TO UMASK SSHFS?
Sometimes we mount a server locally, using sshfs. Very useful. But again, we have permissions issues.
Here is how we mount:
sshfs -o idmap=user -o umask=0113 user@server:/home/ /mnt

NOTE: we use umask = 113 because apparently, sshfs starts from 777 instead of 666, so with 113 we get 664 which is the desired file permission.
But what now happens is that we see all files and directories as if they are 664. We browse in Nautilus to /mnt and:

Right click -> New File (newfile) --- TEST 5
Right click -> New Folder (newfolder) --- TEST 6
Copy and paste a 777 file from our local client --- TEST 7

So let's check on the command line:
user@client:/mnt$ ls -l
total 8
-rw-rw-r-- 1 user 1007    3 Feb  5 18:05 copyfile (664)
-rw-rw-r-- 1 user 1007    0 Feb  5 18:15 newfile (664)
drw-rw-r-- 1 user 1007 4096 Feb  5 18:15 newfolder (664)

But hey, let's check this same folder on the server-side:
user@server:/home$ ls -l
total 8
-rwxrwxrwx 1 user uploaders    3 2011-02-05 18:05 copyfile (777)
-rw------- 1 user uploaders    0 2011-02-05 18:15 newfile (600)
drwx--x--x 2 user uploaders 4096 2011-02-05 18:15 newfolder (711)

What?! The REAL file permissions are very different from what we see in Nautilus. So does this umask on sshfs just create a 'filter' that shows unreal
file permissions? And I tried to open a file from another user but the same group that had real 600 permissions but 644 'fake' permissions, and I could
still not read this, so what good is this filter??
5. UMASK IS ALL ABOUT FILES. BUT WHAT ABOUT DIRECTORIES?
From my tests I can see that the umask that is being applied also somehow influences the directory permissions. However, I want my files to be 664 (002)
and my directories to be 771 (006). So is it possible to have a different umask for directories?
6. PERHAPS UMASK/PAM IS REALLY COOL, BUT UBUNTU IS JUST BUGGY?
On the one hand, I have read topics of people that have had success with PAM/UMASK and Ubuntu. On the other hand,
I have found many older and newer bugs regarding umask/PAM/fuse on Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/241198
https://bugs.launchpad.net/ubuntu/+source/fuse/+bug/239792
https://bugs.launchpad.net/ubuntu/+source/pam/+bug/253096
https://bugs.launchpad.net/ubuntu/+source/sudo/+bug/549172
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=314796

So I do not know what to believe anymore. Should I just give up? Would ACL solve all my problems?
Or do I have again problems using Ubuntu?

One word of caution with backups using
  tar. Red Hat /Centos distributions
  support acls in the tar program but
  Ubuntu does not support acls when
  backing up. This means that all acls
  will be lost when you create a backup.

I am very willing to upgrade to Ubuntu 10.04 if that would solve my problems too, but first I want to understand what is happening.
Answer: Many things could be going on here.
First thoughts:

yes, pam.d changes take effect immediately
/etc/pam.d/common-session is the best place to set a default umask
any pam.d umask would get overridden by any entry in .bashrc,
but .bashrc only gets read under certain circumstances (interactive, non-login shell)
testfile (711) is very strange

how is /home mounted, and are you using ACLs?
(e.g. what do ls -ld /home and getfacl /home print?)
did testfile already exist before you did the copy,
because scp won't change the permissions on a file that already existed (unless you use the -p flag)

Nautilus is known to create files a different way, not sure why, or what the rules are
umask=0113 will probably cause problems
are the server and client running the same operating system?
for example, if the client has ACLs enabled, or is Cygwin, the behavior can be different
the best way to force sane permissions is to use default ACLs, exactly because,
as you discovered, umask can be overridden by the user in .bashrc and .bash_profile.

Update:

umask=0113 for sshfs is wrong.

Try mounting without specifying a umask
Create a new file inside the mount point using touch.
You should see it only gets e.g. -rw-r--r--, without x bits
By masking out x bits, you might break directories
and compilers may be unable to create executable files properly

Workaround:
If we can't think of anything better, you could either use fam or gamin to watch for new files being created and fix the permissions on them, or even just a script that runs periodically and sets the permissions on all files.
Comment: Thanks! OK, so I removed *ALL* umask settings except for pam.d/common-sessions. This resolved several issues! Test 1 (SCP) and Test 4 (New file in Nautilus) now work well. Hurdles now are copying a file with whatever permissions in Nautilus to server (Test 3 - still 777 instead of 664). And the directory issues. Both server and client are Ubuntu (9.10 vs 10.10 though), and no additional ACLs are enabled.
Comment: Sounds like copying with Nautilus is preserving permissions like `cp -p` or `scp -p` would.
Comment: Any idea how to change this??
Comment: What behavior do you want?  You can set default permissions for new files by using default ACLs.  Nautilus will honor the default ACL.  See http://www.vanemery.com/Linux/ACL/linux-acl.html for an overview and http://www.suse.de/~agruen/acl/linux-acls/online/ for all the details.
Comment: Note that default ACLs do not enforce a maximum set of permissions, they just make those permissions the default.  `cp -p`, `scp -p`, and copying via Nautilus will still try to make the copy's permissions the same as the file being copied.
Comment: Hmmm... my goal is to always get 664 for files and 771 for directories being copied, independently from their original permissions... any way to achieve this in Nautilus?
Comment: Also remember that umask acts over 777 for directories and 666 for files, so you can't really have different umasks for files and dirs.
Answer: ACLs should work fine for you.
Set a default ACL on all folder for the group, which will then be inherited by all future files and directories.
Something like setfacl -m d:g:uploaders:rwx should work.
To fix your existing permissions:
find /shared/folder -type d -exec setfacl -m d:g:uploaders:rwx {} \;
find /shared/folder -type f -exec setfacl -m g:uploaders:rw {} \;

Looks like though if "preserve" is set when a file is copied, the default ACL doesn't work.  In that case, I can only suggest running the find commands in cron or monitoring the filesystem for changes.
Answer: This is not PAM/umask related, but it could be useful for you.
If you setgid a directory, then all the files and directories created inside it are automatically assigned to its group.
root@ricarda ~ # mkdir hello
root@ricarda ~ # chown :users hello
root@ricarda ~ # chmod g+s hello
root@ricarda ~ # ls -l |grep hello
drwxr-sr-x. 2 root users  4096 Feb  7 04:05 hello
root@ricarda ~ # touch hello/some_file
root@ricarda ~ # mkdir hello/some_dir
root@ricarda ~ # ls -l hello/
total 4
drwxr-sr-x. 2 root users 4096 Feb  7 04:16 some_dir
-rw-r--r--. 1 root users    0 Feb  7 04:06 some_file

Comment: Thanks! I did not know about this. It does not solve the permission issues, but it is useful indeed!
Answer: Please read this solution too: How do I set permissions structure for multiple users editing multiple sites in /var/www on Ubuntu 9.10?


Line integral comes out wrong after parameterization

Question: I'm integrating $\int_C xy dx + (x - y)dy$ along the line segment from $(2,0)$ to $(3,2)$. If I write a vector expression for the line using $r_0 - (r_1 - r_0)t$ I get $x = 2+t, y = 2t$. Then the integral becomes $(4t+2t^2)dt + 1/2(2 - t)dt$ from $0$ to $1$. This gives a different answer then if I express the line in terms of $x$ and $y$ so that $y = 2x - 4$ and integrate from $2$ to $3$ over $x$ and $0$ to $2$ over $y$. To me these should be equivalent, though?
Answer: I see mistake in your first working. If line segment is from $(2, 0)$ to $(3, 2)$,
$r(t) = (2 + t, 2 t), 0 \leq t \leq 1 $, which is what you have. But your line integral expression does not seem correct.
Next, $r'(t) = (1, 2)$. So line integral is,
$\displaystyle \int_0^1 (4t+2t^2, 2-t) \cdot (1, 2) \ dt = \int_0^1 { (2t^2 + 2t + 4) } \ dt = \frac{17}{3}$.
Your working shows a different integrand. I do not have details of your alternate working to confirm if you are doing it correctly.


Prove using correlation to do t-test is equivalent to the standard t-test formula in linear regression?

Question: $\newcommand{\Cor}{\operatorname{Cor}} \newcommand{\Cov}{\operatorname{Cov}}$My question is why the following expression holds?
$$t = \frac{\hat{\beta_1}}{\operatorname{se}(\beta_1)} = \frac{\Cor(Y,X)\sqrt{n-2}}{\sqrt{1 - \Cor^2(Y,X)}}$$
Here is what I got so far:
\begin{align}
\frac{\hat{\beta_1}}{\operatorname{se}(\beta_1)} &= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\operatorname{se}(\beta_1)}\\
&= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\sqrt{\frac{\sum(\hat{Y_i} - Y_i)^2}{n-2}\cdot\frac{1}{\sum(X_i - \overline{X})^2}}}\\
&= \frac{\frac{\Cov(Y,X)}{S_x^2}\sqrt{n-2}}{\sqrt{\frac{\sum(\hat{Y_i} - Y_i)^2}{\sum(X_i - \overline{X})^2}}}\label{a}\tag{1}
\end{align}
\begin{align}
\frac{\Cor(Y,X)\sqrt{n-2}}{\sqrt{1 - \Cor^2(Y,X)}}
&= \frac{\frac{\Cov(Y, X)}{S_xS_y}\sqrt{n-2}}{\sqrt{1 - \left(\frac{\Cov(Y, X)}{S_xS_y}\right)^2}} \label{b}\tag{2}
\end{align}
From $\ref{a}$ and $\ref{b}$, we then need to show:
\begin{align}
\frac{1}{S_x\sqrt{\frac{\sum(\hat{Y_i} - Y_i)^2}{\sum(X_i - \overline{X})^2}}} &= \frac{1}{S_y\sqrt{1 - \left(\frac{\Cov(Y, X)}{S_xS_y}\right)^2}}\\
&=\frac{1}{S_y\frac{\sqrt{S_x^2S_y^2 -\Cov^2(Y, X)}}{S_xS_y}}\\
&=\frac{1}{\frac{\sqrt{S_x^2S_y^2 -\Cov^2(Y, X)}}{S_x}}\label{c}\tag{3}
\end{align}
I tried to expand the $\Cov(Y, X)$ term and left $SSE$ and $SSX$ terms in $\ref{c}$, but there is no any further process.
I am wondering how to continue from $\ref{c}$, or my initial direction of the proof is not correct?
Answer: The trick is the following equation:$$\sum(\hat{Y_i} - Y_i)^2 = SSE = (1 - R^2)SST$$
This is why there is a $\sqrt{1 - \Cor^2(Y,X)}$ term in $(2)$. The whole proof is below:
$$
\begin{align}
t &= \frac{\hat{\beta_1}}{\operatorname{se}(\beta_1)}\\
  &= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\sqrt{\frac{\sum(\hat{Y_i} - Y_i)^2}{n-2}\cdot\frac{1}{\sum(X_i - \overline{X})^2}}}\\
  &= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\sqrt{\frac{(1 - R^2)SST}{n-2}\cdot\frac{1}{\sum(X_i - \overline{X})^2}}}\\
  &= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\sqrt{\frac{(1 - R^2)S_y^2(n-1)}{n-2}\cdot\frac{1}{S_x^2(n-1)}}}\\
  &= \frac{\frac{\Cov(Y,X)}{S_x^2}}{\frac{S_y}{S_x}\frac{\sqrt{1 - R^2}}{\sqrt{n - 2}}}\\
  &= \frac{\frac{\Cov(Y,X)}{S_yS_x}\sqrt{n-2}}{\sqrt{1 - R^2}}\\
  &= \frac{\Cor(Y,X)\sqrt{n-2}}{\sqrt{1 - \Cor^2(Y,X)}}
\end{align}
$$


What to do in response to repeat DRAM ECC error notifications for the same memory location?

Question: I woke up this morning to what's a first for me; one of my systems had logged DRAM ECC error notifications. Three of them, in fact, for as far as I can tell the exact same memory location (obviously, the system isn't actually named localhost):
Aug 31 05:00:46 localhost kernel: [719099.816034] [Hardware Error]: CPU:0   MC4_STATUS[-|CE|MiscV|-|AddrV|-|-|CECC]: 0x9c6c40006b080a13
Aug 31 05:00:46 localhost kernel: [719099.816046] [Hardware Error]:         MC4_ADDR: 0x0000000641f49d20
Aug 31 05:00:46 localhost kernel: [719099.816051] [Hardware Error]: Northbridge Error (node 0): DRAM ECC error detected on the NB.
Aug 31 05:00:46 localhost kernel: [719099.816059] EDAC amd64 MC0: CE ERROR_ADDRESS= 0x641f49d20
Aug 31 05:00:46 localhost kernel: [719099.816070] EDAC MC0: CE page 0x641f49, offset 0xd20, grain 0, syndrome 0x6bd8, row 2, channel 0, label "": amd64_edac
Aug 31 05:00:46 localhost kernel: [719099.816075] [Hardware Error]: cache level: L3/GEN, mem/io: MEM, mem-tx: RD, part-proc: RES (no timeout)

The above was followed by an identical notification at system time 05:10:46 (719699.8160) and then one more at 05:20:46 (720299.8160) which also had Over on the CPU:0 MC4_STATUS line (status 0xdc6c40006b080813). So far the system has been stable since, with no further errors logged. System activity is normal, and the system in question has been running with ECC RAM since 2014 but never logged any ECC errors.
I wouldn't be too worried about a single correctable ECC error. The almost exactly ten minutes (down to a few microseconds, in fact) in between the errors being logged could be simply for RAM scrubbing happening every ten minutes; unfortunately, on this particular system, the scrub interval is not exposed as a setting. However, the three consecutive errors in the same memory location (same value for CE ERROR_ADDRESS) does have me a little bit concerned.
Update: The host in question has logged several more since I originally posted this question, all with the same value for CE ERROR_ADDRESS.
How seriously should I take this? What's a good response; order replacement RAM right away and schedule to install it ASAP, treat this as just a momentary glitch, or be on toes to replace RAM if it happens again but no specific action right now?
Answer: ECC RAM tends to be used on critical servers. The system is reporting a hardware failure. If it's not a critical system and you don't mind everything going through it potentially corrupting, sure wait and see what happens, but if you care about your data more than the cost of the RAM replace the faulty RAM ASAP.
Comment: In all fairness, the system is reporting a *corrected* hardware failure. I'm not saying that these should be ignored; I'm asking how urgent they are. Obviously, since more such errors have been logged since I originally asked the question, that does increase the urgency somewhat, but I am still curious what to make of the fact that the errors seem to consistently occur at the exact same memory address.
Comment: If an address is consistently reported, I have to assume there's a fault at that address. ECC can correct a single bit error. If this error gets worse then the error could become uncorrectable any time.
Answer: I'd suggest to run memtest86+
http://www.memtest.org
It's also included in some distributions as standard package.
It may confirm your suspicion on faulty memory module.
Comment: I already know that there is a (as of yet correctable) problem with the RAM -- the system is flat out saying so in the logs, repeatedly even -- so I don't really need to run a testing tool (memtest86+ or otherwise) to confirm that. Doing so might be a good idea if the system is behaving *erratically*, but the whole point of having ECC RAM in the first place is to ensure that the system can gracefully handle memory errors.
Comment: Is it some HP/HPE system? In that case

hpasmcli -s "show dimm"

may signal you which module is wrong too.

Generally, when the error happens on the same address repeatedly, I'd consider the module in the near fault state.
If it would be in different memory locations and not so often, that can happen. Especially with high RAM sizes. However this doesn't seem to be the case.
Comment: From your output I guess module 2 of channel 0 of the first socket is the culprit. Check your RAM layout in the motherboard documetation.
Answer:
I woke up this morning to what's a first for me; one of my systems had logged DRAM ECC error notifications. Three of them, in fact, for ... I wouldn't be too worried about a single correctable ECC error. The almost exactly ten minutes (down to a few microseconds, in fact) in between the errors being logged could be simply for RAM scrubbing happening every ten minutes; unfortunately, on this particular system, the scrub interval is not exposed as a setting.

Wikipedia's webpage on Memory Scrubbing says:

"Over 8% of DIMM modules experience at least one correctable error per year. This can be a problem for DRAM and SRAM based memories. The probability of a soft error at any individual memory bit is very small.".
"In order to not disturb regular memory requests from the CPU and thus prevent decreasing performance, scrubbing is usually only done during idle periods. As the scrubbing consists of normal read and write operations, it may increase power consumption for the memory compared to non-scrubbing operation. Therefore, scrubbing is not performed continuously but periodically. For many servers, the scrub period can be configured in the BIOS setup program.

That webpage contains a link to the SuperMicro X9SRA motherboard manual which explains the scrubbing interval:

"Patrol Scrub
  Patrol Scrubbing is a process that allows the CPU to correct correctable memory errors detected on a memory module and send the correction to the requestor (the original source). When this item is set to Enabled, the North Bridge will read and write back one cache line every 16K cycles, if there is no delay caused by internal processing. By using this method, roughly 64 GB of memory behind the North Bridge will be scrubbed every day. The options are Enabled and Disabled.".

Thus, the cause is not from scrubbing. It's possible that there is a faulty bit. While a fault might occur suddenly it seems odd that it goes away and comes back, especially when it occurs so frequently.

"How seriously should I take this? What's a good response; order replacement RAM right away and schedule to install it ASAP, treat this as just a momentary glitch, or be on toes to replace RAM if it happens again but no specific action right now?"

Pavel Machek, whom invented the nohammer kernel module says:

"It is fairly hard to do rowhammer by accident, so if you are hitting it, someone is probably doing it on purpose. ... Well, there's more than three orders of magnitude difference between cosmic rays and rowhammer. IIRC cosmic rays are expected to cause 2 bit flips a year... rowhammer can do bitflip in 10 minutes, and that is old version, not one of the optimized ones.".

You can exchange the RAM modules and see if the error report follows the chip, sticks with the memory location, or occurs elsewhere.
HPE recommends (for a faulty memory module):

"SYMPTOM: The below error message is found in the OS logs:

host1 kernel: Northbridge Error (node X): DRAM ECC error detected on the NB.

FIX:
  1. Identify the Memory module number that has failed (if mentioned in the error)
  2. Check IML for Error relating to Memory module. Ex Proc x slot x
  3. Update System BIOS
  4. If no errors are found run diagnostics and replace the memory module (5-6 loops of Memory Diagnostics to isolate the memory module)"

Suggested course of action:

Switching RAM in it's sockets will tell you if it's a specific RAM module or if the fault is in other circuitry.
As long as you don't get more than one bit error every few days there's no panic (rush).
If you're getting hit every 10 minutes you might be getting hammered.

See also: "Defending against RowHammer in the kernel" and "ECCploit: ECC Memory Vulnerable to Rowhammer Attacks After All". For ARM processors there's: "Android GuardION patches to mitigate DMA-based Rowhammer attacks on ARM".


Is there a term that define the number of notes in a chord?

Question: There are triads (3 notes), 7th chords (4 notes), 9th chords (5 notes), 11th chords (6 notes), 13 chords (seven notes).
Is there some commonly used term that is used only to describe the quantitative property of a chord. Something like arity..
It seems incorrect to use term "type" for it, because "type" also defines the qualitative properties of a chord (intervals between chord notes).
Comment: 11th and 13th chords more often than not don't play all the notes they could. With 11ths, the 9 is missed, with 13ths, the key notes are 1,3,7 and 13. - not an answer, thus a comment. I think I asked a similar question some time ago.
Comment: Yes I now that commonly only 4 of the chord notes are played.
Answer: As an extension of Michael's reference to post-tonal music theory, in that realm we refer to the cardinality of sets. A tetrachord, for instance, is a set of cardinality four, meaning that there are four pitches (or pitch classes) involved. A pentachord has cardinality five, etc.
Although this terminology is typically reserved for post-tonal music theory, I have heard individuals refer to a tonal sonority of cardinality x.
Answer: As @user45266 says 'tetrad' etc are logical choices, but they aren't commonly used.
Common practice classical music on the whole just used triad, seventh chords, and to some degree ninth chords. In terms of quantity concepts like arity or sets I think the traditional musical terms is voice or part as in four part harmony, etc. Bach's famous two and three part inventions are an example of the usage. I think it would be perfectly acceptable to say something like 'four voice chord' or 'five part chord.' The meaning should be clear.
However, traditionally a large number of parts - like five part harmony - would often involve octave doubling of tones. This loops us back around to common practice harmony. The triad really is the 'default' chord is some seventh chords and fewer ninth chords. If you want to un-ambiguously convey n-voices of unique pitch classes, four-voice, five-voice, etc. will not be absolutely clear.
You may be interested in these mathy music terms: tertian for chords built of thirds, and quartal and quintal for chords built of 4ths and 5ths.
Tetrachord is a common term, but paradoxically it typically refers to a scale-like set of tones rather than a tertian chord.
Pitch set is a musical term and it has all the quantifiers you are looking for, the only potential down side to that term is its association with atonal music. Not that association with atonal music is a negative (that's a joke) but the only musicians who are likely to be familiar with set theory terminology are those with a lot of academic training.
Answer: Not certain of any specific term besides the "-ad" system.
Basically, two-note chords are dyads (whatever you want to say about two-note chords). Three are obviously triads. Four, I imagine, would be a tetrad.
Down the list, we have:

monad
dyad
triad
tetrad
pentad
hexad
heptad
octad
nonad
decad

Thanks to Wiktionary for that list. Not super-useful, though, outside of academia. Most of the time, people say "n-note chord" (except "triad", which is such a common theory term that everyone uses it). I'd consider a ninth chord to be a pentad even if it's only voiced with 4 notes, but I don't know how others feel about that.


Como adicionar Folha de Estilo em um Projeto Slim Framework

Question: Como consigo configurar um projeto que eu possa adcionar arquivos de css, js e bootstrap no slim framework, usei o projeto slim/skeleton, mas não consigo acicionar meus arquivos css e js sem a necessidade de adicionar um longo caminho ex(c:\xampp\htdocs\projeto\template\assests\style.css), existe uma possibilidade onde eu possa apenas adicionar <link href="style.css" />
Comment: Tente adicionando um caminho relativo: `assests/style.css`
Answer: 1) Crie a pasta 'assets' dentro do seu diretório public e dentro delas as pastas 'css', 'js' e 'images'.
2) Você tem que indicar no .htacess as configurações seguintes para que de seu diretório principal onde está o arquivo index.php, a sua pasta assets e os índices de pastas dentro dela sejam visíveis:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /

#Checks to see if the user is attempting to access a valid file,
#such as an image or css document, if this isn't true it sends the
#request to index.php
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond $1 !^(index\.php|assets)
RewriteRule ^(.*)$ index.php?/$1 [L]

</IfModule>

3) Após criar um layout padrão de página seguindo a documentação do Slim para templates, (eu utilizei o Twig), você deve colocar nesta página padrão suas configurações e aí você pode colocar diretamente e só a pasta assets, pois com a configuração acima no .htacess seu template já terá acesso e visibilidade aos arquivos. Segue um exemplo de configuração deste layout padrão:
<!DOCTYPE html>

        {% block head %}
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">

        <link type="text/css" href="/assets/css/bootstrap.min.css" rel="stylesheet">
        <link type="text/css" href="/assets/css/jquery.dataTables.min.css" rel="stylesheet">
        <link type="text/css" href="/assets/css/style.css" rel="stylesheet" />

        <link type="text/css" href="/assets/css/bootstrap.min.css" rel="stylesheet">
        <link type="text/css" href="/assets/css/jquery.dataTables.min.css" rel="stylesheet">
        <link type="text/css" href="/assets/css/style.css" rel="stylesheet" />

        <link  rel = "ícone de atalho"  href = "favicon.ico"  type = "imagem / x-ícone" >
        <link  rel = "icon"  href = "favicon.ico"  type = "imagem / x-ícone" >

        <title>{% block title %}{% endblock %}</title>
        {% endblock %}

{% block content %}{% endblock %}

            {% block footer %}
            © Copyrigh by Corporation
            {% endblock %}

No exemplo acima eu subi os arquivos padrões de css, js dentro das pastas correspondentes no diretório assets. Adicionei uma folha de estilo chamada 'style.css' adicionando algumas configurações minhas de estilo.
4) Ao criar uma nova página em seu diretório de Views, você deve extender o layout padrão (primeira linha do arquivo exemplo abaixo), você pode adicionar seguindo as tags do Twig configurações adicionais dentro das tags block possíveis (como no exemplo abaixo adicionei uma função em javascript, os outros block correspondes abaixo como ahead, title, body, footer):
{% extends "template.html" %}

{% block title %}Corporation{% endblock %}

{% block head %}
{{ parent() }}

{% endblock %}

{% block content %}

    Corporation Title

            Home

            Link 2

            Link 3

            Link 4

Coloque seu contéudo aqui...

{% endblock %}

{% block footer %}

© Corporation -  Todos os direitos reservados

{% endblock %}


How to await the on 'end' event of a Node Stream API?

Question: I need to wait for the task of this API to finish before moving to the next task.
How would I add a promise / await to this syntax?
I need to use the enqueue function and then wait for the stream to complete before continuing any further code. I have tried returning a promise withing the .on('end', {}) call but with no luck.
Any help is much appreciated.
const {SiteChecker} = require('broken-link-checker');

const siteChecker = new SiteChecker(options)
  .on('error', (error) => {})
  .on('robots', (robots, customData) => {})
  .on('html', (tree, robots, response, pageURL, customData) => {})
  .on('queue', () => {})
  .on('junk', (result, customData) => {})
  .on('link', (result, customData) => {})
  .on('page', (error, pageURL, customData) => {})
  .on('site', (error, siteURL, customData) => {})
  .on('end', () => {});

siteChecker.enqueue(siteURL, customData);

Answer: Your code is almost fine, but in order to use Promises you need to wrap the whole code in it. You'd achieve that like this:
const {SiteChecker} = require('broken-link-checker');

const siteChecker = new SiteChecker(options)
  .on('error', (error) => {})
  .on('robots', (robots, customData) => {})
  .on('html', (tree, robots, response, pageURL, customData) => {})
  .on('queue', () => {})
  .on('junk', (result, customData) => {})
  .on('link', (result, customData) => {})
  .on('page', (error, pageURL, customData) => {})
  .on('site', (error, siteURL, customData) => {});

await new Promise((resolve, reject) => siteChecker
       .on('end', resolve)
       // resolve is a function that you call when the work is done
       .on('error' => reject)
       // reject is the function you call when the task fails
       .enqueue(siteURL, customData);
);

Now I'm not sure how the broken-link-checker works in your case, but if you wanted to reuse the whole solution then it would be best to wrap the whole thing in the promise and return the siteChecker reference like this:
const {SiteChecker} = require('broken-link-checker');

const siteChecker = await new Promise((resolve, reject) => {
  const _sitechecker = new SiteChecker(options)
    .on('error', (error) => {})
    .on('robots', (robots, customData) => {})
    .on('html', (tree, robots, response, pageURL, customData) => {})
    .on('queue', () => {})
    .on('junk', (result, customData) => {})
    .on('link', (result, customData) => {})
    .on('page', (error, pageURL, customData) => {})
    .on('site', (error, siteURL, customData) => {});
    .on('end', () => resolve(_sitechecker))
    // notice that the sitechecker is passed here
    .on('error' => reject);

    sitechecker.enqueue(siteURL, customData);
});

Both ways of writing this will wait until completion (or until the end event is emitted).


IMO 2006, A4 Problem, Idea behind the proof

Question: IMO 2006, Problem A4, page 13:

Prove the inequality:
  $$
   \sum_{i<j} \frac{a_{i}a_{j}}{a_{i}+ a_{j}} \le
   \frac{n}{2(a_{1} + a_{2} + \cdots + a_{n}) }\sum_{i<j}a_{i}a_{j}
$$
  for positive real numbers $a_{1}$, $a_{2}$, ..., $a_{n}$.

Solution (when I try to hide it, I've got the formlas broken - sorry)
Let $S = \sum\limits_{i} a_{i}$. Denote by $L$ and $R$ the expressions on the
left and right hand side of the proposed inequality. We transform $L$
and $R$ using the identity:
$$
 \sum_{i < j} (a_{i} + a_{j}) = (n-1) \sum_{i}a_{i}.
$$
And thus:
$$
  L = \sum\limits_{i<j} \frac{a_{i}a_{j}}{a_{i} + a_{j}} = \frac{n-1}{4} S -
  \frac{1}{4}\sum_{i < j} \frac{ (a_{i} - a_{j})^2}{a_{i} + a_{j}}.
$$
To represent $R$ we express the sum $\sum\limits_{i<j}a_{i}a_{j}$ in two ways:
$$
 \sum_{i<j} a_{i}a_{j} = \frac{1}{2}\left(S^2 - \sum_{i} a_{i}^2\right)
$$
$$
   \sum_{i<j}a_{i}a_{j} = \frac{n-1}{2} \sum_{i}a_{i}^2
- \frac{1}{2}\sum_{i<j}(a_{i} - a_{j})^2.
$$
Multiplying the first of these equalities by n-1 and adding the
second one we obtain
$$
  n\sum_{i<j}a_{i}a_{j} = \frac{n-1}{2} S^2 -
  \frac{1}{2}\sum_{i<j}(a_{i} - a_{j})^2
$$
Hence
$$
  R = \frac{n}{2S} \sum_{i<j}a_{i}a_{j} =
\frac{n-1}{4}S - \frac{1}{4}\sum_{i<j}\frac{(a_{i} - a_{j})^2}{S}
$$
Now compare the last two equalities. Since $S \ge a_{i} + a_{j}$
for any $i<j$, the claim $L \ge R$ results.
My question: what is the idea behind this proof, and how to think it out?
Answer: The idea behind this proof: try to rewrite our inequality in the following form: $$\sum_{1\leq i<j\leq n}(a_i-a_j)^2S_{ij}\geq0$$ and to hope that $S_{ij}\geq0$.
This method names SOS (Sum Of Squares). It's a very useful method.
See here:
https://math.stackexchange.com/tags/sos/info
https://math.stackexchange.com/questions/tagged/sos
Comment: What do you mean by $S_{ij}$?
Comment: @Simeon Y In our case $S_{ij}=\frac{\sum\limits_{k=1}^na_k}{a_i+a_j}-1>0$
Comment: Oh, thank you! That is very helpful comment!
Comment: @Simeon Y You are welcome!


Como fazer deploy de uma aplicação em NodeJS local?

Question: Fala pessoal , tudo bem ?
Andei pesquisando na web mas fiquei confuso , sou novo na programação em node fiz uma aplicação mas não sei como fazer o deploy local como um serviço por exemplo. Por onde posso começar ? é possível fazer isso ?
Agradeço pela ajuda desde já.
Answer: Eu costumo usar o pm2 para rodar o servidor como um tipo de serviço. É bem simples de usar.
npm install pm2@latest -g
Para usar se estiver na raiz do projeto, onde se encontra o index.js, mains.js ou server.js:
pm2 start index.js
Para parar é pm2 stop index ou pm2 stop <nome da instancia>
Após o start ele libera o console aí é só monitorar o status pm2 status
Os logs pm2 logs e para mais informações pm2 show <nome da instancia>
ou pm2 show index.
Geralmente eu carrego um script.json contendo várias informações como as variáveis de ambiente, nome da instancia, etc. Também há alguns comandos para salvar a instancia do pm2 e toda vez que o computador ou servidor iniciar/reinicializar ele starta novamente. Na documentação dele explica tudo certinho.

Referência: PM2


C++ Unidentifiable Memory Leak

Question: I have 2 files, one of which is my testing class. My code compiles and executes, but there is still one memory leak concerning my allocating and freeing of my memory. I haven't created any objects personally while writing this code so I am confused as to how a memory leak exists.
#include <algorithm>

class sorted_sc_array {
public:

sorted_sc_array() : size_(0), arraySize(0), ptr_(nullptr) { }

~sorted_sc_array() { delete[] ptr_; }

sorted_sc_array(const sorted_sc_array& A) : size_(A.size_) {

  if(size_ == 0) ptr_ = nullptr;
  else {
    delete[] ptr_;
    ptr_ = new signed char[size_];
    std::copy(A.ptr_, A.ptr_ + size_, ptr_);
  }

  // int x = A.size_;
  // this -> size_ = 0;
  // for(int i = 0; i < x; i++){
  //   this -> insert(A.data()[i]);
  // }
}

sorted_sc_array& operator=(const sorted_sc_array& A){

  if (this ==&A) return *this;
  delete[] ptr_;
  size_ = A.size_;
  if (size_ == 0) ptr_ = nullptr;
  else{
    ptr_ = new signed char[size_];
    std::copy(A.ptr_, A.ptr_ + size_, ptr_);
  }
  return *this;

  //
  // int x = A.size_;
  // this -> size_ = 0;
  // for(int i = 0; i < x; i++){
  //   this -> insert(A.data()[i]);
  //   }

}

// RETURNS SIZE OF THE ARRAY (i.e. HOW MANY ELEMENTS IT STORES)
int size() const { return size_; }

// RETURNS RAW POINTER TO THE ACTUAL DATA, CAN BE INVOKED AT ANY TIME
const signed char* data() const {
   std::sort(ptr_, ptr_ + size_);
   return ptr_;
   delete[] ptr_;
 }

void insert(signed char c){

  if(size() == 0){
    ptr_ = (signed char*)malloc(2*sizeof(char));
    ptr_[0] = c;
    ptr_[1] = 0;
    arraySize = 1;
  } else if(arraySize == size_) {
    ptr_ = (signed char*)realloc(ptr_, (size_ * 2)*sizeof(char));
    arraySize = arraySize * 2;
    ptr_[size_] = c;
  //  ptr_[size_ + 1] = 0;
    //arraySize = arraySize * 2;

  }  else {
  //  ptr_ = (signed char*)realloc(ptr_, (size_ + 2)*sizeof(char));
    ptr_[size_] = c;
  //  ptr_[size_ + 1] = 0;
  }
  size_++;
  // std::sort(ptr_, ptr_ + size_);
  }

private:
int size_;         // size of the array
signed char* ptr_; // pointer to the array
int arraySize;

}; // class sorted_sc_array

#endif // A2_HPP

This is testing the other class:
#include <iostream>
#include "a2.hpp"

int main(int argc, char* argv[]) {
sorted_sc_array A;

{
    sorted_sc_array T;
    for (signed char c = -128; c < 127; ++c) T.insert(c);

    T = T;

    sorted_sc_array V = T;
    A = V;
}

const auto first = A.data();
const auto last = first + A.size();

auto size = A.size();
bool res = std::is_sorted(first, last);

if (!res || (A.size() != 255)) std::cout << "fail";
else std::cout << "pass";

std::cout << std::endl;

return 0;
} // main

And this is the result from valgrind:
==10205== Memcheck, a memory error detector
==10205== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==10205== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==10205== Command: ./a2
==10205==
==10205== Conditional jump or move depends on uninitialised value(s)
==10205==    at 0x400E35: sorted_sc_array::sorted_sc_array(sorted_sc_array
const&) (a2.hpp:30)
==10205==    by 0x400BC2: main (a2.cpp:22)
==10205==
==10205== Mismatched free() / delete / delete []
==10205==    at 0x4C2F74B: operator delete[](void*) (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10205==    by 0x400DF2: sorted_sc_array::~sorted_sc_array() (in
/home/vagrant/A2/a2)
==10205==    by 0x400BED: main (a2.cpp:17)
==10205==  Address 0x5ab6fb0 is 0 bytes inside a block of size 256 alloc'd
==10205==    at 0x4C2FD5F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-
amd64-linux.so)
==10205==    by 0x401036: sorted_sc_array::insert(signed char) (a2.hpp:87)
==10205==    by 0x400B8A: main (a2.cpp:18)
==10205==
pass
==10205==
==10205== HEAP SUMMARY:
==10205==     in use at exit: 72,704 bytes in 1 blocks
==10205==   total heap usage: 13 allocs, 12 frees, 74,750 bytes allocated
==10205==
==10205== LEAK SUMMARY:
==10205==    definitely lost: 0 bytes in 0 blocks
==10205==    indirectly lost: 0 bytes in 0 blocks
==10205==      possibly lost: 0 bytes in 0 blocks
==10205==    still reachable: 72,704 bytes in 1 blocks
==10205==         suppressed: 0 bytes in 0 blocks
==10205== Reachable blocks (those to which a pointer was found) are not
shown.
==10205== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==10205==
==10205== For counts of detected and suppressed errors, rerun with: -v
==10205== Use --track-origins=yes to see where uninitialised values come
from
==10205== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

Comment: This code is a mess. You have `malloc` and `realloc` mixed in with `new` and `delete`. Also, in the `data()` function, you try to `delete` something after the function returns.
Comment: Just use `std::vector`. Hey, `std::vector`. You could just use `std::vector`.
Comment: your copy constructor's `delete[] ptr_;` seems to be problematic. it points to some hyperspace in memory, and you try to `delete[]` it.
Comment: Are you treating the `still reachable` output as a leak?  Did you run with the options `valgrind` told you to use?  `--leak-check=full --show-leak-kinds=all`?
Answer: Look at your valgrind output: it's not complaining about a memory leak, it's complaining about uninitialized memory (you delete an uninitialized pointer in your copy constructor) and mismatching malloc with delete[] (you need to be consistent with allocators).
Remove the offending delete [] and use new [] instead of malloc and realloc.

C++ Unidentifiable Memory Leak


What are the possible reasons why Knn works much better than SVM?

Question: So, I have my training data and my test data and the problem is that using KNN it works very good (in fact works almost perfectly) but using SVM or some CART Models works very bad.
So, I'd like to know some possible reasons why this is happening. I think that the code that I'm using it is well implemented.
This is a problem of classification for more than $2$ classes.
Answer: There are a lot of other things you need to clarify for your question to be answered. kNN and SVM both have different learning methods. Some questions you will need to clarify further are:
What kernel are you using for SVM?
Did you tune the hyperparameters for SVM using grid search etc?
Are you measuring the metric on the the training set itself?
What k did you take for kNN?
What is the dimensionality of your data?
Number of training points?
Knn performing almost perfectly can happen by choosing k=1 and calculating the error metric on the train set itself
Maybe if you clarify these things there will be a more clearer picture of the situation.


Draw random numbers from finite mixture model

Question: Setup
Let X follow a fininte mixture model with density
\begin{equation}
f=\lambda f_1+(1-\lambda) f_2
\end{equation}
Where $f_1$ and $f_2$ are both log-normal densities with parameters $(\mu_1, \sigma_1)$, and $(\mu_2, \sigma_2)$ respectively and $\lambda \in [0,1]$ is the mixing probability. Let $Z_i$ be a random variable with density $f_i$, i=1,2
I want to generate a set of N realisations from the random variable X.
In practice I saw two ways to do this:
Method 1
For each draw: choose with probability $\lambda$ a realisation from $Z_1$ and with probability $(1-\lambda)$ a realisation from $Z_2$.
Method 2
Draw N realisations from $Z_1$ and N realisations from $Z_2$, to obtain the two vectors $\mathbf{z}_1=(z_{11},z_{21}, ...z_{N1})^T $ and $\mathbf{z}_2=(z_{12},z_{22}, ...z_{N2})^T$.
Then obtain the desired vector of N realisations as
\begin{equation}
\mathbf{x}=\lambda \mathbf{z_1}+(1-\lambda) \mathbf{z_2}
\end{equation}
Question
I find Method 1 intuitive, but Method 2 seems strange. If we would impose limited support for $f_1$ and $f_2$ and allow, for example, only integer values, then the draws of X would contain values which are not in the support of either $Z_1$ or $Z_2$. This said, I did some simulations, and the histograms I plotted are identical. Hence, the question:
Are Method 1 and Method 2 asymptotically equivalent? Are they equivalent in small sample as well? Is one of the two methods just wrong?
Thanks a lot for your help!
Comment: For the reasons you point out, Method 2 is obviously wrong.  It draws from a *linear combination* of the two distributions rather than their mixture.  It does not have $f$ for its density.
Comment: Problem solved. Thank you very much!
Answer: The second method is not only strange but wrong. The first method is correct. The definition of mixture distribution says that the $\lambda$ and $1-\lambda$ weights are applied to the distribution functions, not to the random variables*. To convince yourself, try both methods. You can find such an example below. The example uses a mixture of Gaussians, where the probability density function is shown in red. As you can see, in the first case the histogram matched the probability density function. In the second case, the histogram of the samples is unimodal vs the target distribution is bimodal, so it's clearly wrong. In fact, we know that sum of Gaussians is Gaussian, and this is what we see on the second plot.

If you wanted to simplify the sampling, you can notice that the number of the samples $n_1$ from $Z_1$ would follow a binomial distribution with a probability of success $\lambda$ and the sample size $N$, so you can draw $n_1$ from the binomial distribution and then take $n_1$ samples for $Z_1$ and $N-n_1$ for $Z_2$ and they would together form a sample from the mixture of size $N$.
* - Think of a trivial example where we multiply Bernoulli distributed random variable by $2$. If you multiplied the values, you have a random variable with two possible values $0$ and $2$ occurring with probabilities $p$ and $1-p$. If you multiplied the probability distribution function by $2$, you would have two values $0$ and $1$ with probabilities that $2p$ and $2(1-p)$ which can be higher than $1$ so invalid. Multiplying the distribution vs the random variable are different things.
Comment: Thanks a lot! Yea I should have done simulations with normals rather than log normals. Your histograms look much more clear than mine. Thank you so much.


How to combine where and first in go range

Question: I am new to Go and Hugo site generator and currently creating a simple theme. I am trying to combine a where filter along with first function and I am not able to make it work.
What I want is to get first 10 items in the post section
{{ range where .Data.Pages "Section" "post" }}

{{.Title}} {{.Summary}}

{{ end }}

The above works fine, but how do I make it return only the first 10 items (the below does not work):
{{ range first 10 where .Data.Pages "Section" "post" }}

{{.Title}} {{.Summary}}

{{ end }}

Answer: Here's an example from the Hugo Template Functions documentation that I think means you're just missing parentheses:
{{ range first 5 (where .Data.Pages "Section" "post") }}
   {{ .Content }}
{{ end }}


Criar pastas a partir de informações das células, com o arquivo "teste.txt" dentro de cada pasta criada (VBA)

Question: Eu tenho uma planilha com algumas informações, eu preciso que estas informações se tornem pastas e que dentro de cada pasta haja um arquivo chamado "teste.txt"
Em uma pesquisa, encontrei o seguinte código:
Sub MakeFolders()
    Dim Rng As Range
    Dim maxRows, maxCols, r, c As Integer
    Set Rng = Selection
    maxRows = Rng.Rows.Count
    maxCols = Rng.Columns.Count
    For c = 1 To maxCols
        r = 1
        Do While r <= maxRows
            If Len(Dir(ActiveWorkbook.Path & "\" & Rng(r, c), vbDirectory)) = 0 Then
                MkDir (ActiveWorkbook.Path & "\" & Rng(r, c))
                On Error Resume Next
            End If
            r = r + 1
        Loop
    Next c
End Sub

Com esse código é possível apenas criar as pastas sem nenhum arquivo dentro, alguém pode me ajudar a completar o código?
Answer: Declarar como Integer
Primeiro uma observação:

Observação: Declarar a Maxrows e Maxcols como Long (Dim Maxrows As Long), pois muitos tutoriais antigos utilizam o Integer, que possui 2 bytes e
  o intervalo de     –32 768 a 32 767. Portanto, se a versão do Excel for
  maior do que 2007, o programa irá parar após a linha 32767. Já o Long
  possui 4 bytes e intervalo de -2 147 483 648 a 2 147 486 647. Em que o
  Excel possui um limite de 1 048 576 linhas.

E declarando da forma:
Dim maxRows, maxCols, r, c As Integer
Somente c está como Integer, todas as outras variáveis são Variant.
O correto seria:
Dim maxRows As Long, maxCols As Long, r As Long, c As Long
De resto o código está correto e funcionaria. Ele funciona com Selection, então primeiro o intervalo desejado deve ser selecionado.
Solução
Eu gosto de verificar se o caminho existe antes de tentar criar a pasta, para evitar erros (O que já ocorre no código com If Len(Dir(ActiveWorkbook.Path & "\" & Rng(r, c), vbDirectory)) = 0 Then).
Com ActiveWorkbook.Path & "\" & Rng(r, c) o caminho do diretório do arquivo Excel está sendo utilizado como referência para criar as pastas.
Segue a maneira que eu utilizo adaptado ao seu código, que não precisa necessariamente estar na mesma pasta que o arquivo Excel, mas o caminho completo deve ser inserido:
Sub MakeFolders()
    Dim Rng As Range
    Dim maxRows As Long, maxCols As Long, r As Long, c As Long
    Set Rng = Selection
    maxRows = Rng.Rows.Count
    maxCols = Rng.Columns.Count
    For c = 1 To maxCols
        r = 1
        Do While r <= maxRows
            CriarCaminho (Rng(r, c))
            r = r + 1
        Loop
    Next c
End Sub

Public Function CriarCaminho(ByVal path As String) As Boolean

    CriarCaminho = True
    Dim fso As Object
    Set fso = CreateObject("Scripting.FileSystemObject")

    If fso.FolderExists(path) Then
        Exit Function
    Else
        On Error GoTo DeadInTheWater
        fso.CreateFolder path
        Exit Function
    End If

DeadInTheWater:
    MsgBox "A folder could not be created for the following path: " & path & ". Check the path name and try again."
    CriarCaminho = False
    Exit Function

End Function

Referência
Para utilizar o Scripting.FileSystemObject, a referência deve ser adicionada no VBE.
Em Ferramentas -> Referências...
Adicione o 'Microsoft Scripting Runtime'
Comment: Muito obrigado pela ajuda!

Por algum motivo travou o Excel  mas deve ser hardware kk

O código abaixo funcionou.
Answer: Acrescentei a sub geraArqTXT abaixo do seu código, basicamente, essa sub cria uma nova planilha e a salva em formato de texto. Dentro do seu código, chamei a sub, enviando o nome da pasta concatenado com o nome do arquivo (e uma '\').
Sub MakeFolders()
    Dim Rng As Range
    Dim maxRows, maxCols, r, c As Integer
    Set Rng = Selection
    maxRows = Rng.Rows.Count
    maxCols = Rng.Columns.Count
    For c = 1 To maxCols
        r = 1
        Do While r <= maxRows
            If Len(Dir(ActiveWorkbook.Path & "\" & Rng(r, c), vbDirectory)) = 0 Then
                MkDir (ActiveWorkbook.Path & "\" & Rng(r, c))
                filename = (ActiveWorkbook.Path & "\" & Rng(r, c)) & "\teste.txt" 'sua pasta + teste.txt
                geraArqTXT filename 'passando pra função
                On Error Resume Next
            End If
            r = r + 1
        Loop
    Next c
End Sub

Public Sub geraArqTXT(fn As String)

    Dim work As Workbook
    Dim sht As Worksheet
    Set work = Workbooks.Add 'cria uma nova planilha
    Set sht = work.Worksheets.Add 'cria uma nova pasta

sht.SaveAs fn, xlTextPrinter 'salva sua pasta de trabalho como texto no destino recebido (fn)

work.Close SaveChanges:=True
    Set work = Nothing

End Sub

Comment: Muito obrigado pela ajuda.

O código no início deu erro mas depois acrescentei () que faltou em filename e deu super certo.


VAR with trend-stationary variables

Question: I'm currently trying to estimate a VAR with 3 variables - consumption, investment and a credit spread. I have inspected the variables and run ADF tests to determine that they are in-fact trend-stationary rather than processes with a unit root. On running a Johansen test to check if they cointegrate, I find that the LR equilibrium matrix has a rank of 1. I have 2 questions:

Does it make sense to run a cointegration test on trend stationary variables?
In order to estimate the VAR, should I detrend the variables and use the detrended data in my estimation?

Any help would be much appreciated!
Comment: Apologies, I didn't know I was meant to do that, thank you very much for the help!
Comment: No problem, you are welcome!
Answer:
No, it does not make sense to run a cointegration test on trend stationary variables. Cointegration requires variables with unit roots.
You can either detrend the variables beforehand or include a trend in the VAR model as an exogenous regressor.

VAR with trend-stationary variables


Script Not Working

Question: I am trying to integrate 3rd party jquery plugin Turnjs 3rd edition available on github in my website.
For this I have uploaded turnjs script on my server and called it in website through my theme.info file and my layout.inc
Installed jquery update and selected jquery 1.7
created a content type and added multi image field through which I have created this node
http://patelbrothers.com.pk/content/mag1
which displays all the images of the field like following

and the CSS
    .field--name-field-flipping-book-images .field__items{
width:600px;
height:392px;
}
.field--name-field-flipping-book-images .field__items{
background-color:#ccc;
background-size:100% 100%;
}

After that using js injector module I have inserted the following before the closing body tag but still the script is not working.
    $(window).ready(function() {
    $('.field--name-field-flipping-book-images .field__items').turn({
                        display: 'double',
                        acceleration: true,
                        gradients: !$.isTouch,
                        elevation:50,
                        when: {
                            turned: function(e, page) {
                                /*console.log('Current view: ', $(this).turn('view'));*/
                            }
                        }
                    });
});

$(window).bind('keydown', function(e){

    if (e.keyCode==37)
        $('.field--name-field-flipping-book-images .field__items').turn('previous');
    else if (e.keyCode==39)
        $('.field--name-field-flipping-book-images .field__items').turn('next');

});

Comment: Did you check if all required assets, specifically the Turnjs script, are loaded in the browser? If not this may be a Drupal problem. Otherwise, this question seems off-topic to me.
Comment: Please check my answer below, Drupal require to wrap the code which I inserted through jsinjector module in
(function ($) {
// All your code here
})(jQuery);
Answer: It worked :)
Just wrapped my code inn
(function ($) {
// All your code here
})(jQuery);

For Reference https://www.drupal.org/update/modules/6/7#javascript_compatibility

Script Not Working


Goodness of fit test for a mixture in R

Question: I just estimated the parameters for a mixture of two gaussians with different means and different sigmas, I would like to test if the data adjusts well to the explicit form of the mixture, do I necessarily need to simulate data from the mixture or how can I test the goodness of fit? I am using the mixtools package.
Comment: Question flagged for migration, so you don't have to add a new question to that site.
Comment: A nonparametric option is the [Kolmogorov-Smirnov test](http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test) but there are probably some ad hoc tools for the case of Gaussian mixtures.
Answer: You can write a function that calculates the relevant value for a given test under the null hypothesis based on the outputs from, say, normalmixEM, then do the test using that.  For example, for the Kolmogorov-Smirnov test, we need the CDF given a set of parameters:
# CDF of mixture of two normals
pmnorm <- function(x, mu, sigma, pmix) {
  pmix[1]*pnorm(x,mu[1],sigma[1]) + (1-pmix[1])*pnorm(x,mu[2],sigma[2])
}

We then run the K-S test in the usual way:
# Sample run
x <- c(rnorm(50), rnorm(50,2))

foo <- normalmixEM(x)
test <- ks.test(x, pmnorm, mu=foo$mu, sigma=foo$sigma, pmix=foo$lambda)
test

    One-sample Kolmogorov-Smirnov test

data:  x
D = 0.0559, p-value = 0.914
alternative hypothesis: two-sided

Always keeping in mind the fact that we estimated the parameters from the same data we are using to do the test, thus biasing the test towards failure to reject H0.
We can overcome that latter bias to some extent via a parametric bootstrap - generating many samples from a mixture of normals parameterized by the estimates from normalmixEM, then estimating the parameters of the samples and calculating the test statistics for each sample using the estimated parameters.  Under this construction, the null hypothesis is always true.  In the code below, I'm helping the EM algorithm out by starting at the true parameters for the sample, which is also cheating a little, as it makes the EM algorithm more likely to find values near the true values than with the original sample, but greatly reduces the number of error messages.
# Bootstrap estimation of ks statistic distribution
N <- length(x)
ks.boot <- rep(0,1000)
for (i in 1:1000) {
  z <- rbinom(N, 1, foo$lambda[1])
  x.b <- z*rnorm(N, foo$mu[1], foo$sigma[1]) + (1-z)*rnorm(N, foo$mu[2], foo$sigma[2])
  foo.b <- normalmixEM(x.b, maxit=10000, lambda=foo$lambda, mu=foo$mu, sigma=foo$sigma)
  ks.boot[i] <- ks.test(x.b, pmnorm, mu=foo.b$mu, sigma=foo.b$sigma, pmix=foo.b$lambda)$statistic
}

mean(test$statistic <= ks.boot)
[1] 0.323

So instead of a p-value of 0.914, we get a p-value of 0.323.  Interesting, but not particularly important in this case.
Comment: Does anything change in the following case? I have 100 simulated obs from a mixture of two Normals. So i know the means and variances of the "true" mixture. I generate from the predictive distribution 20000 samples. How can i test if those samples are from the specific mixture? I believe i have to skip the foo variable and substitute the known means and variances in the ks.test right? or i am missing something bigger here?
Comment: @Christos - Your intuition is correct - better to substitute the known means and variances in the ks test, and, if you do so, you don't need to bootstrap the ks test statistic either.
Comment: Could you please explain how a $p=0.914$ implies that the data from the mixture distribution? I thought failure to reject H0 did not mean that.
Comment: @Jacob - where did I say that a p-value of 0.914 implies that the data is from a mixture distribution?
Comment: @jbowman: I'm sorry, you obviously don't. However, is there a statistical test to establish equivalence? I.e. given a CDF and a small sample of data, is there a statistical test which can prove that the data follows the CDF?
Comment: @Jacob - unfortunately, no.  Consider for example data that is drawn from a Normal(0,1) distribution.  Now consider an alternative where the data is drawn from a Normal(0, 1 + 1.0e-9999999999999999999999999) distribution.  You would need a large, not small, sample size to distinguish between the two.  And if you had a large enough sample, we can always construct even smaller differences between the two CDFs that your sample won't be large enough to distinguish between.
Comment: @jbowman: Thank you for explaining this to me. Would things be different if I wanted to test the equality of a statistic? E.g. non-parametric equivalence of means test. I'm trying to find what I need from Stefan Wellek's "Testing Statistical Hypotheses of Equivalence" ; do you think that's the right direction?

Goodness of fit test for a mixture in R


Definir retorno de resultado de função ou método

Question: Edição: Uma função anterior gera uma lista assim:
['Responsável atual', 'Número do processo', 'Foro de tramitação', 'Data da intimação 05/10/2017', 'Data da ação 15/08/2011']

Essa é a lista prévia constante em "Dados". Dados é meu relatório final. Ele contém centenas de processos listados. Eu pego um a um desses processos, acesso e ADICIONO as informações de "Partes". Ou seja, eu INCREMENTO essa lista com informações complementares, que são, por exemplo:
['Exequente', 'União', '(CNPJ do exequente)', 'Executado', '(CNPJ fictício) 00.345.123/0001-00', 'Nome da empresa executada']

A minha função PERCORRE essa lista, ENCONTRA o CNPJ (ou o CPF) alvo e TRATA essa informação para determinar quem será o NOVO responsável. Por isso tem vários elif, pois ele considera o número 3 (imediatamente antes da barra). Se ali tiver 0, 9 ou 8 ele IGNORA e pega o anterior (no caso, o 2). Se ali tiver 0, 9 ou 8 ignora e pega o anterior e assim por diante, por isso é meio confuso o critério de determinação do responsável. No final, quando eu tenho determinado quem é o responsável de acordo com esse critério, eu preciso acrescentar todas essas novas informações dessa lista naquela lista prévia (ou seja, acrescento "Partes" + 'distribuicaoPara' a "Dados") para concluir o relatório...
#############################
Tenho uma função:
def PegaDadosPartes(dados):
      distribuicaoPara = VerificaDist(partes)
      dados.append(str(distribuicaoPara) + " @")
      for x in range(len(partes)):
          dados.append(partes[x] + '@')
      del partes[0:len(partes)]
      return dados

Ela recebe um conjunto de dados (lista 'partes') de outra função, e alimenta uma outra lista ('dados') com essas informações, SENDO QUE a variável 'distribuicaoPara' chama a função 'VerificaDist(partes)' para obter seu valor.
Esse valor depende de uma série de tratamentos. Dentre eles, verificar se em 'partes' há CNPJ's ou CPF's, e tratar diferenciadamente caso a caso, assim:
def VerificaDist(partes):
      posicao = []
      temCnpj = 0
      for x in range(len(partes)):
            verificaOutroCnpj = re.search("\d{2}.\d{3}.\d{3}/\d{4}-\d{2}", partes[x])
            if verificaOutroCnpj:
                  posicao.append("CNPJ " + verificaOutroCnpj.group())
                  temCnpj = temCnpj + 1
            verificaCpf = re.search("\d{3}.\d{3}.\d{3}-\d{2}", partes[x]) #Se não achou CNPJ, busca CPF.
            if verificaCpf:
                  posicao.append("CPF " + verificaCpf.group())
      if temCnpj > 0:
            Caso1(posicao) #Primeiro caso: consta CNPJ.
      else:
            Caso2(posicao) #Segundo caso: consta CPF.

Tanto em um quanto no outro caso, as funções de tratamento irão "fatiar" a variável, pois os critérios de tratamento são diferentes. No CNPJ o dígito antes da "/" determina o responsável. No CPF, o oitavo dígito define isso. A função final faz isso:
def TrataDistribuicao(alvos): #("Alvos" são as partes "fatiadas" da informação)
      analisa = open('C:\\Regras.txt', 'r') #É o arquivo que contém as regras de designação de responsável, tipo "Se o dígito é 8 responsável fulano, se 7 cicrano" e assim por diante...
      regrasDeDistribuicao = analisa.readlines()
      for x in range(len(regrasDeDistribuicao)):
            regrasDeDistribuicao[x] = regrasDeDistribuicao[x].split(',')
      for x in range(len(regrasDeDistribuicao)):
            if alvos[0] == regrasDeDistribuicao[x][0]:
                  distribuicaoPara = str("Distribuir para " + regrasDeDistribuicao[x][1] )
            elif alvos[1] == regrasDeDistribuicao[x][0]:
                  distribuicaoPara = str("Distribuir para " + regrasDeDistribuicao[x][1])
            elif alvos[2] == regrasDeDistribuicao[x][0]:
                  distribuicaoPara = str("Distribuir para " + regrasDeDistribuicao[x][1] )
            elif alvos[3] == regrasDeDistribuicao[x][0]:
                  distribuicaoPara = str("Distribuir para " + regrasDeDistribuicao[x][1])
analisa.close()

Ocorre que, no final de tudo isso, não estou conseguindo "setar" a variável "distribuiçãoPara". Como fazer?
Comment: Sinceramente, eu reli sua pergunta umas 5x e ainda não consegui entender. Você tem uma função, `PegaDadosPartes`, que recebe apenas um parâmetro, `dados`, e essa função depende do retorno de outra função, `VerificaDist`, que recebe como parâmetro `partes`, que nem pertence ao escopo; o retorno desta função é adicionado a `dados`, mas depois você itera sobre `partes`, que, de novo, não pertence ao escopo, adicionando seus elementos em `dados`, para no final retornar `dados`. Não fez sentido. Poderia explicar exatamente o que está tentando fazer, pois acho que esta sua solução não é viável.
Comment: E por quê na função `TrataDistribuicao` há 4 `if/elif` que fazem exatamente a mesma coisa?
Comment: Eu editei a pergunta tentando ser mais claro. Obrigado pela atenção!!!
Answer: Eu adicionei um return distribuicaoPara em cada função posterior, prevendo todas as possibilidades de definição da variável, e o valor finalmente foi retornado! Grato a todos.
Alterei também o laço. Ficou assim:
def TrataDistribuicao(alvos):
analisa = open('C:\\Regras.txt', 'r')
regrasDeDistribuicao = analisa.readlines()
for x in range(len(regrasDeDistribuicao)):
    regrasDeDistribuicao[x] = regrasDeDistribuicao[x].split(',')

resultado = None #Acrescentei essa variável para servir de critério pra saída do laço.
for x in range(len(alvos)):
    for y in range(len(regrasDeDistribuicao)):
        if alvos[x] == regrasDeDistribuicao[y][0] and resultado == None:
            resultado = ("Distribuir para " + regrasDeDistribuicao[y][1] + " considerado " + dados[0])

if resultado == None:
    distribuicaoPara = "CNPJ ou CPF não coincidente com nenhuma situação previamente prevista. Verificação manual."
    return distribuicaoPara
else:
    distribuicaoPara = resultado
    return distribuicaoPara
analisa.close()


A doubt regarding change of variables in Double Integrals.

Question: So I have recently learnt to calculate "Double Integrals" and I have a doubt on change of variables. Lets say we are evaluating a double integral of the form I = $\int\int _R F(x,y) dx dy $ , where $F(x,y)$ is an integrable smooth function and $R$ is the region of integration in the $xy$ plane. Now I want to convert to polar co-ordinates $x(r,\theta)=r \cos(\theta)$ and $y(r,\theta) = r \sin(\theta)$. Now the region $R$ in the $xy$ palne can be suitably converted to some region $R'$ in the $r \theta$ plane. The function $F(x,y)$ may take the form of some smooth and integrable $G(r,\theta)$. My main doubt is with the change of the area element. According to my textbook, $dx dy = \frac{\partial (x,y)}{\partial(r,\theta)} = rdrd\theta$. So the area element in the $r\theta$ plane is $rdr d\theta$.
Now I want to experiment a little bit with my understanding : I write $dx = d(r\cos\theta) = -r\sin\theta d\theta + \cos \theta dr$  and $dy = d(r\sin\theta)=r\cos\theta d\theta + \sin\theta dr $. And now I multiply the two expression for $dx$ and $dy$ : $dxdy= -r^2\sin\theta\cos\theta d\theta d\theta - r \sin^2 \theta drd\theta + r \cos^2 \theta dr d\theta + \sin \theta \cos\theta dr dr$. Why isn't this expression equal to $rdrd\theta$ ? The $d\theta d\theta$ and $dr dr$ terms look very odd to me. Am I commiting some abuse of notation or something ? Also there are no higher order terms like $dr dr d\theta$ which I can ignore. I am totally confused with all this. Can someone explain this to me a bit clearly? Thanks in Advance :).
EDIT 1 : I know differentials are not the "usual" kind of algebraic quantities and we can't just add and multiply them casually. I am just multiplying $dx$ with $dy$ with the spirit of multiplying some $\delta x$ and $\delta y$ and letting both $\delta x$ and $\delta y$ tend to zero.
EDIT 2 : Is there a branch of Mathematics where addition and multiplication of differentials is very common? Is there something like differential of a differential ?
Comment: Edit 2: Differential geometry. The symbol $dx$ is a differential form.
Comment: This appears to be a duplicate of [Why can't we convert the area element $dA$ to polar by multiplying the polar expressions for $dx$ and $dy$?](https://math.stackexchange.com/questions/3769936/), but I cannot vote to close because of the bounty. There is a very old [meta post](https://math.meta.stackexchange.com/questions/11210/) about this sort of situation, but I'm not sure what would be appropriate these days. I am not sure if I should repost my answer here and vote to close the other question as a duplicate, for instance.
Answer: Side Question

Is there a branch of Mathematics where addition and multiplication of differentials is very common? Is there something like differential of a differential ?

There are lots of different contexts/formalizations with involved manipulation of differentials. Trying to give an overview of all of them would be a big undertaking worthy of a whole separate question. Very briefly: The most common thing that is taught/that I would expect a mathematician to think of is the theory of "differential forms". But there are also many other relevant things like "geometric calculus", "Smooth Infinitesimal Analysis", Bartlett and Khurshudyan's approach to manipulating higher order differentials (arXiv link), debatably: Robinson's approach to nonstandard analysis, and probably a couple other things I'm forgetting or haven't encountered.
Main Question

Why isn't this expression equal to $rdrd\theta$ ?

Disclaimer
This is almost entirely copied from my answer to the similar question Why can't we convert the area element $dA$ to polar by multiplying the polar expressions for $dx$ and $dy$?.
Intro
There are two main types of ways to think about things like $\mathrm{d}x\mathrm{d}y$ in multivariable calculus, and we often switch between them depending on the context. (This clarification was inspired in part by Terry Tao's preprint on "differential forms and integration".) $\mathrm{d}x$ can either act kind of like a number, or act kind of like a vector.
For the "number" interpretation, there are things like limit arguments or infinitesimals in nonstandard analysis to make things rigorous. For the "vector" interpretation, there are things like "differential forms" or "geometric calculus" to make things rigorous. But I'm going to gloss over those details because there are many ways to make things formal, and the exact choices don't affect the intuition here.
Numbers
One way to think about things is that $\mathrm{d}x$ and $\mathrm{d}y$ are in some way like tiny positive numbers representing the width and length of a tiny rectangle, so that $\mathrm{d}x\mathrm{d}y$ is the area of a tiny rectangle. Then when we write something like $\iint f\left(x,y\right)\,\mathrm{d}x\mathrm{d}y$ or $\iint g\left(r,\theta\right)\,\mathrm{d}r\mathrm{d}\theta$, we just add up the signed volumes (in case $f$ or $g$ is negative) of thin rectangular prisms with cross-sectional area represented by $\mathrm{d}x\mathrm{d}y$ or $\mathrm{d}r\mathrm{d}\theta$.
Under this interpretation, $\mathrm{d}x=\mathrm d(r\cos\theta)=\cos\theta\mathrm{d}r-r\sin\theta\mathrm{d}\theta$ doesn't make too much sense. For example, if $\theta=\pi/2$, then we would have $\mathrm{d}x=-r\mathrm{d}\theta$, so that $\mathrm{d}\theta$ and $\mathrm{d}x$ couldn't both represent positive lengths. But we can still understand the relationship between the areas $\mathrm{d}x\mathrm{d}y$ and $\mathrm{d}r\mathrm{d}\theta$ with arguments like the geometric one in this answer by Mike Spivey.
Vectors
The other way to think about things is that $\mathrm{d}x$ and $\mathrm{d}y$ are in some way like tiny vectors whose direction we care about, and this leads to a slightly different discussion. To emphasize this vector idea, I will use some nonstandard notation. Let's write $\overrightarrow{\mathrm{d}x}=\left\langle \Delta x,0,0\right\rangle$  for some positive $\Delta x$, and $\overrightarrow{\mathrm{d}y}=\left\langle 0,\Delta y,0\right\rangle$  for some positive $\Delta y$. So $\overrightarrow{\mathrm{d}x}$ points to the right in the $xy$-plane and $\overrightarrow{\mathrm{d}y}$ points "up" in the $xy$-plane. Then the area of the little rectangle they make is $\left\Vert \overrightarrow{\mathrm{d}x}\times\overrightarrow{\mathrm{d}y}\right\Vert =\Delta x\Delta y$.
However, now that we have vectors, we could choose to care about the orientation. When we think about a usual integral like $\int_{\left[a,b\right]}f\left(x\right)\,\mathrm{d}x$ when $f$ is negative, we decide to count that area in a negative way. Similarly, we could count the "signed area" of the oriented rectangle with sides $\overrightarrow{\mathrm{d}x},\overrightarrow{\mathrm{d}y}$ as the positive value $\Delta x\Delta y$ (which is just the $z$-component of $\overrightarrow{\mathrm{d}x}\times\overrightarrow{\mathrm{d}y}$). But the signed area of the oriented rectangle with sides $\overrightarrow{\mathrm{d}y},\overrightarrow{\mathrm{d}x}$ would be the negative value $-\Delta x\Delta y$ (which is just the $z$-component of $\overrightarrow{\mathrm{d}y}\times\overrightarrow{\mathrm{d}x}$).
Then when we write something like $\iint f\left(x,y\right)\,\mathrm{d}x\mathrm{d}y$ or $\iint g\left(r,\theta\right)\,\mathrm{d}r\mathrm{d}\theta$, we could (if we chose) care about the order of things, with something like an implied $\overrightarrow{\mathrm{d}x}\times\overrightarrow{\mathrm{d}y}$ or $\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}$ in our heads, if not always our notation.
When using things like vectors, then something like $\overrightarrow{\mathrm{d}x}=\cos\theta\overrightarrow{\mathrm{d}r}-r\sin\theta\overrightarrow{\mathrm{d}\theta}$ makes a lot of sense. $\overrightarrow{\mathrm{d}r}$ points away from the origin, and $\overrightarrow{\mathrm{d}\theta}$ points perpendicularly to $\overrightarrow{\mathrm{d}r}$ in the counterclockwise way (so that $\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}$ points in the same positive-$z$ direction as $\overrightarrow{\mathrm{d}x}\times\overrightarrow{\mathrm{d}y}$).
Finally, we can correct your calculation:
$$\overrightarrow{\mathrm{d}x}\times\overrightarrow{\mathrm{d}y}=\left(\cos\theta\overrightarrow{\mathrm{d}r}-r\sin\theta\overrightarrow{\mathrm{d}\theta}\right)\times\left(\sin\theta\overrightarrow{\mathrm{d}r}+r\cos\theta\overrightarrow{\mathrm{d}\theta}\right)$$
$$=\cos\theta\sin\theta\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}r}-r^{2}\sin\theta\cos\theta\overrightarrow{\mathrm{d}\theta}\times\overrightarrow{\mathrm{d}\theta}+r\cos^{2}\theta\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}-r\sin^{2}\theta\overrightarrow{\mathrm{d}\theta}\times\overrightarrow{\mathrm{d}r}$$
$$=\overrightarrow{0}-\overrightarrow{0}+r\cos^{2}\theta\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}-r\sin^{2}\theta\left(-\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}\right)=\boxed{r\,\overrightarrow{\mathrm{d}r}\times\overrightarrow{\mathrm{d}\theta}}$$
Answer: My understanding of this is that in integration the area element which we denote $dA = dxdy$ is in fact a differential form as Miguel mentioned. A differential 2-form measures a certain amount of area, and is made up of differential 1-forms (which measure length) through the wedge product $\wedge$. The wedge product provides natural measures for a space through what's called the exterior algebra of the space. For example, given two vectors $\vec{a}$ and $\vec{b}$ the area of the parallelopiped they span is given by the scalar part of $\vec{a}\wedge \vec{b}$.
That is, when integrating over an oriented surface the area 2-form is $dA = dx\wedge dy.$ Because it describes oriented areas the wedge product is anticommutative, i.e. $dx\wedge dy = -dy\wedge dx$. Then when changing variables, you encounter the quadratic forms $d\theta d\theta$ for example. Interpreting the differentials here as measures (like in differential geometry), when you substitute your change of variables into $dx\wedge dy$ (called a pull-back if you want to get fancy), the terms with repeated differential 1-forms must vanish: $$d\theta \wedge d\theta = -d\theta\wedge d\theta = 0.$$
Differential forms in integration should be distinguished from Riemannian notation like $ds^2$, which really denotes a tensor product. A good elementary book on differential forms is "Differential Forms: Theory and Practice" by Weintraub. Also check out the page on exterior algebra on Wikipedia.
Comment: I would like to add as reference Agricola and Friedrich's book Global Analysis.
Answer: The change of variables transforms a small rectangular region $R$ with sides $\Delta x$ and $\Delta y$ in a region $R'$, which is not necessarily rectangular, so an area of $R'$ can differ from a product $\Delta r\cdot\Delta\theta$ of its "dimensions" $\Delta r$ and $\Delta\theta$.
For the case of polar coordinates, $R'$ is bounded by circles of radii $r$ and $r+\Delta r$ and radial lines directed from the origin at angles $\theta$ and $\theta+\Delta\theta $, so the area of $R'$ equals $$\frac 12\Delta\theta((r+\Delta r)^2-r^2)= r\Delta r\Delta\theta+\frac 12(\Delta r)^2\Delta\theta.$$


Melodic lines are diatonic, harmony doesn't seem to be

Question: Okay so this is something that I came across as I was writing my canons and analyzing the harmony to try to figure out which canon is best for my symphony. There is a section in one of my canons where the harmony seems to be outside of the key. Now I did get this sort of thing in another canon where I had a C power chord and then the rest of it was dissonant. But here, I seem to be getting something which is not only out of key but not all that related to the key. The key I am doing these canons in is Bb major.
Here is the section I am referring to:

My first instinct was that it was an F major chord in second inversion but without the root because of the key being Bb major. This though did not make sense. So then I decided on it being A minor. But then I started questioning myself because there is no E natural to make it clear that it is A minor and the C and the A are a sixth apart. So then I thought maybe it is A diminished without the tritone since it lands on Bb major 1.5 measures after the last statement of the chord in that section but once again it did not make sense. The tritone is what makes it sound diminished. Without the tritone, it seems wrong to call that chord A diminished.
So is the harmony in that section that I circled A minor or is it F major? And yes, the second line is a second up from the first line and delayed by 1.5 measures. If the harmony in the section I circled is indeed A minor, why did I get non-diatonic harmony out of a completely diatonic melody?
Answer: Trained musical ears tend to do two things when identifying what they hear:

Make a determination that is most in line with prevailing musical practice.
Make a determination that makes the least unnecessary assumptions (in short, Occam's Razor).

This is why, when we hear a harmonic intervals 4 half steps wide, we'll typically hear it as a major third instead of a diminished fourth: because prevailing musical practice results in major thirds being more common than diminished fourths, and because a diminished fourth academically requires some type of extra accidentals.
So when I see your excerpt, I view this as an incomplete A-diminished triad. Hearing it as A minor breaks both of the above rules, because:

A minor is less common than A diminished in a piece that begins on a B♭.
A minor makes the assumption that either a) that opening B♭ is not tonic or b) the A minor is a chromatic chord.

From the standpoint of common-practice music theory, triads can be incomplete by omitting their chordal fifth; that's all that you've done here. But our ears will tend to assume the chordal fifth is E♭, not a chromatic E♮. (Our ears also probably won't assume a missing root, especially one that results in a non-standard second-inversion F-major chord.)
But you're right, not having the E♭ (and thus not having the tritone) definitely softens the sound. I would shy away from calling it A minor (for the reasons listed above), but you also might just choose to call it an A/C dyad.


Probability Dice Game

Question: Paul, Dave and Sarah are rolling a fair six sided die. Paul will go first, always followed by Dave, who is always followed by Sarah, who is always followed by Paul, and so on...
What is the probability that Sarah will be the first one to roll a six?
Now I know the probability of Sarah rolling a six first on the first sequence would be
(5/6)(5/6)(1/6)
But what would be the probability of her rolling a six, if she failed to on the first sequence, the second sequence.
This lead to me finding the equation
(((5/6)(5/6)(5/6))^s)((5/6)(5/6)(1/6))
The first part being the number of sequences, s, she did not roll a six and the second part being when she finally rolls a six.
So I am doing this right?
Could someone help me figure out what to do next, or if I am completely wrong point me in the right direction?
Comment: Hint: Sarah rolls the first $6$ if and only if **one** of the following events occurred: (i) 2 rolls resulted in non-6s and she rolled a 6 on her turn (3rd roll overall).  (ii) 5 rolls resulted in non-6s and she rolled a 6 on her turn (6th roll overall), (iii) 8 rolls resulted in non-6s and she rolled a 6 on her turn (9th roll overall, (iv).....  So, calculate the probability of each and add up the results.  The following series might help (after some judicious removal of common factors: $$x^2 + x^3 + x^4 + \cdots = \frac{x^2}{1-x}, \quad |x| < 1.$$
Answer: Hint.  There are many ways in which Sarah could be the first to roll a six:

Paul doesn't, Dave doesn't, Sarah does;

OR

Paul doesn't, Dave doesn't, Sarah doesn't, Paul doesn't, Dave doesn't, Sarah does;

OR

Paul doesn't, Dave doesn't, Sarah doesn't, Paul doesn't, Dave doesn't, Sarah doesn't, Paul doesn't, Dave doesn't, Sarah does;

OR etc.  I think you can work out the probability of each case - you have pretty much done that already.  Now how do you combine all the different cases to get your final answer?
Answer: Let $p$ be the probability that Sarah (ultimately) wins. This can happen in two ways:
(i) Paul and Dave strike out on their first tosses, and Sarah gets a $6$. This has probability $\frac{5}{6}\cdot\frac{5}{6}\cdot\frac{1}{6}$.
(ii) Paul, Dave, and Sarah strike out on their first tosses, but Sarah ultimately wins. Since after the three failures, the game in essence starts again, this has probability $\left(\frac{5}{6}\right)^3p$.
Thus
$$p=\frac{5^2}{6^3}+\frac{5^3}{6^3}p.$$
we have a linear equation for $p$. Solve. After some simplification we get $\frac{5^2}{6^3-5^3}$.
Answer: The probability that Sarah is the first to roll a $6$ is $\frac{25}{91}$. Think it this way: Suppose $n$ people roll a die, one by one, in a fixed order, until someone gets a $6$. Let $P_{ij}$ be the probability that player $i$ gets that first $6$ at round $j$, we have:
· Round $1$:
$P_{11} = \frac{1}{6}, \, P_{21} = \frac{1}{6}\left(\frac{5}{6}\right), \, P_{31} = \frac{1}{6}\left(\frac{5}{6}\right)^{2}, \, \ldots \,, \, P_{n1} = \frac{1}{6}{\left(\frac{5}{6}\right)}^{n-1}$
· Round $2$:
$P_{12} = {\left(\frac{5}{6}\right)}^{n}\frac{1}{6}, \, P_{22} = {\left(\frac{5}{6}\right)}^{n+1}\frac{1}{6}, \, P_{32} = {\left(\frac{5}{6}\right)}^{n+2}\frac{1}{6}, \, \ldots \,, \, P_{n2} = {\left(\frac{5}{6}\right)}^{2n-1}\frac{1}{6}$
· Round $3$:
$P_{13} = {\left(\frac{5}{6}\right)}^{2n}\frac{1}{6}, \, P_{23} = {\left(\frac{5}{6}\right)}^{2n+1}\frac{1}{6}, \, P_{33} = {\left(\frac{5}{6}\right)}^{2n+2}\frac{1}{6}, \, \ldots \,, \, P_{n3} = {\left(\frac{5}{6}\right)}^{3n-1}\frac{1}{6}$
· Round $k$:
$P_{1k} = {\left(\frac{5}{6}\right)}^{(k-1)n}\frac{1}{6}, \, P_{2k} = {\left(\frac{5}{6}\right)}^{(k-1)n+1}\frac{1}{6}, \, P_{3k} = {\left(\frac{5}{6}\right)}^{(k-1)n+2}\frac{1}{6}, \, \ldots \,, \, P_{nk} = {\left(\frac{5}{6}\right)}^{kn-1}\frac{1}{6}$
So $P_{ij} = \frac{1}{6}{\left(\frac{5}{6}\right)}^{(j-1)n + (i-1)}$. The probability that player $i$ gets the first $6$ is the sum of the probabilities over each of his turns: $P_i = \displaystyle \sum_{j = 1}^{\infty} \frac{1}{6}{\left(\frac{5}{6}\right)}^{(j-1)n + (i-1)} = \frac{{\left(\frac{5}{6}\right)}^{i-1}}{6\left(1 - {\left(\frac{5}{6}\right)}^{n}\right)}$
In your particular case, $n=3$ and Sarah is the third player so $i=3$.

Probability Dice Game


How to get node Bundle in hook_form_alter on field config form?

Question: Can you wonderful people help me out please? I am altering the field config form (content types > plan > manage fields > edit field) and cannot seem to get the proper bundle out. Every time I call the bundle() function it returns the string "field_config". However the bundle is supposed to be "plan". Perhaps it's different because I am on the field config form, and not a node form?
Below is my code, the things I've tried and the field object showing the proper bundle. Does anyone know of another solution for me to get "plan" out?
<?php

use Drupal\Core\Config\Entity\ThirdPartySettingsInterface;
use Drupal\Core\Form\FormStateInterface;
use Drupal\Core\Entity\ContentEntityForm;

/**
 * Implements hook_form_alter().
 */
function rent_plan_custom_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state, $form_id) {
// print $form_id;

  switch ($form_id) {

    case 'field_config_edit_form':

        print $form['#title'];

        $field = $form_state->getFormObject()->getEntity();
        $field_name = $field->getName();  //returns correct field_plan_purpose

        //first try
        print $field->bundle(); //<---THIS RETURNS "field_config"

        //second try
        $bundle = FALSE;
        if ($form_object = $form_state->getFormObject()) {
            if ($form_object instanceof Drupal\Core\Entity\ContentEntityForm) {
                $bundle = $form_object->getEntity()->bundle();
            } elseif ($form_object instanceof Drupal\field_ui\Form\FieldConfigEditForm) {
                $bundle = $form_object->getEntity()->bundle();
            }
            print $bundle; //<---THIS RETURNS "field_config"
        }

        //Field Object Results Below
        print '
';
        print_r($field);
        print '
';

    break;
  }
}

Results from print_r($field)
Drupal\field\Entity\FieldConfig Object
(
    [deleted:protected] =>
    [fieldStorage:protected] => Drupal\field\Entity\FieldStorageConfig Object
        (
            [id:protected] => node.field_plan_purpose
            [field_name:protected] => field_plan_purpose
            [entity_type:protected] => node
            [type:protected] => text_long
            [module:protected] => text
            [settings:protected] => Array

            ... lots of more stuff
        )

    [id:protected] => node.plan.field_plan_purpose
    [field_name:protected] => field_plan_purpose
    [field_type:protected] => text_long
    [entity_type:protected] => node
    [bundle:protected] => plan
    [label:protected] => Purpose
    [description:protected] =>
    [settings:protected] => Array
        (
        )

    ...lots more stuff
)

Answer: You're looking for FieldDefinitionInterface::getTargetBundle

Gets the bundle the field is attached to.
This method should not be confused with EntityInterface::bundle() (configurable fields are config entities, and thus implement both interfaces):

FieldDefinitionInterface::getTargetBundle() answers "as a field, which bundle are you attached to?".
EntityInterface::bundle() answers "as a (config) entity, what is your own bundle?" (not relevant in our case, the config entity types used to store the definitions of configurable fields do not have bundles).

e.g.
$target_bundle = $field->getTargetBundle();

Comment: Thank you! This was perfect for my needs and I see now how these methods are defined in FieldConfigBase
Answer: FieldConfigBase has a method on it(*) called  getTargetBundle(). That will retrieve the field name.  To be safe, you should also probably use getTargetEntityTypeId() to make sure you are dealing with a node:
$entity_type = $field->getTargetEntityTypeId();
$bundle = $field->getTargetBundle();
if ($entity_type === 'node' && $bundle === 'plan') {
  do_something_awesome();
}

Using a class-aware editor, like PHPStorm, will make things like this a lot easier because you add type-hints, and then it will autocomplete for you.  In most OO code and some procedural code, a good IDE will automagiclly figure out your types, so you don't need to add the hinting.
(*) You should really be looking at the interface(s) that these define.  The one of interest in this case is FieldDefinitionInterface.