Numerical Identities

Question: Please can someone explain if this identity is correct:
|a| = $\sqrt{a^2} \ $
I thought it should be: |a| = $(\sqrt{a})^2\ $
being that the former would produce an answer that is either positive or negative.
Thank you for your help.
PS: The full question was comparing say:
|2x|/|3y| with $(\sqrt{(-2x/3y)} )^2 \ $
Comment: $\sqrt{x}$ is always positive, when it is defined (for $x\ge 0$).
Comment: Yes, but the question does not state x >= 0. That is why I am not sure
Answer: Since $\sqrt{x}$ is defined in $\mathbb{R}$ only for $x\ge 0$ and it's always positive:
the first is correct and the absolute value is necessary , e.g. $\sqrt{(-2)^2}=|-2|=2$
the second is redundant since the square root exists only if $a>0$
An answer to the PS. require a discussion of the sign of $x/y$. Can you do this?
Comment: Thanks a lot. I understand.
Comment: You are welcome !  don't forget to accept if you like :)
Comment: Sorry, I am new to formatting, I must have mistakenly put the minus outside the brackets. I have corrected it. Is this the discussion you had in mind?
Comment: We must have $-x/y >0$ and this is done if $x$ and $y$ have opposite signs.
Answer: You are confusing the standard meaning of the square root symbol with a correct understanding of squares and square roots. There are in fact two solutions to the equation $x^2 = 9$. namely $3$ and $-3$. But the accepted convention is that $\sqrt{9} = 3$, not $\pm 3$.
Comment: Modded up; specifically, the symbol $\sqrt{9}$ is called the "principal square root" of 9.
Answer: If we examine your suggestion, we would have:
$$|(-3)|=\left(\sqrt{-3}\right)^{2}=\left(\sqrt{3}i\right)^{2} = 3i^{2} = -3 \neq 3$$
Moreover, the $\sqrt{\cdot}$ operator is defined to be non-negative for all non-negative real arguments, so $\sqrt{x^{2}}$ does give us a well defined value $\forall x \in \mathbb{R}$.
In response to your comment that $x \geq 0$ is not stated, we must remember that $x^{2}:\mathbb{R}\to \mathbb{R}^{*}$, where $\mathbb{R}^{*}$ is the set of non-negative real numbers. And so the square-root operator in $\sqrt{x^{2}}$ only ever has a non-negative argument.
Comment: Thanks a lot for your response.

Numerical Identities


Obtaining random number from a mixture of two normal distributions

Question: I want to sample from mixed normal distribution, first one is $N(1,2)$, second one is $N(5,4)$. I used rnorm(100, c(mean=c(1,5), sd=c(2,4))). Is this correct?
The problem I am trying to solve is sampling from the 2 distribution above, first one with 75%, second one with 25%.  Am I on the right track?
Edit:  I will rewrite the problem for clearance, with easier numbers. :)
I want to sample from $N(0,1)$ with 70% probability, and $N(100,10)$ with 30% probability.
Of course, that's just for sake of discussion, the actually distribution I am working with is n(21, 3.3), n(26,4).
Comment: how about `c(rnorm(75, 1, 5), rnorm(25, 2, 4)`
Comment: @David - you might want to make an answer out of the comment, so you can get "answer" credit for it.  Also, it should be rnorm(75,1,2) and rnorm(25,5,4).
Comment: @David: The 75, and 25 will give me exactly 75 and 25 samples for each distribution.  But the 75% and 25% are just the probabilities getting each, not exactly those numbers.
Comment: ^ +1. If you want to permute this result, install `gregmisc` and use the `permute` function.
Comment: @Arun: can I avoid using any package?
Comment: @Arun `permute` is a trivial wrapper of R built-in `sample` function, so you need no external packages.
Answer: If you want to sample unequally (with probability 0.7 and 0.3) from two gaussians with parameters $(\mu_1,\sigma_1^2)$ and $(\mu_2,\sigma_2^2)$, then you can probably try something like that:
n <- 100
yn <- rbinom(n, 1, .7)
# draw n units from a mixture of N(0,1) and N(100,3^2)
s <- rnorm(n, 0 + 100*yn, 1 + 2*yn)

In fact, this is one of the illustrations provided in Modern Applied Statistics with S, by Venables and Ripley (Springer, 2002; §5.2, pp. 110-111).
With different parameters, you can use an ifelse expression to select the mean and SD according to the binomial sequence given in yn, e.g. rnorm(n, mean=ifelse(yn, 21, 26), sd=ifelse(yn, 3.3, 4)). (No need to cast yn to a logical with as.logical.)
Comment: elegant. impressed! :)
Comment: @mike If you want to get the variance of the combined sample, you need to account for the different location parameters (or their estimates) as well.
Comment: @chl: I am looking at it, I don't see that page.
Comment: @mike My first example is exactly the same as the code you wrote: as the two theoretical distributions have the same mean but different variance, they only varied the SD, which defaults to 3 when the $i$th Bernoulli event (which has probability 0.05 of occurring) equals 1, 1 otherwise.
Comment: @chl: I am a little confused.  Could you explain what the ifelse() does in here?  I am guessing it matches up mu1 with sd1, mu2 with sd2?
Comment: @chl: Actually, can you explain what binom(n,size=3,prob=0.7) does?  How can you do size=3, meaning you are picking from numbers 1,2,3, isn't binomial just 0's and 1's?  Thanks a lot, you are a great help!
Comment: @mike (a) `ifelse(yn, 21, 26)` gives a vector of 21 (if `yn==1`) and 26's (if `yn==0`). (b) `binom(n,size=1,prob=0.7)` yields $n$ random 0/1 deviates drawn from a ${\cal B}(n, 0.7)$ (binomial with probability of observing a success, or 1, of 0.7).
Comment: @chl: I definitely need to look at what you said and understand it, it worked well.  The true mixture mean 0.72*21+0.28*26, and the mean from simulation have been very close. :)  But why the true variance is (0.72*3.3)^2+(0.28*4)^2=6.9, but my simulation gives around 17.39.
Comment: @chl: how would you carry out a simulation with this mixed variance: (0.72*3.3)^2+(0.28*4)^2, the sd probability simply don't add up!
Comment: @chi your answer is nice and I have learned from it, but I should note that mine is as much as 2x faster (at least on my machine): `n <- 1000000; p <- 0.7; print('chi'); system.time({yn <- rbinom(n, 1, p);  rnorm(n,0+100*yn, 1+2*yn)}); print('David'); system.time({yn <- runif(n, 0, 1) > p;  c(rnorm(sum(yn),0, 1), rnorm(sum(!yn), 100,2))})`
Comment: Mike, for your particular values the expression merely becomes `s <- rnorm(n, 21 + 5*yn, 3.3 + 0.7*yn)`  Here, $5=26-21$ and $0.7=4-3.3$.
Comment: @whuber: I suppose you are referring to my mixed variance question above? why 26-21 and 4-3.3?  That does not give me mixed variance (0.72*3.3)^2+(0.28*4)^2=6.6899.
Comment: @whuber: nevermind, I got what you are saying.  But var(s)=4.3 is still NOT close to (0.72*3.3)^2+(0.28*4)^2=6.6899, what is wrong?
Comment: @David I didn't see your update (+1) when I posted my reply. Per Elvis's suggestion, you may want to resample the resulting vector.
Comment: The formula you are using, Mike--a sum of squares of weighted standard deviations--does not correspond to anything useful.  It certainly is not a variance.
Answer: To accomplish the goal of sampling from an uneven mixture of distributions, the most straightforward approach is to sample separately, in proportion to the desired ratio:
 p <- 0.70 #P(from N(mu1, sd1))
 n.samps <- 10000
 mu1 <- 0
 sd1 <- 1
 mu2 <- 100
 sd2 <- 10

 x <- vector()
 for(i in 1:n.samps){
    b <- runif(1, 0, 1)
    if(b < p){
       x[i] <- rnorm(1, mu1, sd1)
     } else {
       x[i] <- rnorm(1, mu2, sd2)
     }
   }

this can be done ~50 x faster:
 binary <- runif(n.samps, 0, 1) > p
 x <- c(rnorm(sum(binary), 1, 2), rnorm(sum(!binary), 5, 4)

then to draw a sample:
sample(x, 1)

or to reshuffle:
x <- sample(x, n.samp)

Comment: A defect of this method is that `x` is ordered, you would need to shuffle it with `sample(x, length(x))`. The value of `sum(binary)` could be generated directly as `s <- rbinom(1,n,p)` and of course the value of `sum(!binary)` is `n-s`.
Comment: I edit the question, hopefully, it's clear now.  I have seen packages doing mixture normal, my guess is you can't do it with one line in rnorm()?  Or maybe it is very complicated?
Comment: @elvis thanks for pointing that out; the for loop output is shuffled, I just tried to vectorize after seeing chi's answer.


How to inspect a css file generated by Less?

Question: I just get running Less in a drupal site. I was exited until I try to inspect a css file generated by a less file.
The line numbers in the generated css file don't match the the line numbers in the less file.
Even though this is very understandable since the css file is a generated file, this is a big problem to me. I'm very used to inspect (using Chrome) to look for the line number of the style of the inspected element,then I would go to css file to look for this line and make the changes I need.
Now with less this is imposible
How do you manage this?
Comment: anything for chrome?
Comment: There was a bit much info for a comment, I've put it in an answer
Answer: There are extensions available for Firefox (sadly not for any other browsers that I can find).
There's the FireLess Plugin which is:

a Firebug extension that makes Firebug display the Less filenames and line numbers of LessPHP-generated CSS styles rather than those of the generated CSS

Also, if you're able to switch to using SASS (which is quite similar), there's FireSass, which:

allows Firebug to display the original Sass filename and line number of Sass-generated CSS styles.

There's nothing for Chrome at the moment (see Less/Sass debugging in Chrome Dev Tools/Firebug)
Comment: I see. I sadly have to say then that Less is useless for me.
Comment: Yup, I feel your pain; this is the main reason I haven't started using LESS/SASS yet as well :/
Answer: Add this line to the LESS source code at line 672
loadStyleSheets(function (e, css, _, sheet, webInfo) {
            if (e) {
                errors.add(e, e.href || sheet.href);
                reject(e);
            }
<code>        /*--->>>> THIS LINE---->>>*/ window.__lastCSS = css;

        if (webInfo.local) {
            less.logger.info("loading " + sheet.href + " from cache.");
        } else {
            less.logger.info("rendered " + sheet.href + " successfully.");
            css = postProcessCSS(css);
            browser.createCSS(window.document, css, sheet);
            cache.setCSS(sheet.href, webInfo.lastModified, css);
        }
        less.logger.info("css for " + sheet.href + " generated in " + (new Date() - endTime) + 'ms');
        if (webInfo.remaining === 0) {
            totalMilliseconds = new Date() - startTime;
            less.logger.info("less has finished. css generated in " + totalMilliseconds + 'ms');
            resolve({
                startTime: startTime,
                endTime: endTime,
                totalMilliseconds: totalMilliseconds,
                sheets: less.sheets.length
            });
        }
        endTime = new Date();
    }, reload, modifyVars);
</code>


Align centered components

Question: I don't know much CSS and I am facing the following problem:
I have some elements in a div like below and I want to align them. (make them all start from the same x coordinate). I also want to keep them in the middle of the page.

      {{ user.username }}

For now, they are centered like this:

The page looks like:
Comment: so right now, there seems to be text-align: center; called on them. is that necessary or can you just switch to text-align: left; and maybe add some padding where needed?
Comment: you could wrap them in a wrapper element that is only as wide it's content (with sth like display: inline-block; oder width: max-content;) and center that, while setting the elements alignment to text-align: left;
But depending on the length of some of the names, it will not look optically centered – that's just in the nature of this design.
Comment: I want them to be centered on the page, but at the same time aligned. I will add a photo with the whole look of the page, maybe it helps.
Answer: If your users div is already at center then you can remove the height:100vh and align-items: center property.

.main-div{
 display: flex;
 height:100vh;
 width: 100%;
 justify-content: center;
 align-items: center
}

.avatar {
  display: flex;
}

      {{ user.username }}

Align centered components


"/usr/bin/ld: cannot find -lz" but libz exists

Question: While making my program on Redhat based linux (amd64), I get this error:
/usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status

Searching internet reveals that I have t install zlib 32bit version. A quick search shows that I have that package
[root@localhost trunk]# find / -name libz*
/lib64/libz.so.1
/lib64/libz.so.1.2.3
/lib/libz.so.1
/lib/libz.so.1.2.3
/usr/lib64/libz.so
/usr/lib64/purple-2/libzephyr.so
/usr/lib64/libzip.so.1.0.0
/usr/lib64/libzip.so.1

As you can see both /lib and /lib64 contain the libz file. Next I tried to find the ldd search path and it shows that both 32 bit and 64 bit folders are included
[root@localhost trunk]# ldconfig -v | grep -v ^$'\t'
/usr/lib64/ctapi:
/usr/lib64/mysql:
/usr/lib64/tcl8.5/tclx8.4:
/usr/lib64/xulrunner-2:
/lib:
/lib64:
/usr/lib:
/usr/lib64:
/lib/i686: (hwcap: 0x0008000000000000)
/lib64/tls: (hwcap: 0x8000000000000000)
/usr/lib64/tls: (hwcap: 0x8000000000000000)
/usr/lib64/sse2: (hwcap: 0x0000000004000000)
/lib/i686/nosegneg: (hwcap: 0x0028000000000000)

SO I wonder why still I get that error!!!!
UPDATE
I installed zlib-devel (both i686 and x86_64) and here is the confirmation
 [root@localhost trunk]# yum list zlib-devel
 Installed Packages
 zlib-devel.i686     1.2.3-27.el6             @sl
 zlib-devel.x86_64   1.2.3-27.el6             @sl

However still I get the same error.
UPDATE 2
After fixing the incorrect symbolic link, I now have
[root@localhost trunk]# ls -l /lib/libz.so*
lrwxrwxrwx. 1 root root    18 Feb 23 22:58 /lib/libz.so -> /usr/lib64/libz.so
lrwxrwxrwx. 1 root root    13 Feb 18 12:06 /lib/libz.so.1 -> libz.so.1.2.3
-rwxr-xr-x. 1 root root 75332 Dec 10  2011 /lib/libz.so.1.2.3

But error still exists
SOLUTION
SOLUTION
I found that I have to install static libraries. Don't know why Makefile searches for static libraries! Please refer to my comments for the name of packages.
Thanks fuero for help
Comment: If an answer helped you resolve your issue, you should indicate so by upvoting and accepting it. This also indicates to the system, and community, that the matter has been resolved.
Answer: Install zlib-devel or add a libz.so symlink to the lib directories.
yum list lists ALL packages, not just the installed ones. To find out whether it's installed, try this:
yum list installed | grep zlib

This is from a RHEL 6.3 reference system:
# rpm -ql zlib-devel
/usr/include/zconf.h
/usr/include/zlib.h
/usr/lib64/libz.so                  <---- the symlink I mentioned
/usr/lib64/pkgconfig/zlib.pc
/usr/share/doc/zlib-devel-1.2.3
/usr/share/doc/zlib-devel-1.2.3/README
/usr/share/doc/zlib-devel-1.2.3/algorithm.txt
/usr/share/doc/zlib-devel-1.2.3/example.c
/usr/share/doc/zlib-devel-1.2.3/minigzip.c
/usr/share/man/man3/zlib.3.gz
# rpm -ql zlib-devel.i686
/usr/include/zconf.h
/usr/include/zlib.h
/usr/lib/libz.so                    <---- the symlink I mentioned
/usr/lib/pkgconfig/zlib.pc
/usr/share/doc/zlib-devel-1.2.3
/usr/share/doc/zlib-devel-1.2.3/README
/usr/share/doc/zlib-devel-1.2.3/algorithm.txt
/usr/share/doc/zlib-devel-1.2.3/example.c
/usr/share/doc/zlib-devel-1.2.3/minigzip.c
/usr/share/man/man3/zlib.3.gz

What the symlinks point to:
# ls /usr/lib/libz.so -lhA
lrwxrwxrwx. 1 root root 23 Feb 23 20:32 /usr/lib/libz.so -> ../../lib/libz.so.1.2.3
# ls /usr/lib64/libz.so -lhA
lrwxrwxrwx. 1 root root 25 Feb 23 20:24 /usr/lib64/libz.so -> ../../lib64/libz.so.1.2.3

Comment: see the updated answer
Comment: I think a wrong path is confusing `ld`. How can I find that?
Comment: another update. don't forget to run `ldconfig`. Perhaps sharing how you call the compiler helps.
Comment: OK. The problem with `-lz` will be solved by installing static version of zlib named `zlib-static`. Now I am getting some other `-l` errors like `-lm` and that will be solved by installing `glibc-static`. I don't know why the program searches for static librsries
Answer: If it helps anyone - my issue was that cmake was running /usr/bin/g++ -static-libstdc++ which caused the error output: /usr/bin/ld: cannot find -lstdc++.
The fix (on RHEL/CentOS) was to run sudo yum install libstdc++-static.x86_64 since it was looking for the static lib.
Comment: It did help me, thank you :-)


How to obtain cross-correlation function from cross-spectrum?

Question: I have implemented a piece of code based on the Lomb-Scargle approach for determining the cross-spectrum of two time series. My cross spectrum contains complex numbers and I have used the basic fft function in R with the option inverse=true to apply a inverse Fast Fourier Transform to obtain the cross-correlation but i get a complex valued cross correlation function which is not correct. What other methods should I use?
Answer: I will give you an answer for the mapping between the autocovariances and the spectrum of a time series. Your question is about the cross-correlation and cross-spectrum, but the ideas below may be helpful.
The mapping between the autocovariances or order $\tau$, $\gamma(\tau)$ and the spectral density, $f(\omega)$, can be expressed as follows:
$$
\gamma(\tau) = \int_{-\pi}^\pi e^{i\omega\tau} f(\omega) \, d\omega \,.
$$
Given the spectral density, the autocovariances can be obtained in R as follows:
# generate a sample series
set.seed(123)
x <- arima.sim(n=200, model=list(ar=c(0.6,-0.3,0.4)))
# sample autocovariances (to be revored from the periodogram)
gamma <- acf(x, type="cov", plot = FALSE)$acf[,,1]
# spectral density estimate (smoothed periodogram)
sde <- spectrum(x, spans = c(3,3), plot=FALSE)
# apply the mapping in the equation above for the frequencies at which
# the spectral density was estimated
w <- sde$freq # frequencies (omega)
b <- rep(NA, length(gamma))
for (tau in seq.int(0, length(gamma)-1))
{
  tmp <- exp(1i * tau * w * 2 * pi) * sde$spec
  b[tau+1] <- sum(tmp + Conj(tmp))/length(x)
}
head(cbind(gamma, Re(b)))
# [1,] 1.21408770 1.27297299
# [2,] 0.47336471 0.50364341
# [3,] 0.03438318 0.05592229
# [4,] 0.30775323 0.36804157
# [5,] 0.28104297 0.32918129
# [6,] 0.09878623 0.11652643
tail(cbind(gamma, Re(b)))
# [19,] -0.07139406 -0.087051153
# [20,] -0.14771390 -0.158636206
# [21,] -0.19801861 -0.194581525
# [22,] -0.07825159 -0.075446230
# [23,]  0.02632276 -0.004064849
# [24,] -0.10278970 -0.105784687
# Imaginary terms cancel out
Im(b)
# [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
# sample variance
var(x)
# [1] 1.220189
# variance inferred from the mapping
Re(b)[1]
# [1] 1.272973

Sample autocovariances and the values inferred from the mapping are close to each other:
plot(seq.int(0, length(gamma)-1), gamma, xlab = "lag order")
lines(seq.int(0, length(gamma)-1), Re(b))
legend("topright", bty = "n", lty = c(0, 1), pch = c(1, NA), legend =
  c("sample autocovariances", "autocovariances mapped from the spectral density"))

The following points will also apply for the cross-spectrum and cross-covariances:

More accurate values are expected when a smoothed version of the periodogram
(as obtained by spectrum) rather than the raw periogram Mod(fft(x))/(2*pi*length(x)).
Depending on the range and scale of frequencies used to compute
the spectrum, your definition of $\omega$ (w in the code) may change.
Observe that in the code above the integral is summed over the range $(0, \pi]$. The conjugate values are then added up so that the complete range $(-\pi, \pi]$ is covered. The imaginary part of the complex numbers will therefore cancel out.
There may be a way to use the inverse Fourier transform that you mention, but for a test version it may be safer to obtain the products $e^{i\omega\tau} f(\omega)$ in an explicit loop.
Comment: I initially start with two time series of sample size equal to 100, then I sample them to have two irregularly sampled time series of mean sampling intervals 1 and 4. In this case, what would be the normalization factor at the last line( the one where you divide by the length of x).
Comment: Probably the number of observations in the irregularly spaced series, but I'm not familiar with the periodogram for unevenly sampled time series.


Support vector machines and regression

Question: There's already been an excellent discussion on how support vector machines handle classification, but I'm very confused about how support vector machines generalize to regression.
Anyone care to enlighten me?
Answer: Basically they generalize in the same way. The kernel based approach to regression is to transform the feature, call it $\mathbf{x}$ to some vector space, then perform a linear regression in that vector space. To avoid the 'curse of dimensionality', the linear regression in the transformed space is somewhat different than ordinary least squares. The upshot is that the regression in the transformed space can be expressed as $\ell(\mathbf{x}) = \sum_i w_i \phi(\mathbf{x_i}) \cdot \phi(\mathbf{x})$, where $\mathbf{x_i}$ are observations from the training set, $\phi(\cdot)$ is the transform applied to data, and the dot is the dot product.  Thus the linear regression is 'supported' by a few (preferrably a very small number of) training vectors.
All the mathematical details are hidden in the weird regression done in the transformed space ('epsilon-insensitive tube' or whatever) and the choice of transform, $\phi$. For a practitioner, there are also questions of a few free parameters (usually in the definition of $\phi$ and the regression), as well as featurization, which is where domain knowledge is usually helpful.
Comment: From an intuition viewpoint, is this almost like a single-class classification, where the class "boundary" line ends up running through the points instead of between the points of two classes?
Comment: @Wayne, that's my understanding, yes.  I'm not 100%, however.
Comment: I am confused by this answer. To explain how SVM or SVR works, for the linear case, no kernel method is involved. Shouldn't we be able to explain the relationship between SVM and SVR without talking about the kernel method ?
Answer: For an overview of SVM: How does a Support Vector Machine (SVM) work?
Regarding support vector regression (SVR), I find these slides from http://cs.adelaide.edu.au/~chhshen/teaching/ML_SVR.pdf  (mirror) very clear:

The Matlab documentation also has a decent  explanation and additionally goes over the optimization solving algorithm: https://www.mathworks.com/help/stats/understanding-support-vector-machine-regression.html (mirror).
So far this answer has presented the so-called epsilon-insensitive SVM (ε-SVM) regression.  There exists a more recent variant of SVM for either classification of regression: Least squares support vector machine.
Additionally, SVR may be extended for multi-output a.k.a. multi-target, e.g. see {1}.

References:

{1} Borchani, Hanen, Gherardo Varando, Concha Bielza, and Pedro Larrañaga. "A survey on multi‐output regression." Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5, no. 5 (2015): 216-233. https://scholar.google.com/scholar?cluster=10208375872303977988&hl=en&as_sdt=0,14; https://web.archive.org/web/20170628222235/http://oa.upm.es/40804/1/INVE_MEM_2015_204213.pdf

Support vector machines and regression


Question about soloing

Question: I am soloing over a chord progression of A,D, and C#m. A G# note fits all but the D chord. Can I still hit the G# note when playing a D chord?
It sounds good to me but I'm not sure. Musically, does this logic make sense?
Answer: Most of the time, if it sounds good to you it's okay. If it sounds good to the band, that's even better. If it sounds good to the audience, that's the best.
But you don't know until they hear it!
Answer: Technically what's happening is the G# over an A chord gives Amaj7, over the C#m is just a 5, within the chord, and over the D makes a b5, as in blues. So yes it'll do the job. As Matthew says, if it sounds good, it usually is.Ears are so good at this skill. I wouldn't just hold that note while the 3 chords are being played consecutively, although it will work. Probably better to reference G# over each in turn - if the chords are, indeed, in sequence.
One can make ANY note fit ANYWHERE with a bit of skill and ingenuity, so give others a try. Your ears and those of the audience will be the best judges.
Comment: @11684 No worries.  I saw the first use of temperature, and was thinking "ooo, cool blues, hot jazz, etc.", but then it kept appearing. :)
Comment: What is a 'b5'? A lowered fifth? In case that's what you meant, a G# in D Major is actually a #4 (following the assumed notation - a fourth with a sharp). If that's not what you meant, could you explain that? Otherwise, +1.
Comment: A b5  just happens to be the same note as a #4.So technically the G# would be a #4, BUT it sounds like a b5 as in Ab.I'm guessing (how else?) that the OP is a guitarist, and tends to think # rather than b, but at the end of the day, the effect is still the same, a bluesy feel to that note in that situation. So, #4 or b5, same end product:sounds good.
Comment: It depends on the temperature. But even if you play in equal temperature the function is still different (and if you are playing an instrument on which you can intonate - like the guitar - you may choose to adjust, because equal temperature is slightly off pitch.)
Comment: @11684 I think you mean temperament?
Comment: Thank you, yes. Talking about musical theory in a different language than the one you learnt it in is quite tricky. @JoshuaTaylor
Comment: Temperament is a bit different from temperature, but I'm not too hot on that...
Comment: Could you, honestly, spot the difference between an Ab and a G# in a piece that you listen to?
Answer: I really love melodies that incorporate that pitch interval (7 half steps, tritone, diminished 5th, whatever you want to call it) as a leading tone, especially when used this way in a major key song. As Tim pointed out, it's probably best used with some movement as opposed to just holding it out over the chord structure. For example, a rising pattern like G# A E evokes a feeling of overcoming some obstacle. Depending on the mood of the rest of the song, it can be very powerful.
Answer: Don't let music theory dictate what would sound good and what doesn't. If every musician played by the rules we would not have Jazz, Blues, Rock etc. Everything would sound boring. Majority of listeners don't know what notes you are playing, they just want something that sounds good.

Question about soloing


Get first value from List of ViewBag in View

Question: Im trying to get first value of column article_name from ViewBag.articlelist.
Controller
ViewBag.articlelist = (from a in dbcontext.cms_article select a ).ToList();

View
I just try to do like this .
ViewBag.articlelist['article_name'][0];

Thank You!
Comment: I not recommand getting first value after `ToList`,it'll get all table data to memory first.
Answer: ViewBag is of type dynamic so you should be able to get the value like this:
var articleName = ViewBag.articlelist?[0]?.article_name;

where I'm using the null-propagation operator (?[] and ?.) to handle null values.
If you're iterating over the collection you will need to cast it to a list:
@foreach (var article in (List<cms_article>)ViewBag.articlelist)
{
    ...
}

Comment: Thank You!. This exactly what im looking for =D


Same Code, Different output, Python Data frame for record comparison

Question: Good morning, the goal of my code is to compare records in multiple dataframes. If the record ID is present in another dataframe, output the months that the record is present. As an example, if ID 1 is present in months march, april, output March April. However, when i run this code on a test sample it works perfectly. When i run it on the original dataset the desired output is all or nothing instead of individual months.
The code below works perfectly with the desired output as a test sample.
Code
#Import of Libraries
import pandas as pd
import numpy as np
import xlsxwriter
import nltk
from itertools import chain

data1 = {'aa',1,3,2,12,3,4,5,'bb',6,7,8,9,100,65,56,'cc',70}
data2 = {'aa',11,12,3,4,5,123,12,14,8,'cc',100,56}
data3 = {'aa',12,111,33,13,5,6,4,555,'bb',3333,65,634,7,8,8888,100}
data4 = {'aa',44,33,5,6,7,8,999,'bb',4,2,66,3,70,1,1,2}

df1 = pd.DataFrame(data1,columns=['RPN'])
df2 = pd.DataFrame(data2,columns=['RPN'])
df3 = pd.DataFrame(data3,columns=['RPN'])
df4 = pd.DataFrame(data4,columns=['RPN'])

df1 = df1.astype(str)
df2 = df2.astype(str)
df3 = df3.astype(str)
df4 = df4.astype(str)

#Creates list of Source and RPN to compare data
march = df4['RPN'].values.tolist()
april = df3['RPN'].values.tolist()
may = df2['RPN'].values.tolist()
june = df1['RPN'].values.tolist()

#turns list of each month into sets.
june = set(june)
may = set(may)
april = set(april)
march = set(march)

#creates list of every record in all months
setlist = [june,may,april,march]

#creats an interestion of all like values in the list of months
setall = set.intersection(*setlist)
setall

#Checks to see if current dataframe RPN and Source is in the previous audit report data
compare = []
for index,x in df1.iterrows():
    RPN = x['RPN']
    if RPN in setall:
        compare.append('All Months')
    elif RPN not in chain(setall, april, may) and RPN in march:
        compare.append('March')
    elif RPN not in chain(setall, march, may) and RPN in april:
        compare.append('April')
    elif RPN not in chain(setall, march, april) and RPN in may:
        compare.append('May')
    elif RPN not in chain(setall,march) and RPN in may and april:
        compare.append('April and May')
    elif RPN not in chain(setall,april) and RPN in may and march:
        compare.append('March and May')
    elif RPN not in chain(setall,may) and RPN in april and march:
        compare.append('March and April')
    else:
        compare.append('New Record')

df1['Aging'] = compare
df1

Correct Output
RPN Aging
0   1   March
1   2   March
2   3   March and May
3   4   All Months
4   5   All Months
5   bb  March and April
6   6   March and April
7   7   March and April
8   8   All Months
9   9   New Record
10  100 April and May
11  12  April and May
12  65  April
13  cc  May
14  70  March
15  aa  All Months
16  56  May

The problem I am having is when i introduce this exact same code and format to the original dataset there is an ALL or NOTHING result, instead of showing differenation between each record.
#Import of Libraries
import pandas as pd
import numpy as np
import xlsxwriter
import nltk
from itertools import chain

#Creates dataframes
#Current Month
bucket='sagemaker-bucket-826404949026/Provider Data/Audit Comparison'
data_key = 'AuditJune2019.xlsx'
data_location = 's3://{}/{}'.format(bucket, data_key)
df = pd.read_excel(data_location)
df.info()

#Previous Month
bucket2 ='sagemaker-bucket-826404949026/Provider Data/Audit Comparison'
data_key2 = 'AuditMay2019.xlsx'
data_location2 = 's3://{}/{}'.format(bucket2, data_key2)
dfprev2 = pd.read_excel(data_location2)
dfprev2.info()

#April Month
bucket3 ='sagemaker-bucket-826404949026/Provider Data/Audit Comparison'
data_key3 = 'AuditApril2019.xlsx'
data_location3 = 's3://{}/{}'.format(bucket3, data_key2)
dfprev3 = pd.read_excel(data_location3)
dfprev3.info()

#March Month
bucket4 ='sagemaker-bucket-826404949026/Provider Data/Audit Comparison'
data_key4 = 'AuditMarch2019.xlsx'
data_location4 = 's3://{}/{}'.format(bucket4, data_key2)
dfprev4 = pd.read_excel(data_location4)
dfprev4.info()

#Creates list of Source and RPN to compare data
dfprev4 = dfprev4.fillna('0')
dfprev3 = dfprev3.fillna('0')
dfprev2 = dfprev2.fillna('0')
df = df.fillna('0')

df = df.astype(str)
dfprev2 = dfprev2.astype(str)
dfprev3 = dfprev3.astype(str)
dfprev4 = dfprev4.astype(str)

dfprev4['RPN'] = dfprev4['RPN'] + dfprev4['SOURCE']
dfprev3['RPN'] = dfprev3['RPN'] + dfprev3['SOURCE']
dfprev2['RPN'] = dfprev2['RPN'] + dfprev2['SOURCE']
df['RPN'] = df['RPN'] + df['SOURCE']

#Creates list of Source and RPN to compare data
march = dfprev4['RPN'].values.tolist()
april = dfprev3['RPN'].values.tolist()
may = dfprev2['RPN'].values.tolist()
june = df['RPN'].values.tolist()

#turns list of each month into sets.
june = set(june)
may = set(may)
april = set(april)
march = set(march)

#creates list of every record in all months
setlist = [june,may,april,march]

#creats an interestion of all like values in the list of months
setall = set.intersection(*setlist)
setall

#creates a dataframe of just RPN
df1 = pd.DataFrame(df['RPN'],columns = ['RPN'])

#Checks to see if current dataframe RPN and Source is in the previous audit report data
compare = []
for index,x in df1.iterrows():
    RPN = x['RPN']
    if RPN in setall:
        compare.append('All Months')
    elif RPN not in chain(setall, april, may) and RPN in march:
        compare.append('March')
    elif RPN not in chain(setall, march, may) and RPN in april:
        compare.append('April')
    elif RPN not in chain(setall, march, april) and RPN in may:
        compare.append('May')
    elif RPN not in chain(setall,march) and RPN in may and april:
        compare.append('April and May')
    elif RPN not in chain(setall,april) and RPN in may and march:
        compare.append('March and May')
    elif RPN not in chain(setall,may) and RPN in april and march:
        compare.append('March and April')
    else:
        compare.append('New Record')

df1['Aging'] = compare
df1

INCORRECT OUPUT
RPN Aging
0   testPORTICO New Record
1   test123PORTICO  New Record
2   AG50001PORTICO  New Record
3   AG50001FACETS   New Record
4   0370001PORTICO  New Record
5   0370001FACETS   New Record
6   JY00001PORTICO  New Record
7   JY00001FACETS   New Record
8   JQ00001PORTICO  New Record
9   JQ00001FACETS   New Record
10  DH70001PORTICO  All Months
11  DH70001FACETS   All Months
12  8120001PORTICO  All Months
13  8120001FACETS   All Months
14  J760001PORTICO  All Months
15  J760001FACETS   All Months
16  MS200012PORTICO All Months
17  MS200012FACETS  All Months
18  MS200012FACETS  All Months
19  BZ400013PORTICO All Months
20  BZ400013FACETS  All Months

I am thinking its a problem with the import of the dataframe maybe? im not to sure please help!
Answer: Consider the following modification of your code. Four prints allow you to check the data visually (in a tabular form) so may help you I believe.
import pandas as pd

# dictionary of sets
data = {
    'June':
        {'aa', 1, 3, 2, 12, 3, 4, 5, 'bb', 6, 7, 8, 9, 100, 65, 56, 'cc', 70},
    'May':
        {'aa', 11, 12, 3, 4, 5, 123, 12, 14, 8, 'cc', 100, 56},
    'April':
        {'aa', 12, 111, 33, 13, 5, 6, 4, 555, 'bb', 3333, 65, 634, 7, 8, 8888, 100},
    'March':
        {'aa', 44, 33, 5, 6, 7, 8, 999, 'bb', 4, 2, 66, 3, 70, 1, 1, 2}}
# print(*(f'{key:>5}: {val}' for key, val in data.items()), sep='\n')

# dictionary with months only
months = {month: 0 for month in data}

# transform dictionary of sets into dictionary of dictionaries
new_data = dict()
for month in data:
    for value in data[month]:
        new_data.setdefault(str(value), months.copy())[month] = 1
# print(*(f'{key:>5}: {val}' for key, val in new_data.items()), sep='\n')

# create dataframe
df = pd.DataFrame.from_dict(new_data, orient='index')
df.index.name = 'RPN'
# print(df)

def compare(srs):
    if srs.sum() == len(months):
        return 'All months'
    else:
        return ', '.join(month
                         for month, check
                         in zip(srs.index, srs)
                         if check)

# add aging string
df['Aging'] = df.apply(compare, axis=1)
# print(df)

EDIT
Suppose your data is loaded from files with pandas in the form of dataframes then you may create data like this:
# kinda loaded data
a = pd.DataFrame({'RPN': ['aa', 1, 3, 2, 12, 3, 4, 5, 'bb', 6, 7, 8, 9, 100, 65, 56, 'cc', 70]})
b = pd.DataFrame({'RPN': ['aa', 11, 12, 3, 4, 5, 123, 12, 14, 8, 'cc', 100, 56]})
c = pd.DataFrame({'RPN': ['aa', 12, 111, 33, 13, 5, 6, 4, 555, 'bb', 3333, 65, 634, 7, 8, 8888, 100]})
d = pd.DataFrame({'RPN': ['aa', 44, 33, 5, 6, 7, 8, 999, 'bb', 4, 2, 66, 3, 70, 1, 1, 2]})

# dictionary of sets
data = {'June': a['RPN'], 'May': b['RPN'],
        'April': c['RPN'], 'March': d['RPN']}

Comment: thanks so much for your response! this code worls! however the code you supplied is for the test code, not the main data set. the main data set is not in a dictionary of sets format. i am going buy a column 'RPN' is this code still possible with converting a dataframe column to a dictionary of sets? if i am saying this correctly? @Poolka
Comment: @JamesScott I don't know what your data is. I guess you may create dictionary of sets/lists/arrays/pandas_series pretty easily like `{'march': set(dfprev4['RPN'].values.tolist()), ...}` / `{'march': dfprev4['RPN'].values.tolist(), ...}` / `{'march': dfprev4['RPN'].values, ...}` / `{'march': dfprev4['RPN'], ...}`. All ways should work.
Comment: @JamesScott Check the EDIT addition.
Comment: Thanks!! your code works perfectly and everything runs smoothly. If you want to know. the reason my code was different, when it was the exact same code, was the import of the data from an AWS S3 Bucket to bringing in the data on the juypter notebook itself. or some reason i dont know why this caused a difference in the runnning of the code and output but it did.


how can we do reverse iteration of last four items in the XmlNodeList

Question: At this point It is displaying items in reverse order with no limit. XmlNodeList can have many items. I would like to display only last four items. How can I find or display last four items in the list? Anyone?
XmlNodeList MyTestList = MyRssDocument.SelectNodes("test/holder/item");

string Title = "";
string Link = "";

for (int i = MyTestList.Count - 1; i >= 0; i--)
{
    XmlNode MyTestDetail;

    MyTestDetail = MyTestList.Item(i).SelectSingleNode("title");
    if (MyTestDetail != null)
        Title = MyTestDetail.InnerText;
    else
        Title = "";

    MyTestDetail = MyTestList.Item(i).SelectSingleNode("link");
    if (MyTestDetail != null)
        Link = MyTestDetail.InnerText;
    else
        Link = "";
}

Answer: You can just replace the following line:
for (int i = MyTestList.Count - 1; i >= 0; i--)

By this:
for (int i = MyTestList.Count - 1; i >= (MyTestList.Count - 4) ; i--)

Answer: for (int i = MyTestList.Count; i >= 4; i--)
{
    XmlNode MyTestDetail;

    MyTestDetail = MyTestList.Item(i).SelectSingleNode("title");
    if (MyTestDetail != null)
        Title = MyTestDetail.InnerText;
    else
        Title = "";

    MyTestDetail = MyTestList.Item(i).SelectSingleNode("link");
    if (MyTestDetail != null)
        Link = MyTestDetail.InnerText;
    else
        Link = "";
}


What are more examples of functions whose input and output have a constant product?

Question: I was recently thinking about functions and asked myself what would happen if a function had the property that $a\cdot f(a)$ = $b\cdot f(b)$ for any real numbers $a$ and $b$. My initial train of thought led me to conclude that the only function that satisfies that is $f(x)=0$. My reasoning was that since $a\cdot f(a)$ = $b\cdot f(b)$ no matter what values you use for $a$ and $b$, that the input and output of the function must have a constant product, meaning $x\cdot f(x)=c$ for any real number x and some constant c. Then you plug in $x=0$ to get $0\cdot f(0)=c\Rightarrow 0=c$ so that $x\cdot f(x)=0$. But the only way that equation is true for all x is if $f(x)=0$.
But then I accidentally found a counterexample: $f(x)=\frac1x$
$$a\cdot f(a)=b\cdot f(b)$$
$$a\cdot \frac1a=b\cdot \frac1b$$
$$1=1$$
In my argument, I plugged in $x=0$ so that $0\cdot f(0)=0$ since any zero times any real number is zero, but I guess that assumes that f(x) is even defined for $x=0$. This leaves me with two questions. What are some more examples of functions that fit this property besides $f(x)=0$? And are there any of those which are defined for $x=0$?
Answer: For any pair of real values $A,B$, the function
$$f_{A,B}(x)=\begin{cases}\frac{A}{x}&x\neq 0\\ B&x=0\end{cases}$$ satisfies your condition.
But not only that, you can also show that those are all the functions that satisfy your condition. Take any function $f$ that has the property, and take any $x\in\mathbb R\setminus\{0\}$.
Then, $1\cdot f(1)=x\cdot f(x)$, meaning that $$f(x)=\frac{1\cdot f(1)}{x} = \frac{A}{x}$$ for $A=f(1)$. So, this arbitrary function that satisfies your condition is in fact the function $f_{A,B}$ for $A=f(1)$ and $B=f(0)$.


Running Drupal's drush cron jobs with a newer PHP version from Software Collections

Question: How do you run Drupal's drush cron jobs when your server has a newer PHP version installed from Software Collections?
The server is running Scientific Linux 7, which comes with PHP 5.4, but we need, and have installed, PHP 7.1 using the Software Collections rh-php71 package. Apache and therefore Drupal work fine with PHP 7.1, but we also need to add the drush cron jobs to our crontab, and ensure that drush is using the newer version of PHP when it does so. We have tested to see if drush (we have drush 8.1.10 installed) will run with the default PHP 5.4, just in case, but the cron jobs understandably crash with errors.
We have tried to test the cron command (in the way that you would normally attempt to run a command within a scl wrapper), by running the following as root:
scl enable rh-php71 drush cron -r /data/webs/www/siteroot

but this will not run, returning the error:
Unable to open /etc/scl/conf/drush!
Further testing (in a terminal) shows that we can however successfully run the drush command within a subshell which is within a scl wrapper:
scl enable rh-php71 bash
drush cron -r /data/webs/www/siteroot
exit

Is there some way that we can run the necessary drush commands just as a 'one-liner' in the crontab, or do I have no option but to have to create a separate shell script containing the above commands, and call that as the cron job command instead?
(I have also asked this question on Unix.SE, as it seems more of a Linux question than a Drupal one as such, but have had no responses.)
Comment: First you need to explain what `scl enable rh-php71`does? Does it set some environmental variables or ... You can tell Drush to use a specific version of PHP with DRUSH_PHP - see https://drupal.stackexchange.com/a/242258/24060
Comment: Thanks, that's useful to know about DRUSH_PHP. I don't know if that would work in this case as I'm afraid I don't know exactly what the `scl` wrapper actually does (but I think it involves an element of heavy lifting as regards to reconfiguring the environment when invoked)?
Comment: You might not need all the changes introduced by `scl enable rh-php71`. Running `env` or just `echo $PATH` after `scl enable rh-php71 bash` should tell you enough. Anyway, the problem could be related to the user running scl. Cron is often using root. Does your scl wrapper work if you run it as root. And so on until you understand what the problem is :-)
Answer: After reading a more technical section of the Software Collections guide it turns out that when trying to run a command which has arguments within a scl wrapper, you need to enclose the command in single quotes or separate it from the wrapper part of the line with two minus signs:

If you are running a command with multiple arguments, remember to enclose the command and its arguments in quotes:

scl action software_collection_1 software_collection_2 'command --argument'

Alternatively, use a -- command separator to run a command with multiple arguments:

scl action software_collection_1 software_collection_2 -- command --argument
My drush command runs successfully now.


Dynamic DNS with split horizon DNS or hairpin NAT

Question: I would like to host a CalDAV and CardDAV server for devices that will roam between being on the LAN to outside my network (such as my iPhone and laptops). A static IP is prohibitively expensive.
I have a my own domain, a Mac Mini with Lion Server, a SonicWall TZ 200 and a dyn.com dynamic address (the SonicWall keeps the dynamic account updated). I would normally set up a static IP and use split horizon DNS so those roaming devices can point to server.mydomain.com and function on and off my LAN.
Is a similar setup possible with the dynamic address? I am happy to set up the clients with either server.mydomain.com or myaccount.dyndns.org but I would prefer to use my domain.
Would either hairpin NAT on the SonicWall or split horizon DNS using the Lion Server allow this to work?
Answer: The reason split horizon DNS is set up is almost always because an internal service cannot be reached off of the LAN. For example, Active Directory domain controllers that also hold internal DNS. If your all of your services are intended to be externally reachable then split DNS isn't necessary, strictly speaking. Discard any use of split DNS in your design until you find a sure reason to use it.
Hairpin NAT / tromboning would be how anyone behind your firewall would reach your server, which is acceptable. The biggest downside to hairpinning is that it "unnecessarily" engages your SPI engine and anything else running in the firewall like UTM services. However, if that's how you intentionally designed it, then it's hard to call it "unnecessary" - the TZ 200 can take a decent amount of traffic and I highly doubt a CalDAV/CardDAV service will be getting thousands of requests per second from your internal network. =)
Comment: I've set up hairpin NAT like this  http://www.fuzeqna.com/sonicwallkb/consumer/kbdetail.asp?kbid=8612&formaction=faqalert which works for the standard port, but I can't connect on the SSL port 8443. If I enter a local IP and select SSL I can connect so the problem must be in the server name and/or cert which is self signed, and not as dyndns.org of course.


Get Item from Array with Property equal to Value

Question: This questions is for my LightSwitch project but I'm not sure that's renavent.  I've never done javascript or jquery before and i'm pretty sure this will have nothing to do with LightSwitch and everything to do with the latter.
The following code works perfectly setting test equal to 4 in my case:
myapp.BrowseTreeNodes.TreeNodes_render = function (element, contentItem) {
    var screen = contentItem.screen;
    var result = screen.MyArray.data[0];
    var test = result.Id;
}

What I need to do is instead of setting result to the first item in the array I need to set result to the item with a specific Id, for this example let's say 4.
Here's what i've tried:
var result = $.grep(screen.MyArray.data, function (e) { return e.Id === 4; })[0];
var result = screen.MyArray.data.filter(function (v) { return v.Id === 4; })[0];
var result = screen.MyArray.data.find(x => x.Id === 4)[0];

Thank you in advance.
Comment: your problem may be the `===` , `===` checks for type as well, so `"4" === 4` is `false`  while `"4" == 4` is `true`, check if your id property is stored as a string or an integer...
Comment: OMG thank you, I was tearing my hair out over this.  The $.grep line worked great when I changed it to == 4.  I'll mark this as the solution if you submit it as an answer rather than a comment.
Comment: great!, glad i could help, i posted the comment as an answer
Answer: your problem may be the === , === checks for type as well, so "4" === 4 is false  while "4" == 4 is true, check if your id property is stored as a string or an integer...


Programmatically open PrefPane with section

Question: I need to open the network-prefpane in my App. This works fine with
<code>[[NSWorkspace sharedWorkspace] openFile:@"/Path/To/PrefPane"];
</code>
But how can i directly open the "proxy-settings", which is in the "network-pref" under "advanced"?
You can see this in the Safari-Settings under "Advanced->Proxies"
Thanks
Answer: The easiest and cleanest way I've found is to use Applescript.
<code>NSAppleScript *a = [[NSAppleScript alloc] initWithSource:@"tell application \"System Preferences\"\nactivate\nset current pane to pane \"com.apple.preference.universalaccess\"\nend tell"];
[a executeAndReturnError:nil];
[a release];
</code>
Replace com.apple.preference.universalaccess with the name of the pane. Here is a full list of preference pane names.
Comment: This could be usable for me. Thanks!
Answer: While it's doable with UI scripting, it also requires access to assistive devices (System Preferences->Universal Access). That doesn't feel right because if a user prefers the access disabled, that just won't work. If you need to modify proxy settings you could use SystemConfiguration.framework directly, going through the necessary authentication.
Comment: I don't want to modify it. Just open it that the user can modify it himself.
Thanks for the info with the option in "universal access"


Number of ordered set partitions

Question: I have encountered a counting problem which involves counting all ordered partitions.
I give an example: Given a set of two elements $\{A,B\}$, there are two partitions, $\{\{A,B\}\}$ and $\{\{A\},\{B\}\}$, but there are three ordered partitions, $(\{A,B\})$, $(\{A\},\{B\})$, and $(\{B\},\{A\})$.
I have to count them for a set of order $2^N$ with $N$ growing.
I am aware of the Stirling number $k\brace r$ of the second kind, which describes the number of $k$-partitions of a set with cardinality $r$.
So I get some thing like this
$$ \sum_{k=0}^{2^n} k!{{2^n}\brace k} $$
Are there closed formulas for this problem? Is this problem wellstudied in combinatorics? Are asymptotic formulas known?
I am happy for any advice or reference I can get.
Comment: I don't know if this helps but I had a think about this last night... If you take the set of all elements of all partitions of the cardinality of your set, the closed form is the sum of the factorials of all those elements.  So suppose you begin with a set of cardinality 3: 3 can be partitioned into {1,1,1}, {1,2}, {2,1}. So your result is $5\times1!+2\times2!$. That looks trivially equivalent to Brian's answer below.
Comment: Related: https://math.stackexchange.com/questions/625703/how-many-linear-order-types-are-there-on-a-set-of-n-elements
Answer: These are the ordered Bell numbers (or Fubini numbers)
$$a(n)=\sum_{k=0}^nk!{n\brace k}\;;$$
asymptotically they satisfy
$$a(n)\approx\frac{n!}{2(\ln 2)^{n+1}}\;.$$
The sequence of these numbers is OEIS A000670, where you'll find many references and much information; there don't seem to be any nice closed forms. You're interested specifically in $a(n)$ when $n$ is a power of $2$; it does not appear to me that this helps.
Comment: Isn't it weird that that number is smaller than $n!$, but if we restrict ourselves to the case of partitions of type (1, 1, .., 1) we end up with $n!$ ordered partitions?

Number of ordered set partitions


Como um site é publicado e hospedado na internet?

Question: A minha duvida é como um site feito com PHP e MySQL é publicado na internet? E como se hospeda nele num servidor online?
Comment: Você não tem as configurações da conexão do mysql nos seus arquivos php? localhost etc... Basta mudar para o servidor de hospedagem, você não tem os arquivos organizados nas suas pastas do localhost? Basta colocados na pasta do servidor de hospedagem...
Comment: Relacionada: https://pt.stackoverflow.com/q/167437/18246
Comment: Entendo, e um site como um portal de noticias, como ele é atualizado em cada noticia no servidor de hospedagem? Seria outra pergunta isso?
Answer: Um servidor Web é a grosso modo uma máquina com um sistema operacional, um software de servidor web(no caso do php o Apache ou outro), um software de servidor de banco de dados e um software de aplicação, o site, com seu front-end(html,js) e back-end(php).
Um ip fixo é atribuído a essa máquina, um servidor de DNS fica responsável por traduzir o domínio(www.exemplo.com.br) para o ip correspondente (22.214.171.124).
A partir do momento que o ip é atribuído ao servidor que por sua vez tem uma porta configurada para ficar full time aguardando requisições, o site está no ar sendo acessado pelo ip (http://22.214.171.124) ou pelo domínio configurado (http://www.exemplo.com).
Quando uma requisição é realizada ex: http://www.exemplo.com/uma_página.php,
o servidor vai procurar em sua pasta raiz o arquivo uma_página.php, que por sua vez exibirá o resultado de seu processamento como resposta para a requisição.
Sua pergunta é ampla, então não é de costume haver respostas genéricas como essa, que tem como principal objetivo incentivá-lo a pesquisar as minúcias de como isso funciona aqui no site, existem respostas muito boas.


jquery doing two ajax calls under on function

Question: if have the following two scripts, when a button is clicked i wish for them both to be loaded, at the moment they are conflicting? any suggestions? the first needs to run before the second. I have tried calling two separate function but still i get a conflict
function showUser3(str)
{

if (str=="")
  {
  document.getElementById("basketShow").innerHTML="";
  return;
  }
if (window.XMLHttpRequest)
  {// code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }
else
  {// code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
xmlhttp.onreadystatechange=function()
  {
  if (xmlhttp.readyState==4 && xmlhttp.status==200)
    {
    document.getElementById("basketShow").innerHTML=xmlhttp.responseText;
    }
  }

        var Id = str;

        var qty = $("#"+Id).find("#qty").val();
        var productID = $("#"+Id).find("#productID").val();
        var categoryID = $("#"+Id).find("#categoryID").val();
        var priceID = $("#"+Id).find("#priceID").val();

        var url = 'ajaxAddBasket.php?productID='+productID+'&categoryID='+categoryID+'&qty='+qty+'&priceID='+priceID+'&Id='+Id;

xmlhttp.open("GET",url,true);
xmlhttp.send();

if (str=="")
  {
  document.getElementById("ajaxPallet").innerHTML="";
  return;
  }
if (window.XMLHttpRequest)
  {// code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }
else
  {// code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
xmlhttp.onreadystatechange=function()
  {
  if (xmlhttp.readyState==4 && xmlhttp.status==200)
    {
    document.getElementById("ajaxPallet").innerHTML=xmlhttp.responseText;
    }
  }

var url = 'ajaxPallet.php';
xmlhttp.open("GET",url,true);
xmlhttp.send();

}

Comment: Why are you not using jQuery's `ajax` function if you are already using jQuery?
Answer: If you convert your Ajax calls to jQuery's ajax [docs] method, you can make use of deferred objects [docs] and pipe [docs] to chain the calls:
$.ajax({...}).pipe(function() {
   return $.ajax({...});
});

Have a look at the documentation for more examples.
Answer: I would just us the basic $.ajax function of jquery. And on the complete/succes event of the first call the second one with $.ajax?
Check this link out: http://api.jquery.com/jQuery.ajax/


How to remove the last element of an array in bash?

Question: The syntax to delete an element from an array can be found here: Remove an element from a Bash array
Also, here is how to find the last element of an array: https://unix.stackexchange.com/questions/198787/is-there-a-way-of-reading-the-last-element-of-an-array-with-bash
But how can I mix them (if possible) together to remove the last element of the array ?
I tried this:
TABLE_COLUMNS=${TABLE_COLUMNS[@]/${TABLE_COLUMNS[-1]}}

But it throws:

bad array subscript
Comment: Does this answer your question? [How do I subtract 1 from an array item in BASH?](https://stackoverflow.com/questions/46745252/how-do-i-subtract-1-from-an-array-item-in-bash)
Comment: @bobdylan That question is about subtracting 1 from one element, this question is about removing an element.
Comment: https://stackoverflow.com/questions/7174354/bash-array-indexing-minus-the-last-array - wouldn't let me flag more than one question
Comment: Notice that your code works in Bash 4.3 or newer, but it has the (big) flaw of not removing complete elements, but also substring matches.
Answer: You can use unset to remove a specific element of an array given its position.
$ foo=(1 2 3 4 5)
$ printf "%s\n" "${foo[@]}"
1
2
3
4
5
$ unset 'foo[-1]'
$ printf "%s\n" "${foo[@]}"
1
2
3
4

Answer: Edit: This is useful for printing elements except the last without altering the array. See chepner's answer for a far more convenient solution to OP.

Substring expansions* could be used on arrays for extracting subarrays, like:
TABLE_COLUMNS=("${TABLE_COLUMNS[@]::${#TABLE_COLUMNS[@]}-1}")

* The syntax is:
${parameter:offset:length}

Both offset and length are arithmetic expressions, an empty offset implies 1. Used on array expansions (i.e. when parameter is an array name subscripted with * or @), the result is at most length elements starting from offset.


Rails: belongs_to/has_many with non-id foreign/primary keys

Question: I'm working with a legacy database where a story belongs_to a user and a user has_many stories.  The problem is that it's all uploaded data, so they can't connect with the id field like Rails usually does.
Here's the structure for the Story table:
  create_table "stories", force: :cascade do |t|
    t.string "title"
    t.integer "authorId"
    ...
  end

And here's the structure for the User table:
create_table "users", force: :cascade do |t|
    ...
    t.integer "uid"
    ...
end

I need to somehow join them so that the story's authorId connects to matching values for a user's uid.
I've looked at questions like this and this, as well as the documentation, but I can't find model code that works.
Right now what I have is, for story.rb:
class Story < ApplicationRecord
    belongs_to :user, class_name: 'User', foreign_key: 'authorId'
end

And for user.rb (I'm using Devise.):
class User < ApplicationRecord
  # Include default devise modules. Others available are:
  # :confirmable, :lockable, :timeoutable, and :omniauthable
  devise :database_authenticatable, :registerable, :trackable,
         :recoverable, :rememberable, :validatable

  has_many :stories, class_name: "Story", foreign_key: 'uid'
end

Can anyone see how to get these models properly linked up in a belongs_to/has_many?
Answer: I believe by your post that you cannot update the AuthorID to user_id, and change uid to id. After that, you would have a clean relationship.
But, try to define the primary_key and foreign_key and customize your association:
class Story < ApplicationRecord
 belongs_to :user, primary_key: "uid", foreign_key: "authorId"
end

class User < ApplicationRecord
  has_many :stories, primary_key: "authorId"
end

or this:
class Story < ApplicationRecord
 belongs_to :user, primary_key: "uid"
end

class User < ApplicationRecord
  has_many :stories, primary_key: "authorId", foreign_key: "uid"
end

I think this maybe works.
Comment: The first customization worked perfectly!  Thank you!


IBM WAS 7.5 SSLException SSLSocketFactory is null

Question: Hi we have a problem with our WAS 7.5 production environment, we are getting the following error with an application that had implemented google recaptcha:
javax.net.ssl.SSLException: SSLSocketFactory is null. This can occur if javax.net.ssl.SSLSocketFactory.getDefault() is called to create a socket and javax.net.ssl.* properties are not set.
We have the following code:
<code>System.setProperty("javax.net.ssl.keyStoreType","jks");
System.setProperty("javax.net.ssl.trustStoreType","jks");
System.setProperty("javax.net.ssl.keyStore","/jks/ClientKeyStore.jks");
System.setProperty("javax.net.ssl.trustStore","/jks/ClientTrustStore.jks");
System.setProperty("javax.net.debug","ssl");
System.setProperty("javax.net.ssl.keyStorePassword","password$");
System.setProperty("javax.net.ssl.trustStorePassword","password$");

SSLSocketFactory sslsocketfactory = (SSLSocketFactory)
SSLSocketFactory.getDefault();
</code>
In our certification environment everything is working fine, anyone had a similar issue ? thanks in advance.
Comment: How are you creating the `SSLSocketFactory`?
Comment: Hello yes we have the following code:
SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();

URL obj = new URL(url);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
con.setSSLSocketFactory(sslsocketfactory);
Comment: I'd encourage you to update the question with that information in addition to how the related `javax.net.ssl.*` properties are set. Without that information there is really not that much anybody can do to help
Answer: The problem was resolved editing the java.security file of JRE, changing the following lines:
<code># Default JSSE socket factories
ssl.SocketFactory.provider=com.ibm.jsse2.SSLSocketFactoryImpl
ssl.ServerSocketFactory.provider=com.ibm.jsse2.SSLServerSocketFactoryImpl
# WebSphere socket factories (in cryptosf.jar)
#ssl.SocketFactory.provider=com.ibm.websphere.ssl.protocol.SSLSocketFactory
</code>
#ssl.ServerSocketFactory.provider=com.ibm.websphere.ssl.protocol.SSLServerSocketFactory
After the app severs were restarted the recaptcha code works.


Inner com 3 tabelas PHP/SQL

Question: Estou com 3 tabelas e quero comparar o valor delas, eu consigo fazer com 2 usando o JOIN mas quando coloco JOIN na terceira o php não dá certo.
Código em PHP:
$agora = "select * from usuario u
          join dia_usuario du
          on u.usuario_id = du.id_diarista
          join diarista d
          on d.id = ud.id_diarista";
$sim = mys[![inserir a descrição da imagem aqui][1]][1]qli_query($conn,$agora);
while ($row_usuario2 = mysqli_fetch_assoc($sim)) {
    $ola2 = $row_usuario2['nome'];
}

Tabela usuario

Tabela dia_usuario

Tabela diarista

Retorno da consulta
Comment: qual erro o php mostra?
Comment: @Enerson Silva coloca a resposta do Anderson como correta pf.
Comment: Simples, seu problema está na segunda tabela, você só tem o id_diarista 3 e 5 então ele só trás o usuário emerson que tem o usuario_id igual a 3
Answer: Seu erro está no segundo igual, você colocou "ud" e não "du"
$agora = "select * from usuario u
          inner join dia_usuario du
          on u.usuario_id = du.id_diarista
          inner join diarista d
          on d.id = du.id_diarista";
$sim = mysqli_query($conn,$agora);
while ($row_usuario2 = mysqli_fetch_assoc($sim)) {
    $ola2 = $row_usuario2['nome'];
}

=========
Você só encontra o usuário enerson por causa do seguinte...
Você está pegando todos os dados de usuário de diferentes tabelas onde o id do usuário seja igual ao id diarista e o id da tabela diarista(Não o id_diarista) seja igual ao o id_diarista...
Sendo assim o usuário enerson tem o id 3 que é igual ao id_diarista 3 que é igual ao id da tabela diarista que também é 3... O outro valor é 5 e não há nenhum usuário com o id 5 no momento, Deu para sacar?
Comment: Por favor evitem longas discussões nos comentários; a conversa de vocês foi [movida para o chat](https://chat.stackexchange.com/rooms/82805/discussion-on-answer-by-anderson-henrique-inner-com-3-tabelas-php-sql)
Answer: O que está acontecendo é o fato de você estar restringido sua busca pelo conteúdo das tabelas relacionadas.
O que isso significa?
O seu join restringe os retornos do conjunto final filtrando os resultados que estão nos conjuntos relacionados ao conjunto principal.
Se você quer trazer os dados que estão presentes na tabela principal e fazer a coalescência do que não existe nas tabelas B e C da sua busca, você deve usar o left join.
no caso, sua query ficaria parecida com isso:
select
*
from usuario u
left join dia_usuario du
  on u.usuario_id = du.id
left join diarista d
  on d.id = du.id;

Não quero parecer grosseiro nem pretensioso, mas vejo que você parece estar iniciando nos estudos sobre banco de dados, pude perceber que sua modelagem ainda não reflete bem a realidade do que você quer com sua aplicação.
Sugiro estudar um pouco mais como funcionam os joins nas consultas, para isso segue abaixo um artigo pequeno porém interessante:
http://dan-scientia.blogspot.com/2012/10/teoria-dos-conjuntos-em-juncoes-sql.html
Boa sorte aí na sua aplicação, e espero que minha contribuição seja útil.
Comment: me ajudou, não foi grosseiro sou iniciante em banco de dados.
Comment: Show de bola Enerson Silva. Bons estudos!

Inner com 3 tabelas PHP/SQL


Algorithmic time complexity of Newton's method vs bisection method

Question: In the context of root finding, it is often stated that the bisection method is slower than Newton's method due to linear convergence. However, I am trying to understand why this is the case from an algorithmic time complexity viewpoint.
My understanding is that the time complexity of Newton's method is $O(log(N)F(N))$ where $F(N)$ is the cost of calculating $\frac{f(x)}{f'(x)}$ with $N$-digit precision. But given the architecture of the bisection method, which halves the search interval at each iteration, I was under the impression that its time complexity was also logarithmic. I was therefore wondering whether anyone could shed some light on why the bisection method is slower than Newton's method from a complexity point of view? Any comments or references to literature would be greatly appreciated.
Comment: "I was under the impression that its time complexity was also logarithmic": just an impression, it is linear in the number of digits (exactly one bit per function evaluation).
Answer: Per every bit you need one bisection step, so the cost is $\sim N·eval(f)$, while Newton is, as you wrote, $\sim 3·\log(N)·eval(f)$, where it is used that $eval(f,f')=3·eval(f)$ if automatic/algorithmic differentiation is used.


Como passar os dados de um checkbox checked para um modal?

Question: Gostaria de saber como faço para, ao selecionar um checkbox, pegar esses dados e lança-los num modal. No código abaixo, quero pegar apenas os charters selecionados de um post e fazer aparecer apenas esses selecionados no modal. Como faço? Já tentei ver em outros tutoriais, mas não consegui achar algo que me ajudasse. Muito obrigado!

    <input type="checkbox" ng-model="charter.selected">   {{'{{charter}}'}}

          <button type="button" class="close" data-dismiss="modal" aria-label="Close">
              ×
          </button>

Edit Post Components

     <form>

              Component:
              <input type="text" class="form-control" ng-model="post.components" id="txtComponent" autocomplete="off" placeholder="Insert a component">
              <button ng-click="searchCharters(post.components)" type="button" class="btn btn-primary">See charters</button>

     </form>

             {{'{{charter}}'}}

    <button type="button" class="btn btn-default" data-dismiss="modal">Close</button>
    <button ng-click="updatePost()" type="button" class="btn btn-primary">Update</button>

Comment: poste um exemplo minimo, por favor, o código angular? deve ser algo simples de fazer, mas, para gerar um exemplo, precisamos do seu, se não sai um exemplo genérico ...
Comment: Sou novo em angular, cara. Realmente não tenho ideia de como seria.
Answer:

function setaDadosModal(nome) {

  document.getElementById('nome_confirma').innerHTML = nome;

}

<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet" />

  <button  type="button" class="btn btn-block btn-danger" onclick="setaDadosModal('id do campo ou palavha')">Enviar para modal</button>

        <button type="button" class="close" data-dismiss="modal">×</button>

modal headero

          você enviou o campo ?

        <input style="display: none" type="text" name="id_inativar" id="id_inativar">

        <button type="button" class="btn btn-default" data-dismiss="modal">Cancelar</button>

function setaDadosModal(nome,id) {

    document.getElementById('nome_confirma').innerHTML = nome;
    document.getElementById('id_inativar').value = id;
}
Comment: agradeço a dica. Mas não teria como me mostrar um exemplo usando o checkbox mesmo? Sou novo em programação Web e Angular.


Keybinding for Format Code -> Shift+Alt+F in VS Code does not seem to work

Question: On this link the following keybinding is described:

What I need is the ability to load an XML and have it properly formatted, as you would with Ctrl+K +D in visual studio.
I had prepared this xml content:
<root><level1><sublevel1A></sublevel1A><sublevel1B></sublevel1B></level1><level2><sublevel2A></sublevel2A></level2></root>

Which would need to be formatted to this:
<root>
    <level1>
        <sublevel1A>
        </sublevel1A>
        <sublevel1B>
        </sublevel1B>
    </level1>
    <level2>
        <sublevel2A>
        </sublevel2A>
    </level2>
</root>

Nothing happens though if I use the key binding. Nor does it help if I select everything first.
Any ideas?
Answer: There is no built-in support for XML formatting in VSCode. To get XML formatting you need install this extension - https://marketplace.visualstudio.com/items?itemName=fabianlauer.vs-code-xml-format

Also, you have a typo in your example XML, </subleve1B> instead of </sublevel1B>
Comment: It looks like this could work, but I get a timeout, possibly my firewall blocks the request?
Answer: The XML extension from Red Hat provides, among many other features, formatting support for XML document:
It requires a JDK >= 8 to run.


Expression for complex type property

Question: I have a scenario where I dynamically create a WHERE clause from a client-side grid's configuration. The client sends some json to the server which I then parse and subsequently convert into an expression so it can be passed into the repository as a where clause.
For the moment I'm struggling with creating expressions for complex property types, like this one:
public partial class Resource
{
  public string DisplayName { get; set; }
  public virtual ResourceType ResourceType { get; set; }
}

The code below that does the translation to an expression works well for simple types like the Displayname property. The expression will be something like:
x => x.DisplayName == "ValueEnteredByUserInTheUI"

However, when a value is entered in the grid for the ResourceType property, the expression will ultimately be something like:
x => x.ResourceType == "ValueEnteredByUserInTheUI"

I'm missing one step to convert into this:
 x => x.ResourceType.Name == "ValueEnteredByuserInTheUI"

Anyone can point me in the right direction here?
public Expression<Func<T, bool>> GetExpression<TEntity>(string field, string operation, object value, string ignoreCase)
{
 Expression<Func<T, bool>> whereClause = default(Expression<Func<T, bool>>);

// Define lambda
ParameterExpression param = Expression.Parameter(typeof(T), "x");
 MemberExpression member = Expression.Property(param, field);

 // Get property type
  Type propertyType = member.Type;

  // Get converter for type
   TypeConverter converter = TypeDescriptor.GetConverter(propertyType);

   // Convert property to type
   var result = converter.ConvertFrom(value.ToString());

   // Convert value to constant value
   ConstantExpression constant = Expression.Constant(result);

    Expression comparingExpression = default(BinaryExpression);
    switch (operation)
    {
      case "like":
           comparingExpression = Expression.Equal(member, Expression.Convert(constant, member.Type));
           break;
       default:
           break;
   }

  var lambda = Expression.Lambda<Func<T, bool>>(comparingExpression, param);
   whereClause = whereClause == default(Expression<Func<T, bool>>) ? lambda : NewPredicateBuilder.And(whereClause, lambda);
        return whereClause;
  }

Comment: You're right, wrong choice of words here. I changed the question.
Answer: Apparently not a lot of people are keen on expressions. Anyway, I have created a workaround for this issue. I added another parameter in the method indicating the property name of the complex property of the root object.
 // Get property of root object
 MemberExpression member = Expression.Property(param, field);

 // Get property of property
 MemberExpression memberField = Expression.PropertyOrField(member, complexProperty);

It's not very scalable nor generic but this will do it for now.
Comment: What do you pass as the complexProperty? Basically could you provide both in your answer? Have you found an aswwer if you wanted to go deeper? Thanks.
Comment: It would be a dot-separated string of your class' graph navigation such as "ResourceType.Name" where "ResourceType" is a class with a "Name" property. Not sure what you mean by going deeper. Not too long ago,  I uploaded a little project on GitHub: https://github.com/dimenics/dime-expressions. Take a look and see if this works for you.
Comment: Hello hbulens. Thanks for the response.I will definetely take a look to your link. Your answer works greatly and i managed to solve my issue. Just to clarify. What I mean by going deeper...Imagine having a class like your resource class. 'Resource.ResourceType.ResourceCode.Name', meaning Resource code was another object and a property to your ResourceType. How would you go accessing the ResourceCode.Name property?

Expression for complex type property


question about inner product space

Question:
$\mathbb{F}$ is $\mathbb{C}$ or $\mathbb{R}$.
given $u\in\mathbb{F}^n$ such as $\langle v,u \rangle=0$ for each $v\in\mathbb{F}^n$.
prove that $u=\underline{0}$.

by contradiction, assume that $u \neq \underline{0}$. now, because $v$ and $u$ are both in $\mathbb{F}^n$ I can choose $v=u$ so: $\langle v,u \rangle = \langle u,u \rangle = 0$.
and from positivity this means that $u=\underline{0}$.
Is this correct?
I'm not sure because I learn this as part of diagonalization and don't see how this is connected to the topic.
Thanks in advance.
Comment: Yes, your argument is true.
Comment: @AmitaiYuval: thank you.
Comment: See also http://en.wikipedia.org/wiki/Inner_product_space.
Comment: General comment on style - the contradiction is not needed. The exact same argument follows without the contradiction.
Answer: No contradiction is needed:
$$
0 = u \cdot u = \lVert u \rVert^2 = \sum_i \left|u_i\right|^2
\Rightarrow u_i = 0
\Rightarrow u = 0
$$
Comment: @GitGud Thanks, I forgot to make it $\mathbb{C}$-proof.

question about inner product space


SSH keypair generation: RSA or DSA?

Question: SSH supports two signature algorithms for key pairs: RSA and DSA. Which is preferred, if any?  For RSA, what is the minimum acceptable key length?
Answer: RSA is generally preferred (now that the patent issue is over with) because it can go up to 4096 bits, where DSA has to be exactly 1024 bits (in the opinion of ssh-keygen).  2048 bits is ssh-keygen's default length for RSA keys, and I don't see any particular reason to use shorter ones.  (The minimum possible is 768 bits; whether that's "acceptable" is situational, I suppose.)
Comment: The bank SSL connection is a different kind of cipher being used and more specifically the key used for the main parts of the transaction is generated and used solely for that transaction and then discarded.
Comment: msanford: Like Ophidian said they are different kinds of encryptions. 256bit RSA keys would be completely useless for pretty much any purpose. The symmetric keys are in the range 128-512bit, while assymetric *start* at 768bit, and are secure at about 1500-2000bit and up. 768bit public keys can and have been broken. 128bit symmetric cannot (in itself) practically be broken.
Comment: @xenoterracide, ssh uses the openssl libraries. When SSH first starts it uses RSA/DSA keys to do host verification and to setup the symmetrical keys for the session. This is the same procedure that SSL servers and client follow, so you will find that we people talk about SSH, they will often refer to allow the research and documentation done for SSL
Comment: I have often wondered why people feel the need to secure their ssh connections with 2048 bit key, when your bank, which you presumably trust, is very unlikely to go over 256 bits, and is more likely to stick with 128 bits.

I'm certainly not saying there's anything wrong with using a large key, I'm just…sayin'.
Comment: actually RSA goes up to 4096. I always specify with a -b 4096. @spence what does SSL have to do with ssh? I use 4096 bit keys because generation time is a 1 time thing, and compared to what would happen if my keys fell into the wrong hands and were cracked. I don't notice any performance loss using a 4096 bit key after it's generated.
Comment: It is not true that RSA can only go up to 2048 bits. It is perfectly possible to create, say, a 4096 bit key.
Comment: The reason is actually to do with scaling.  encryption isn't cheap and the higher your keystrength is the less SSL connections you can serve.  If you have a retail bank with every consumer trying to use SSL then you are going to have to pick a compatible key suite but one that also matches your hardware.
Comment: GoDaddy now uses 2048-bit keys for SSL and others have pointed out that 1024-bit is now considered "at-risk" due to advances in computing power.
Answer: It's not about symmetric or asymetric encryption. It's about particular algorithms (RSA and DSA) requiring higher bit count to achieve acceptable security level. For example, ECC is also an asymmetric encryption, but it provides good security at much lower bit counts than RSA or DSA.
Answer: If you have a recent implementation of SSH, you may also consider ECDSA !
Answer: 256 bit certificates that banks use for SSL connections are symmetric cyphers such as 3DES or AES, hence the lower bit count.  When you see 1024 and 2048 (and even 4096) these are asymmetric ciphers.
Answer: DSA keys have much shorter signaures than RSA keys (for the same "level of security"), so one reason for using DSA would be in constrained environments.

SSH keypair generation: RSA or DSA?


Kolmogorov-Smirnov test with median ranks instead of traditional empirical distribution function

Question: I know that Kolmogorov-Smirnov test uses the empirical distribution function of the sample studied $\widehat{F}(X_i) = \frac{i}{n}$ and then measures the adequacy of function $\widehat{F}$ to function $F$.
In reliability, it is common to use the median ranks method (also known as Benard's approximation) instead of the traditional empirical distribution function, arguing that for small data samples, it's a more accurate approximation of the distribution.
Basically, it consists in calculating $\widehat{F}(X_i)=\frac{i-0.3}{n+0.4}$
My question is therefore : can I use Kolmogorov-Smirnov test approximating the distribution $\widehat{F}$ with Benard's approximation instead of empirical distribution function ?
In particular, it seems that the test relies on the fact that $\sqrt{n} \left\| \widehat{F} - F \right\|$ with $\widehat{F}$ being the traditional empirical distribution function, not the Benard's approximation. I wonder if the mathematical demonstration and properties (unbiased estimation, etc.) still holds if replacing it by Benard's approximation?
Thank you in advance for your help!
Answer: People create their own estimators all the time for various reasons, and it is fine to do so and explore/derive/prove their properties. However, it is a mistake to assume that an exotic estimator has the same properties as the usual estimator just because it estimates the same quantity. Consequently, you are free to create this KS-style test and explore/derive/prove its properties, but it is a mistake to believe that the test statistic and p-value will behave the same as the usual KS test. You are, after all, creating a new hypothesis test.
An early question of mine is related to this. I wanted to run the usual t-test but estimate the variance in a different way. The accepted answer notes that the test properties will change (though it does not seem that the changes will be so dramatic).
Comment: Thank you for you answer! Consequently, I guess that it would therefore be incorrect to use the table of critical values of the KS test in this case?


How to do $\frac{ \partial { \mathrm{tr}(XX^TXX^T)}}{\partial X}$

Question: How to do the derivative
\begin{equation}
\frac{ \partial {\mathrm{tr}(XX^TXX^T)}}{\partial X}\quad ?
\end{equation}
I have no idea where to start.
Comment: You could start here http://en.wikipedia.org/wiki/Matrix_calculus#Derivatives_with_matrices
Comment: What does $\frac{\partial}{\partial X}$ mean if $X$ is a matrix?
Comment: @Learner Thanks but is there any trick I can use if the function is $tr(XX^TXX^T)$?
Comment: @Rein, the derivative is really the "linear part" of a function, so if you want to take the derivative at $A$, write $X = A+H$, and capture the linear part involving $H$.
Answer: Write it down in terms of components. You want to know
$$\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T)
=\frac{\partial}{\partial X_{ij}} ( X_{kl} X_{ml} X_{mn} X_{kn}), $$
where summation over repeated indices is implied. Using the fact that
$$\frac{\partial}{\partial X_{ij}} X_{mn} =\delta_{im} \delta_{jn}$$
yields
$$\begin{align}\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T)
&= \delta_{ik} \delta_{jl} X_{ml} X_{mn} X_{kn}
 + \delta_{im} \delta_{jl} X_{kl} X_{mn} X_{kn}
+ \delta_{im} \delta_{jn} X_{kl} X_{ml}  X_{kn}
+ \delta_{ik} \delta_{jn}  X_{kl} X_{ml} X_{mn} \\
&= X_{mj} X_{mn} X_{in}+X_{kj} X_{in} X_{kn}
 + X_{kl} X_{il}  X_{kj} +X_{il} X_{ml} X_{mj}\\
&=4(X X^T X)_{ij} .
\end{align}$$
Or in short notation
$$\frac{\partial}{\partial X} \mathop{\rm tr}(X X^T X X^T) = 4 X X^T X.$$
Comment: You are assuming that the matrices are square and commute? We can not assume these properties.
Comment: @Elias: I did not assume any of those. Or where do you spot that I assume something like that?
Answer: Define a new matrix variable $$M=XX^T$$ and write the function in terms of this new variable and the double-dot (aka Frobenius) product.
When written in this form, finding the differential and gradient is easy
$$\eqalign{
 f &= M:M \cr
\cr
df &= 2\,M:dM \cr
  &= 2\,M:(dX\,X^T+X\,dX^T) \cr
  &= 2\,(M+M^T):dX\,X^T \cr
  &= 4\,MX:dX \cr
\cr
\frac{\partial f}{\partial X} &= 4\,MX \cr
 &= 4\,XX^TX \cr
\cr
}$$
For reference, the double-dot product is defined such that $$A:B=\operatorname{tr}(A^TB)$$
Answer: By definition the derivative of $F(X)=tr(XX^TXX^T)$, in the point $X$, is the only linear functional $DF(X):{\rm M}_{n\times n}(\mathbb{R})\to \mathbb{R}$  such that
$$
F(x+H)=F(X)+DF(X)\cdot H+r(H)
$$
with $\lim_{H\to 0} \frac{r(H)}{\|H\|}=0$. Let's get $DF(X)(H)$ and $r(H)$ by the expansion of $F(X+H)$. But first we must do an algebraic manipulation to expand $(X+H)(X+H)^T(X+H)(X+H)^T$. In fact,
\begin{align}
(X+\color{red}{H})(X+\color{red}{H})^T(X+\color{red}{H})(X+\color{red}{H})^T
=&
(X+\color{red}{H})(X^T+\color{red}{H}^T)\big(XX^T+X\color{red}{H}^T+\color{red}{H}X^T+\color{red}{H}\color{red}{H}^T\big)
\\
=&(X+\color{red}{H})\Big(X^TXX^T+X^TX\color{red}{H}^T+X^T\color{red}{H}X^T+X^T\color{red}{H}\color{red}{H}^T
\\
&\hspace{12mm}+\color{red}{H}^TXX^T+\color{red}{H}^TX\color{red}{H}^T+\color{red}{H}^T\color{red}{H}X^T+\color{red}{H}^T\color{red}{H}\color{red}{H}^T\Big)
\\
=&\;\;\;\;\,XX^TXX^T+XX^TX\color{red}{H}^T+XX^T\color{red}{H}X^T+XX^T\color{red}{H}\color{red}{H}^T
\\
&+X\color{red}{H}^TXX^T+X\color{red}{H}^TX\color{red}{H}^T+X\color{red}{H}^T\color{red}{H}X^T+X\color{red}{H}^T\color{red}{H}\color{red}{H}^T
\\
&+\color{red}{H}X^TXX^T+\color{red}{H}X^TX\color{red}{H}^T+\color{red}{H}X^T\color{red}{H}X^T+\color{red}{H}X^T\color{red}{H}\color{red}{H}^T
\\
&+\color{red}{H}\color{red}{H}^TXX^T+\color{red}{H}\color{red}{H}^TX\color{red}{H}^T+\color{red}{H}\color{red}{H}^T\color{red}{H}X^T+\color{red}{H}\color{red}{H}^T\color{red}{H}\color{red}{H}^T
\end{align}
Extracting $XX^TXX^T$ and the portions where $H$ or $H^T$ appears only once and applying $tr$ we have
\begin{align}
F(X+H)=&tr\Big(
(X+\color{red}{H})(X^T+\color{red}{H}^T)(X+\color{red}{H})(X^T+\color{red}{H}^T)
\Big)
\\
=&\underbrace{tr \big(XX^TXX^T\big)}_{F(X)}
+\underbrace{tr\big(
 XX^TX\color{red}{H}^T+XX^T\color{red}{H}X^T
+X\color{red}{H}^TXX^T+\color{red}{H}X^TXX^T
\big)}_{DF(X)\cdot H}
\\
&+tr\Big(XX^T\color{red}{H}\color{red}{H}^T
+X\color{red}{H}^TX\color{red}{H}^T+X\color{red}{H}^T\color{red}{H}X^T+X\color{red}{H}^T\color{red}{H}\color{red}{H}^T
\\
&\hspace{12mm}+\color{red}{H}X^TX\color{red}{H}^T+\color{red}{H}X^T\color{red}{H}X^T+\color{red}{H}X^T\color{red}{H}\color{red}{H}^T
\\
&\underbrace{\hspace{12mm}+\color{red}{H}\color{red}{H}^TXX^T+\color{red}{H}\color{red}{H}^TX\color{red}{H}^T+\color{red}{H}\color{red}{H}^T\color{red}{H}X^T+\color{red}{H}\color{red}{H}^T\color{red}{H}\color{red}{H}^T\Big)}_{r(H)}
\end{align}
Here $\|H\|=\sqrt{tr(HH^T)}$ is the Frobenius norm and $\displaystyle\lim_{H\to 0}\frac{r(H)}{H}=0$. Then the total derivative is
\begin{align}
\mathcal{D}F(X)\cdot H =
&
tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg)
\\
+
&
tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg).
\\
\end{align}
The directional derivative is
$$
\frac{\partial}{\partial V}F(X)=\mathcal{D}F(X)\cdot V
$$
and the partial derivative is
$$
\frac{\partial}{\partial E_{ij}}F(X)=\mathcal{D}F(X)\cdot E_{ij}.
$$
Here $E_{ij}=[\delta_{ij}]_{n\times m}$.


send email to many recipients in C# check which one fail

Question: References to this link, I understand how to send email to many recipients my question is if one of them fail this mean Send(Msg) will throw error for all ? or just for specific recipient appreciate any help thanks
Comment: What  do you mean by fail?
Comment: You can't do that in your application. When the recipient is valid format but doesn't exist, the recipient SMTP server just send an email to the expeditor. So your application will not know about that at least you make  a process that check for specific email received from recipients.
Comment: Well, you could have checked the [documentation for SmtpClient.Send](https://msdn.microsoft.com/en-us/library/swas0fwc(v=vs.110).aspx) - _"When sending e-mail using Send to multiple recipients and the SMTP server accepts some recipients as valid and rejects others, Send sends e-mail to the accepted recipients and then a SmtpFailedRecipientsException is thrown. The exception will contain a listing of the recipients that were rejected."_
Comment: throw error @CodeNotFound
Comment: You can have only two kind of errors I think :
- First when the SMTP server is not reachable so the fail concerns all recipient addresses.
- Second when one of the recipient email is invalid so use the same solution in the link you posted in your question.
Comment: fail like email not exist or received the message @CodeNotFound
Comment: in case one of the recipient email is invalid this mean Send(Msg) will throw error? I intend to put send email in retry action, so this means the message will sent again to the other recipient this is my concern @CodeNotFound
Comment: Yes is the answer for your question. For that you need to call Send(Msg) for each recipient.
Comment: I am so grateful for your answer, the last thing is if all recipients are valid as email format but one of them, not real or not receive the message for some reason lets say not enough space, this means Send(Msg) will throw error ?  @CodeNotFound and I also need to call Send(Msg) for each recipient
Comment: No :). Send method just mean => I reach the SMTP server and just give ti the mail to deal with it. Everything else that happen after that your application are not notified about it.
Comment: my SMTP (send from smtp) true @CodeNotFound ?
Comment: @CodeNotFound kindly write your answer so I can mark as the true answer, thanks
Comment: @stuartd thanks it's really help
Answer: Assuming you are using System.Net.Mail.SmtpClient.Send(MailMessage), the documentation shows that it can throw a "SmtpFailedRecipientsException" exception when the message could not be delivered to one or more of the recipients in MailMessage.To, MailMessage.CC, or MailMessage.Bcc.
The SmtpFailedRecipientsException exception has an InnerExceptions property.
Gets one or more SmtpFailedRecipientExceptions that indicate the e-mail recipients with SMTP delivery errors.
https://msdn.microsoft.com/en-us/library/swas0fwc(v=vs.110).aspx
Note: Depending on the issue, the SMTP server may accept the message and fail to deliver it to the recipient further along in the process.
Comment: @amir no, in that situation there is no way to tell from the sending side that it failed. The only way to tell would be check the reply address you set to see if you got a email saying the send failed. Think of it like a real letter you send to a new town, you drop off your letter to the local post office, if  something went wrong on the way to the local post office you would know right away. But if the post office accepted your letter but the address turned out to be a empty lot, how do you find out? You find out via the post office in the other town using the reply address to send it back.
Comment: if  if all recipients are valid as email format but one of them not real or not receive the message for some reason lets say not enough space this means SmtpFailedRecipientsException will throw ?
Comment: which you would need to go out later and check your mailbox to see if you received the returned letter. If something else goes wrong, for example there was a fire at the new town's post office and your letter was burnt up destroying the return address label, there would be no way for the other post office to know where to send a note saying that the letter was not delivered, so just because the letter was not returned to you does not mean it got there successfully. SMTP works exactly the same, the only way you know if the email did not get there is if the server sent a reply email saying so.
Comment: thanks @Scott Chamberlain , so if I am sure the email format 100% true and my send(msg) in retry method no exception will be throw, my concern just not to resend to the other recipients again and the retry stop when no exception throw
Comment: @Scott Chamberlain so if one of them not real or not receive the message for some reason lets say not enough space, no exception will throw ?
Comment: @amir Yes. Just like if you dropped a real letter at the post office, once you drop it off you don't know what happens after.
Comment: Scott Chamberlain , Wiz , CodeNotFound , stuartd you all help me how can I mark you all as write answer :) because really every one help me in some point to understand the complete picture
Comment: @Scott Chamberlain
Comment: @Wiz ,,,,,,,,,,,
Comment: @CodeNotFound ,
Comment: @stuartd ,,,,,,,
Comment: @amir no problem, glad to help.


Visualize the point-line distance formula $\frac{\lvert ax_p+by_p+c\rvert}{\sqrt{a^2+b^2}}$

Question: Given two lines
\begin{cases}
ax+by+c=0\\
ax+by=0
\end{cases}
and a point $V(x_p, y_p)$, the point-line distance formula is:
$$\frac{\lvert ax_p+by_p+c\rvert}{\sqrt{a^2+b^2}}$$
I understand that the $ax_p+by_p$ is the inner product of $(a,b)$ and $V$ as a vector from the origin O, and I understand $\sqrt{a^2+b^2}$ is to make $(a,b)$ a unit vector, but how to draw the $$\frac{|c|}{\sqrt{a^2+b^2}}$$ in the following picture? What is this? Is it a vector?
Answer:
Consider a point $P:=(x_0,y_0)$ on the line $L:ax+by+c=0$. Let $\vec{n}$ be the normal vector to the line $L$. Then we see from the figure that the distance $d$ of the point $Q:=(x_p,y_p)$ from line $L$, is the orthogonal projection of $\vec{PQ}$ on $\vec{n}$, i.e. $$d=|\vec{PQ}|\cos\theta=\frac{|\vec{PQ}||\vec{n}|\cos\theta}{|\vec{n}|}=\frac{\left\langle\vec{PQ},\vec{n}\right\rangle}{|\vec{n}|},$$ where $\langle\cdot,\cdot\rangle$ denotes the inner product. Now $\vec{PQ}=(x_p-x_0,y_p-y_0)$ and $\vec{n}=(a,b)$. So $$\left\langle\vec{PQ},\vec{n}\right\rangle=a(x_p-x_0)+b(y_p-y_0)=ax_p+by_p+c,$$ as $c=-ax_0-by_0$. Also obviously $|\vec{n}|=\sqrt{a^2+b^2}$.
Answer: Hint: $c=-ax_{l}-by_{l}$ where $(x_{l},y_{l})$ lies on line $ax+by+c=0$. Inner product again?


set python recursion limit for a function

Question: I have 2 solutions to a recursion problem that I need for a function (actually a method). I want it to be recursive, but I want to set the recursion limit to 10 and reset it after the function is called (or not mess with recursion limit at all). Can anyone think of a better way to do this or recommend using one over the others? I'm leaning towards the context manager because it keeps my code cleaner and no setting the tracebacklimit, but there might be caveats?
import sys

def func(i=1):
    print i
    if i > 10:
        import sys
        sys.tracebacklimit = 1
        raise ValueError("Recursion Limit")
    i += 1
    func(i)

class recursion_limit(object):
    def __init__(self, val):
        self.val = val
        self.old_val = sys.getrecursionlimit()
    def __enter__(self):
        sys.setrecursionlimit(self.val)
    def __exit__(self, *args):
        sys.setrecursionlimit(self.old_val)
        raise ValueError("Recursion Limit")

def func2(i=1):
    """
    Call as

    with recursion_limit(12):
        func2()
    """
    print i
    i += 1
    func2(i)

if __name__ == "__main__":
    #    print 'Running func1'
    #    func()

    with recursion_limit(12):
        func2()

I do see some odd behavior though with the context manager. If I put in main
with recursion_limit(12):
    func2()

It prints 1 to 10. If I do the same from the interpreter it prints 1 to 11. I assume there is something going on under the hood when I import things?
EDIT: For posterity this is what I have come up with for a function that knows its call depth. I doubt I'd use it in any production code, but it gets the job done.
import sys
import inspect
class KeepTrack(object):
    def __init__(self):
        self.calldepth = sys.maxint

    def func(self):
        zero = len(inspect.stack())
        if zero < self.calldepth:
            self.calldepth = zero
        i = len(inspect.stack())
        print i - self.calldepth
        if i - self.calldepth < 9:
            self.func()

keeping_track = KeepTrack()
keeping_track.func()

Comment: Why not just use `for _ in xrange(N)` to do something N times? Is conceptually simpler than abusing a `RuntimeError`,  allows calling other functions without exiting prematurely (or, worse, requiring the caller to take care of that), and should be more efficient as a bonus.
Comment: When you do it in main, you're already in a function. When you're in the interpreter, you're not.
Comment: Why would you possibly want to do this?
Comment: you can use decorators to check the recursion limit
Comment: can you avoid global state with decorators (and keep the wrapped function call argument list clean)?
Comment: I am trying to grab some json from a server. I don't know how long the response is going to be on the front end, but I want to make the call *at most* ten times. Recursion seems like the way to go. ValueError is of course incorrect. Just a reflex.
Answer: You shouldn't change the system recursion limit at all.  You should code your function to know how deep it is, and end the recursion when it gets too deep.
The reason the recursion limit seems differently applied in your program and the interpreter is because they have different tops of stack: the functions invoked in the interpreter to get to the point of running your code.
Answer: While somewhat tangential (I'd have put it in a comment, but I don't think there's room), it should be noted that setrecursionlimit is somewhat misleadingly named - it actually sets the maximum stack depth:
http://docs.python.org/library/sys.html#sys.setrecursionlimit
That's why the function behaves differently depending on where you call it from.  Also, if func2 were to make a stdlib call (or whatever) that ended up calling a number of functions such that it added more than N to the stack, the exception would trigger early.
Also also, I wouldn't change the sys.tracebacklimit either; that will have an effect on the rest of your program.  Go with Ned's answer.
Answer: ignoring the more general issues, it looks like you can get the current frame depth by looking at the length of inspect.getouterframes().  that would give you a "zero point" from which you can set the depth limit (disclaimer: i haven't tried this).
edit: or len(inspect.stack()) - it's not clear to me what the difference is.  i would be interested in knowing if this works, and whether they were different.
Answer: I'd definitely choose the first approach, it is simpler and self explaining. After all the recursion limit is your explicit choice, so why obfuscate it?


Mahout Scability

Question: Do you know any real world examples of how much Mahout can scale? I wonder how much it can scale in collaborative filtering, clustering, and classification ?
Answer: It's not only about the scalability of Mahout. But the scalability of the model used by the algorithm is just as much a question.
K-means is good for teaching clustering, and it works surprisingly often despite being based on some very limiting assumptions - such as Voronoi cell shaped clusters.
But when you scale this up to big data, these things totally get out of control. With k-means, it is already difficult to choose $k$. But if you scale it up to a data set that is 100 times as big, you likely also should increase $k$. To 10x-100x the $k$. But this will make choosing the right $k$, a good initialization, doing enough itertions to get a sound result etc. a lot harder.
So while Mahout enables you to run k-means on much larger data sets, k-means itself may not be appropriate for data sets of this size anymore. Because the logic of k-means is just so much more likely to be flawed on this amount of data.
Btw, I've seen reports here on stackexchange/stackoverflow that k-means in Mahout might be buggy when run on more than one node. Some user reported that even with the same initialization, he keeps on getting different results when running one a different number of nodes. As the implementation supposedly is exact and only depends on the initialization, this should not happen.

Mahout Scability