Evaluate enhancement using hyphotesis tests

Question: Recently, I took a test in statistics. The test stated something like this (I can't recall exactly the words).
You realized 5 experiments with 100000 users each to evaluate a enhancement after some interface changes. The first one (the control group) received a 10% return. The other 4 experiments had the following returns 7%, 8.5%, 12% and 14%. Is possible to draw any conclusion with 95% of confidence in any of the 4 experiments when compared with control group?
My approach to this was to make a simple z-test over a Bernoulli distribution using $\alpha  = 0.05$. I used the two tailed test, with the null hypothesis as difference between control group probability and the given experiment probability is not significant. It means for example comparing the control group (10%) with the 7% experiment group that
$H_0 = p_{control} - p_{exp7} = 0$
For that I took the following approach

Computed the probability of event occur

$\hat{p} = (10000 + 7000)/(100000+100000) = 0.085$

Computed the standard error

$SE = \sqrt{0.085 * (1 - 0.085) * (\frac{1}{100000} + \frac{1}{100000})}$

Computed the test statistics

$t = (0.1 - 0.07)/ SE \approx 24.054$

Check $z_{\alpha}$ for 95% in two-sided Z-distribution $\approx 1.96$

Since the $t$ is outside the interval $[-z_{\alpha}, z_{\alpha}]$, I assume that I can reject the null-hypothesis that the difference is non-significant, hence I conclude that experiment 7% significantly worse than the control experiment.
Is that correct? Is there better tests or approaches to that? Should I use one-tailed and the null hypothesis to $\bar{X} \le \bar{Y}$? Is there any functions in Python or R that can be used to this kind of experiments, as we have for t-tests with samples**?
** In Python we can use scipy.stats.t.ttest_ind and in R we have t.test()
Comment: That ratio makes no sense.  Your percentages, however, correspond to differences in the thousands, each of which is ten or more times any standard error: that is so large that no testing is needed.
Comment: You're working too hard.  What, approximately, are the standard errors of the total return in each experiment?  How do the differences in total returns compare to those standard errors?
Comment: For each $i$-percentage  we have the given the approximate $i$-th SE:  ${SE}_i = \sqrt{(p * (1-p)) / (1/n)} \approx  [94, 80, 88, 102, 109]$. SE total, computed with $SE_{tot} = \sqrt{\sum{SE_i^2}} \approx 214$. Computing the differences $SE_{tot} - SE_i \approx [119, 134, 126,111, 104]$,
Comment: Good.  And how do the differences in the total returns compare to those SEs?
Comment: Some are bigger the last are smaller. We can compute a ratio $\frac{SE_i}{SE_{tot} - SE_i} \approx [ 0.79, 0.60, 0.70, 0.92, 1.05 ]$.
Comment: Also to stay something else, you probably want some sort of multiple testing correction (see the concept of [Bonferroni correction](https://en.wikipedia.org/wiki/Bonferroni_correction) for example).
Answer: Your testing procedures look okay. The method you are using is Wald CI of binomial proportions. Some errors are

The null hypothesis is "the control and experiment groups have the same probability of return." Hypotheses must be regarding population parameters (i.e. probability) instead of sample estimates (i.e. proportion). Do not associate statistical significance with hypotheses. Here I am confused what return means and whether the percentages have probability interpretation. Do they actually mean percentage increases instead, so an increase from an average score of 80 to 88 leads to 10% return? If so, testing binomial proportions may not be correct.

The standard error of the difference should be $\text{SE} =\sqrt{p_0(1 - p_0)/n_0 + p_1(1 - p_1)/n_1}$. The formula you used assumes equal variances between two groups, which is not appropriate in most cases.

Using a two-sided test, the correct conclusion is that "the control and experiment groups have different probabilities of return" or "the return probability in the experiment group significantly differs from that in the control group." Again, we must talk about population parameters that are not observed but inferred.

Should I use a one-tailed test?

You can but do no have to. One-tailed tests correspond to one-sided CI (the other side is the theoretical limit, here +1 or -1 for a probability).

Is there better tests or approaches to that? Is there any functions in Python or R that can be used to this kind of experiments?

Yes, in R you should use DescTools::BinomDiffCI(x1, n1, x2, n2, conf.level = 0.95, sides = c("two.sided","left","right"), method = c("ac", "wald", "waldcc", "score", "scorecc", "mn", "mee", "blj", "ha", "hal", "jp")). As it shows, there are multiple methods to construct the confidence interval. You can check out the details by sending ??BinomDiffCI in the console and check out the script for formula by Ctrl+click BinomDiffCI() in a line in your script editor. Since your sample size is quite large, they will differ by a tiny amount.
If the task is hypothesis testing instead of confidence intervals, you can use (1) chi-square test of independence in chisq.test() (2) permutation test of independence with resampling coin::chisq_test(). Bootstrapped confidence intervals for proportions is possible but not very meaningful. See https://cran.r-project.org/web/packages/confintr/vignettes/confintr.html. In theory t.test() is not appropriate for binomial proportions although in your case it will still give very similar confidence intervals as DescTools::BinomDiffCI() does.
Comment: This https://math.stackexchange.com/help/someone-answers shows how to accept one's answer.
Comment: Jerry Tao, what is your point with that last comment?


How to generate a large full-rank random correlation matrix with some strong correlations present?

Question: I would like to generate a random correlation matrix $\mathbf C$ of $n \times n$ size such that there are some moderately strong correlations present:

square real symmetric matrix of $n \times n$ size, with e.g. $n=100$;
positive-definite, i.e. with all eigenvalues real and positive;
full rank;
all diagonal elements equal to $1$;
off-diagonal elements should be reasonably uniformly distributed on $(-1, 1)$. Exact distribution does not matter, but I would like to have some moderately large amount (e.g. $10\%$) of moderately large values (e.g. with absolute value of $0.5$ or higher). Basically I want to make sure that $\mathbf C$ is not almost diagonal with all off-diagonal elements $\approx 0$.

Is there a simple way to do it?
The purpose is to use such random matrices to benchmark some algorithms working with correlation (or covariance) matrices.

Methods that do not work
Here are some ways to generate random correlation matrices that I know of, but that do not work for me here:

Generate random $\mathbf X$ of $s \times n$ size, center, standardize and form the correlation matrix $\mathbf C=\frac{1}{s-1}\mathbf X^\top \mathbf X$. If $s>n$, this will generally result in all off-diagonal correlations being around $0$. If $s\ll n$, some correlations will be strong, but $\mathbf C$ will not be full rank.
Generate random positive definite matrix $\mathbf B$ in one of the following ways:

Generate random square $\mathbf A$ and make symmetric positive definite $\mathbf B = \mathbf A \mathbf A^\top$.
Generate random square $\mathbf A$, make symmetric $\mathbf E = \mathbf A + \mathbf A^\top$, and make it positive definite by performing eigen-decomposition $\mathbf E = \mathbf U \mathbf S \mathbf U^\top$ and setting all negative eigenvalues to zero: $\mathbf B = \mathbf U \:\mathrm{max}\{\mathbf S, \mathbf 0\} \:\mathbf U^\top$. NB: this will result in a rank-deficient matrix.
Generate random orthogonal $\mathbf Q$ (e.g. by generating random square $\mathbf A$ and doing its QR decomposition, or via Gram-Schmidt process) and random diagonal $\mathbf D$ with all positive elements; form $\mathbf B = \mathbf Q \mathbf D \mathbf Q^\top$.

Obtained matrix $\mathbf B$ can be easily normalized to have all ones on the diagonal: $\mathbf C = \mathbf D^{-1/2}\mathbf B \mathbf D^{-1/2}$, where $\mathbf D = \mathrm{diag}\:\mathbf B$ is the diagonal matrix with the same diagonal as $\mathbf B$. All three ways listed above to generate $\mathbf B$ result in $\mathbf C$ having off-diagonal elements close $0$.

Update: Older threads
After posting my question, I found two almost duplicates in the past:

How to generate random correlation matrix that has approximately normally distributed off-diagonal entries with given standard deviation?
How to efficiently generate random positive-semidefinite correlation matrices?

Unfortunately, none of these threads contained a satisfactory answer (until now :)
Comment: You might want to look into the LKJ distribution
Comment: @ttnphns: I think I have finally understood that you were right all along: what you suggested is the simplest way to get to the goal. I added an update to my answer implementing essentially what you wrote above.
Comment: You could create random orthogonal matrix by QR or Gram-Schmidt processes. That will be "eigenvectors of PCA". Add scale to its columns (turn into "loadings"). Get the covariance matrix from these loadings. Something like that...
Comment: Uhm, well.. Imagine we want to create a `nXk` loading matrix W, not fully random but the one we want (it will, `WW'+diag(noise)`, define the cov matrix we seek. The only task is to correct the column-normalized W (i.e. the k "eigenvectors") to become orthogonal. Any method to de-correlate correlated variables (here variables are the eigenvectors) will probably do. (This a raw idea.)
Comment: Ah, @whuber, now I see what you mean. Yes, you are right: if all off-diagonal elements are identical and equal to $\rho$, then the matrix is indeed full rank and positive-definite... This is of course not what I had in mind: I would like the distribution of off-diagonal elements *in each matrix* to be reasonably "spread", not the distribution *across matrices*...
Comment: @ssdecontrol: I read the LKJ paper, and it indeed provided an answer. See my own answer here if you are interested. Thanks a lot for the hint!
Comment: Attaching a link to a paper about random correlation matrices. If you make each off diagonal element U(0, 1) then you risk making the matrix itself non-positive definite, so marginally U(0, 1) elements require a strong correlation between elements. http://www.rob-mcculloch.org/some_papers_and_talks/papers/published/barnard-meng-mcculloch.pdf
Comment: @ttnphns: Yes, but if I generate a random orthogonal matrix and choose random eigenvalues, then the resulting covariance/correlation matrix is again pretty much diagonal (I have just tried).
Comment: You can obtain a *perfectly* uniform distribution of all off-diagonal elements in various trivial ways by making them all equal to $\pm \rho$ (with suitable choices of sign patterns) and drawing $\rho$ uniformly in $(-1,0)$ (for some sign choices) or in $(0,1)$ (for other sign choices). I wouldn't recommend this because the set of matrices is so special: the point is that perhaps you should be applying additional criteria to your correlation matrices so that they cover a more realistic set of possibilities.
Comment: Thanks, @whuber, but I am not sure I understand: if I independently generate the off-diagonal values, the resulting matrix will not be positive-definite, i.e. cannot be a covariance/correlation matrix.
Comment: You don't generate them independently.  You can generate a mixture of matrices where (1) all the off-diagonal values equal $\rho$ or (2) they equal $(-1)^{i+j}\rho$ in the $(i,j)$ position.
Answer: Other answers came up with nice tricks to solve my problem in various ways. However, I found a principled approach that I think has a large advantage of being conceptually very clear and easy to adjust.
In this thread: How to efficiently generate random positive-semidefinite correlation matrices? -- I described and provided the code for two efficient algorithms of generating random correlation matrices. Both come from a paper by Lewandowski, Kurowicka, and Joe (2009), that @ssdecontrol referred to in the comments above (thanks a lot!).
Please see my answer there for a lot of figures, explanations, and matlab code. The so called "vine" method allows to generate random correlation matrices with any distribution of partial correlations and can be used to generate correlation matrices with large off-diagonal values. Here is the example figure from that thread:

The only thing that changes between subplots, is one parameter that controls how much the distribution of partial correlations is concentrated around $\pm 1$.
I copy my code to generate these matrices here as well, to show that it is not longer than the other methods suggested here. Please see my linked answer for some explanations. The values of betaparam for the figure above were ${50,20,10,5,2,1}$ (and dimensionality d was $100$).
function S = vineBeta(d, betaparam)
    P = zeros(d);           %// storing partial correlations
    S = eye(d);

    for k = 1:d-1
        for i = k+1:d
            P(k,i) = betarnd(betaparam,betaparam); %// sampling from beta
            P(k,i) = (P(k,i)-0.5)*2;     %// linearly shifting to [-1, 1]
            p = P(k,i);
            for l = (k-1):-1:1 %// converting partial correlation to raw correlation
                p = p * sqrt((1-P(l,i)^2)*(1-P(l,k)^2)) + P(l,i)*P(l,k);
            end
            S(k,i) = p;
            S(i,k) = p;
        end
    end

    %// permuting the variables to make the distribution permutation-invariant
    permutation = randperm(d);
    S = S(permutation, permutation);
end

Update: eigenvalues
@psarka asks about the eigenvalues of these matrices. On the figure below I plot the eigenvalue spectra of the same six correlation matrices as above. Notice that they decrease gradually; in contrast, the method suggested by @psarka generally results in a correlation matrix with one large eigenvalue, but the rest being pretty uniform.

Update. Really simple method: several factors
Similar to what @ttnphns wrote in the comments above and @GottfriedHelms in his answer, one very simple way to achieve my goal is to randomly generate several ($k<n$) factor loadings $\mathbf W$ (random matrix of $k \times n$ size), form the covariance matrix $\mathbf W \mathbf W^\top$ (which of course will not be full rank) and add to it a random diagonal matrix $\mathbf D$ with positive elements to make $\mathbf B = \mathbf W \mathbf W^\top + \mathbf D$ full rank. The resulting covariance matrix can be normalized to become a correlation matrix (as described in my question). This is very simple and does the trick. Here are some example correlation matrices for $k={100, 50, 20, 10, 5, 1}$:

The only downside is that the resulting matrix will have $k$ large eigenvalues and then a sudden drop, as opposed to a nice decay shown above with the vine method. Here are the corresponding spectra:

Here is the code:
d = 100;    %// number of dimensions
k = 5;      %// number of factors

W = randn(d,k);
S = W*W' + diag(rand(1,d));
S = diag(1./sqrt(diag(S))) * S * diag(1./sqrt(diag(S)));

Comment: Translating this into R: `W = replicate(k, rnorm(d)); S = W%*%t(W) + diag(rnorm(d),nrow=d); S = diag(1/sqrt(diag(S)))%*%S%*%diag(1/sqrt(diag(S)))`
Comment: @Mihai, good point and your suggestions are likely the simplest. You could also do `S <- matrix(nearPD(S, corr = TRUE, keepDiag = TRUE)$mat@x,ncol(S),ncol(S))`
Comment: +1. However, here's just a reminder to your last section on "factor method". Strictly right approach calls that columns of `W` are orthogonal (i.e. cosines between them are 0). Simply generating random `W` of course doesn't provide it. If they aren't orthogonal - i.e. factors are oblique (call then `W` as `W_`) - factor theorem is not `WW'` but `W_CW_'` with `C` being "correlations" (cosines) between factors. Now, `C=Q'Q` with `Q` being the nonorthogonal rotation matrix of rotation `W_=inv(Q)'W` (and so `W=W_Q'`). Generate some `Q` - a matrix with column ss = 1 and matrix ss = size of matrix.
Comment: ...typo: not `W_=inv(Q)'W`, of course `W_= W inv(Q)'`.
Comment: @ttnphns: What you are saying is correct, but I don't think it matters for the purposes of *generating* random correlation matrices. If I generate $W$ randomly, then yes, its columns will not be exactly orthogonal, but $WW^\top+D$ (after normalization to get all ones on the diagonal) will still be full rank correlation matrix with some large off-diagonal values, which is what the question was about. Of course it is very easy to orthogonalize columns of $W$ beforehand, I just did not see why this would be necessary in this case.
Comment: I think your randomly generated $WW^{T} + D$ is not positive definite in general, which is a problem.
Comment: @lionel How can it fail to be positive semi-definite? Of course $D$ should have only non-negative elements.
Comment: I don't have a lot of experience or intuition with linear algebra, but generate some data and test for positive-definitess. None of the vcov matrices I generate with this method are positive definite.
Comment: @lionel You must be doing something wrong. The test is presented in my answer. The spectra have all positive eigenvalues.
Comment: you are right sorry. I was confused by the usual definition of a factor model and was adding normal variates. With an all positive uniform(0, 1) it works. Also the resulting distributions of covariances make a lot more sense than when I was using a nearest PD matrix procedure to fix the non-PD vcov matrices.
Comment: Hi. I find myself with a similar problem and I was wondering whether the factor method is guaranteed to also uniformly sample from the space of all possible d X d correlation matrices like the method described in  Lewandowski, Kurowicka, and Joe (2009)? If that is the case, is there any proof or derivation that shows it?
Comment: @ScottWorland Watch out that `diag(rnorm(d),nrow=d)` will likely also draw negative values which will result in a non positive definite `W` matrix. I would replace it with `diag(runif(d),nrow=d)` or tweak the parameters of `rnorm` such that you guarantee positive samples.
Answer: A simple thing but maybe will work for benchmark purposes: took your 2. and injected some correlations into starting matrix. Distribution is somewhat uniform, and changing $a$ you can get concentration near 1 and -1 or near 0.
import numpy as np
from random import choice
import matplotlib.pyplot as plt

n = 100
a = 2

A = np.matrix([np.random.randn(n) + np.random.randn(1)*a for i in range(n)])
A = A*np.transpose(A)
D_half = np.diag(np.diag(A)**(-0.5))
C = D_half*A*D_half

vals = list(np.array(C.ravel())[0])
plt.hist(vals, range=(-1,1))
plt.show()
plt.imshow(C, interpolation=None)
plt.show()

Comment: (+1) Thanks! I edited your answer to add the prettify coloring scheme for python and make it python2 compatible :) Hope it's okay. I also deleted my above comments to remove clutter (you can delete yours as well). I am now trying to understand the logic of your code; do you really need `crs` array? What influence does $k$ have? My feeling is that you can simply add a random number from $[-a, a]$ to each row, no? This is similar to using matrix $X$ with correlated *samples* (not features) and then computing its sample correlation matrix, correct?
Comment: Yes, you are completely right! (Oh boy, that was indeed silly :D). I changed the random part to randn(1)*a and now it's much better.
Comment: Thanks! Now you only need to remove your mentioning of $k$ above the code. Meanwhile, I found an old question that is nearly a duplicate, and posted an answer there, directly inspired by yours here: [How to generate random correlation matrix that has approximately normally distributed off-diagonal entries with given standard deviation?](http://stats.stackexchange.com/questions/10121). Seems to work nice!
Comment: Cool! I wonder if such procedure results in a known distribution. Experimenting with different $a$ and $n$ I can get quite a few different shapes.
Comment: One downside of this method is that the resulting correlation matrix has one large eigenvalue, but the remaining ones are nearly uniform. So this procedure does not yield a "general" correlation matrix... Not that I specified it in my question. But @ssdecontrol mentioned in the comments above that apparently there are ways to sample from all correlation matrices; this looks interesting but much more complicated.
Comment: Thanks, I know nothing about numerical stability, so it was nice to play with it. It seems that `np.linalg.solve` is very accurate to begin with (I got an error of the order `e-14` for well conditioned matrix), so maybe condition number $7\times 10^{11}$ is still not too bad?
Comment: Hi again, I read a paper about sampling random correlation matrices, and one of the methods from there can be used to do exactly what I need. I posted an answer here, you might be interested to take a look! It links to a much more detailed answer I wrote in another thread.
Comment: Very nice! Have you looked at the distribution of the eigenvalues of your method, by the way? I bet it is more interesting than in my example.
Comment: Yes, eigenvalues decrease gradually. I made an update with an additional figure showing eigenvalues.
Answer: Hmm, after I' done an example in my MatMate-language I see that there is already a python-answer, which might be preferable because python is widely used. But because you had still questions I show you my approach using the Matmate-matrix-language, perhaps it is more selfcommenting.
Method 1
(Using MatMate):
v=12         // 12 variables
f=3          // subset-correlation based on 3 common factors
vg = v / f   // variables per subsets

 // generate hidden factor-matrix
             // randomu(rows,cols ,lowbound, ubound) gives uniform random matrix
             //    without explicite bounds the default is: randomu(rows,cols,0,100)
L = {   randomu(vg,f)     || randomu(vg,f)/100  || randomu(vg,f)/100 , _
        randomu(vg,f)/100 || randomu(vg,f)      || randomu(vg,f)/100 , _
        randomu(vg,f)/100 || randomu(vg,f)/100  || randomu(vg,f)     }

 // make sure there is itemspecific variance
 // by appending a diagonal-matrix with random positive entries
L = L || mkdiag(randomu(v,1,10,20))
  // make covariance and correlation matrix
cov = L *'   // L multiplied  with its transpose
cor = covtocorr(cov)
                   set ccdezweite=3 ccfeldweite=8
                   list cor
cor =
   1.000,   0.321,   0.919,   0.489,   0.025,   0.019,   0.019,   0.030,   0.025,   0.017,   0.014,   0.014
   0.321,   1.000,   0.540,   0.923,   0.016,   0.015,   0.012,   0.030,   0.033,   0.016,   0.012,   0.015
   0.919,   0.540,   1.000,   0.679,   0.018,   0.014,   0.012,   0.029,   0.028,   0.014,   0.012,   0.012
   0.489,   0.923,   0.679,   1.000,   0.025,   0.022,   0.020,   0.040,   0.031,   0.014,   0.011,   0.014
   0.025,   0.016,   0.018,   0.025,   1.000,   0.815,   0.909,   0.758,   0.038,   0.012,   0.018,   0.014
   0.019,   0.015,   0.014,   0.022,   0.815,   1.000,   0.943,   0.884,   0.035,   0.012,   0.014,   0.012
   0.019,   0.012,   0.012,   0.020,   0.909,   0.943,   1.000,   0.831,   0.036,   0.013,   0.015,   0.010
   0.030,   0.030,   0.029,   0.040,   0.758,   0.884,   0.831,   1.000,   0.041,   0.017,   0.022,   0.020
   0.025,   0.033,   0.028,   0.031,   0.038,   0.035,   0.036,   0.041,   1.000,   0.831,   0.868,   0.780
   0.017,   0.016,   0.014,   0.014,   0.012,   0.012,   0.013,   0.017,   0.831,   1.000,   0.876,   0.848
   0.014,   0.012,   0.012,   0.011,   0.018,   0.014,   0.015,   0.022,   0.868,   0.876,   1.000,   0.904
   0.014,   0.015,   0.012,   0.014,   0.014,   0.012,   0.010,   0.020,   0.780,   0.848,   0.904,   1.000

The problem here might be, that we define blocks of submatrices which have high correlations within with little correlation between and this is not programmatically but by the constant concatenation-expressions . Maybe this approach could be modeled more elegantly in python.

Method 2(a)
After that, there is a completely different approach, where we fill the possible remaining covariance by random amounts of 100 percent into a factor-loadings-matrix. This is done in Pari/GP:
{L = matrix(8,8);  \\ generate an empty factor-loadings-matrix
for(r=1,8,
   rv=1.0;    \\ remaining variance for variable is 1.0
   for(c=1,8,
        pv=if(c<8,random(100)/100.0,1.0); \\ define randomly part of remaining variance
        cv= pv * rv;  \\ compute current partial variance
        rv = rv - cv;     \\ compute the now remaining variance
        sg = (-1)^(random(100) % 2) ;  \\ also introduce randomly +- signs
        L[r,c] = sg*sqrt(cv) ;  \\ compute factor loading as signed sqrt of cv
       )
     );}

cor = L * L~

and the produced correlation-matrix is
     1.000  -0.7111  -0.08648   -0.7806   0.8394  -0.7674   0.6812    0.2765
   -0.7111    1.000   0.06073    0.7485  -0.7550   0.8052  -0.8273   0.05863
  -0.08648  0.06073     1.000    0.5146  -0.1614   0.1459  -0.4760  -0.01800
   -0.7806   0.7485    0.5146     1.000  -0.8274   0.7644  -0.9373  -0.06388
    0.8394  -0.7550   -0.1614   -0.8274    1.000  -0.5823   0.8065   -0.1929
   -0.7674   0.8052    0.1459    0.7644  -0.5823    1.000  -0.7261   -0.4822
    0.6812  -0.8273   -0.4760   -0.9373   0.8065  -0.7261    1.000   -0.1526
    0.2765  0.05863  -0.01800  -0.06388  -0.1929  -0.4822  -0.1526     1.000

Possibly this generates a correlation-matrix with dominant principal components because of the cumulative generating-rule for the factor-loadings-matrix. Also it might be better to assure positive definiteness by making the last portion of variance a unique factor. I left it in the program to keep the focus on the general principle.
A 100x100 correlation-matrix had the following frequencies of correlations (rounded to 1 dec place)
    e    f            e: entry(rounded) f: frequency
  -----------------------------------------------------
  -1.000, 108.000
  -0.900, 460.000
  -0.800, 582.000
  -0.700, 604.000
  -0.600, 548.000
  -0.500, 540.000
  -0.400, 506.000
  -0.300, 482.000
  -0.200, 488.000
  -0.100, 464.000
   0.000, 434.000
   0.100, 486.000
   0.200, 454.000
   0.300, 468.000
   0.400, 462.000
   0.500, 618.000
   0.600, 556.000
   0.700, 586.000
   0.800, 536.000
   0.900, 420.000
   1.000, 198.000

[update]. Hmm, the 100x100 matrix is badly conditioned; Pari/GP cannot determine the eigenvalues correctly with the polroots(charpoly())-function even with 200 digits precision. I've done a Jacobi-rotation to pca-form on the loadingsmatrix L and find  mostly extremely small eigenvalues, printed them in logarithms to base 10 (which give roughly the position of the decimal point). Read from left to right and then row by row:
log_10(eigenvalues):
   1.684,   1.444,   1.029,   0.818,   0.455,   0.241,   0.117,  -0.423,  -0.664,  -1.040
  -1.647,  -1.799,  -1.959,  -2.298,  -2.729,  -3.059,  -3.497,  -3.833,  -4.014,  -4.467
  -4.992,  -5.396,  -5.511,  -6.366,  -6.615,  -6.834,  -7.535,  -8.138,  -8.263,  -8.766
  -9.082,  -9.482,  -9.940, -10.167, -10.566, -11.110, -11.434, -11.788, -12.079, -12.722
 -13.122, -13.322, -13.444, -13.933, -14.390, -14.614, -15.070, -15.334, -15.904, -16.278
 -16.396, -16.708, -17.022, -17.746, -18.090, -18.358, -18.617, -18.903, -19.186, -19.476
 -19.661, -19.764, -20.342, -20.648, -20.805, -20.922, -21.394, -21.740, -21.991, -22.291
 -22.792, -23.184, -23.680, -24.100, -24.222, -24.631, -24.979, -25.161, -25.282, -26.211
 -27.181, -27.626, -27.861, -28.054, -28.266, -28.369, -29.074, -29.329, -29.539, -29.689
 -30.216, -30.784, -31.269, -31.760, -32.218, -32.446, -32.785, -33.003, -33.448, -34.318

[update 2]
Method 2(b)
An improvement might be to increase the itemspecific variance to some non-marginal level and reduce to a reasonably smaller number of common factors (for instance integer-squareroot of itemnumber):
{  dimr = 100;
   dimc = sqrtint(dimr);        \\ 10 common factors
   L = matrix(dimr,dimr+dimc);  \\ loadings matrix
                                \\     with dimr itemspecific and
                                \\          dimc common factors
   for(r=1,dim,
         vr=1.0;                \\ complete variance per item
         vu=0.05+random(100)/1000.0;   \\ random variance +0.05
                                       \\ for itemspecific variance
         L[r,r]=sqrt(vu);              \\ itemspecific factor loading
         vr=vr-vu;
         for(c=1,dimc,
                cv=if(c<dimc,random(100)/100,1.0)*vr;
                vr=vr-cv;
                L[r,dimr+c]=(-1)^(random(100) % 2)*sqrt(cv)
             )
        );}

   cov=L*L~
   cp=charpoly(cov)   \\ does not work even with 200 digits precision
   pr=polroots(cp)    \\ spurious negative and complex eigenvalues...

The structure of the result
in term of the distribution of correlations:
remains similar (also the nasty non decomposability by PariGP), but the eigenvalues, when found by jacobi-rotation of the loadingsmatrix, have now a better structure, for a newly computed example I got the eigenvalues as
log_10(eigenvalues):
   1.677,   1.326,   1.063,   0.754,   0.415,   0.116,  -0.262,  -0.516,  -0.587,  -0.783
  -0.835,  -0.844,  -0.851,  -0.854,  -0.858,  -0.862,  -0.862,  -0.868,  -0.872,  -0.873
  -0.878,  -0.882,  -0.884,  -0.890,  -0.895,  -0.896,  -0.896,  -0.898,  -0.902,  -0.904
  -0.904,  -0.909,  -0.911,  -0.914,  -0.920,  -0.923,  -0.925,  -0.927,  -0.931,  -0.935
  -0.939,  -0.939,  -0.943,  -0.948,  -0.951,  -0.955,  -0.956,  -0.960,  -0.967,  -0.969
  -0.973,  -0.981,  -0.986,  -0.989,  -0.997,  -1.003,  -1.005,  -1.011,  -1.014,  -1.019
  -1.022,  -1.024,  -1.031,  -1.038,  -1.040,  -1.048,  -1.051,  -1.061,  -1.064,  -1.068
  -1.070,  -1.074,  -1.092,  -1.092,  -1.108,  -1.113,  -1.120,  -1.134,  -1.139,  -1.147
  -1.150,  -1.155,  -1.158,  -1.166,  -1.171,  -1.175,  -1.184,  -1.184,  -1.192,  -1.196
  -1.200,  -1.220,  -1.237,  -1.245,  -1.252,  -1.262,  -1.269,  -1.282,  -1.287,  -1.290

Comment: Thanks a lot! Very interesting, but will take me some time to digest...
Comment: I still have to go carefully through your answer, but in the meantime I read a paper about sampling random correlation matrices, and one of the methods from there can be used to do exactly what I need. I posted an answer here, you might be interested to take a look! It links to a much more detailed answer I wrote in another thread.
Comment: @amoeba: happy you found something nicely working for you! It's an interesting question, I'll come back later to this myself, perhaps improve/adapt the MatMate-procedures (and make them subroutines) according to the paper that you've worked on.
Answer: Interesting question (as always!).  How about finding a set of example matrices that exhibit the properties you desire, and then take convex combinations thereof, since if $A$ and $B$ are positive definite, then so is $\lambda A + (1-\lambda)B$.  As a bonus, no rescaling of the diagonals will be necessary, by the convexity of the operation.  By adjusting the $\lambda$ to being more concentrated towards 0 and 1 versus uniformly distributed, you could concentrate the samples on the edges of the polytope, or the interior.  (You could use a beta/Dirichlet distribution to control the concentration vs uniformity).
For example, you could let $A$ to be component-symmetric, and $B$ be toeplitz.  Of course, you can always add another class $C$, and take $\lambda_A A + \lambda_B B + \lambda_C C$ such that $\sum \lambda = 1$ and $\lambda \geq 0$, and so on.
Comment: Thanks for the suggestion, Andrew, but of course it would be nicer to have an unbiased method that does not need to start with some predefined $A$ and $B$... In the comments to my original question @ssdecontrol referred to a paper describing algorithms to sample correlation matrices uniformly (in a certain precise sense), or biased towards identity matrix, but I cannot find a way yet to sample them biased *away* from identity... I have also found a couple of old threads here asking almost the same question, maybe you will be interested, see my update.
Comment: Ah, but from such an algorithm, and a suitable diversity in the  "vertices" (that is, matrices) that define your polytope of positive-definite correlation matrices, you can use rejection sampling to get whatever distribution of eigenvalues, uniformity of entries, etc, that you desire.  However, it's not clear to me what a good basis would be.  Sounds like a question for someone who has studied abstract algebra more recently than I.
Comment: Hi again, I read a paper about sampling random correlation matrices, and one of the methods from there can be used to do exactly what I need. I posted an answer here, you might be interested to take a look! It links to a much more detailed answer I wrote in another thread.
Answer: R has a package (clusterGeneration) that implements the method in:

Joe, H. (2006) Generating Random Correlation Matrices Based on Partial Correlations. Journal of Multivariate Analysis, 97, 2177--2189.

Example:
> (cormat10 = clusterGeneration::rcorrmatrix(10, alphad = 1/100000000000000))
        [,1]   [,2]    [,3]     [,4]     [,5]   [,6]   [,7]    [,8]     [,9]   [,10]
 [1,]  1.000  0.344 -0.1406 -0.65786 -0.19411  0.246  0.688 -0.6146  0.36971 -0.1052
 [2,]  0.344  1.000 -0.4256 -0.35512  0.15973  0.192  0.340 -0.4907 -0.30539 -0.6104
 [3,] -0.141 -0.426  1.0000  0.01775 -0.61507 -0.485 -0.273  0.3492 -0.30284  0.1647
 [4,] -0.658 -0.355  0.0178  1.00000  0.00528 -0.335 -0.124  0.5256 -0.00583 -0.0737
 [5,] -0.194  0.160 -0.6151  0.00528  1.00000  0.273 -0.350 -0.0785  0.08285  0.0985
 [6,]  0.246  0.192 -0.4847 -0.33531  0.27342  1.000  0.278 -0.2220 -0.11010  0.0720
 [7,]  0.688  0.340 -0.2734 -0.12363 -0.34972  0.278  1.000 -0.6409  0.40314 -0.2800
 [8,] -0.615 -0.491  0.3492  0.52557 -0.07852 -0.222 -0.641  1.0000 -0.50796  0.1461
 [9,]  0.370 -0.305 -0.3028 -0.00583  0.08285 -0.110  0.403 -0.5080  1.00000  0.3219
[10,] -0.105 -0.610  0.1647 -0.07373  0.09847  0.072 -0.280  0.1461  0.32185  1.0000
> cormat10[lower.tri(cormat10)] %>% psych::describe()
   vars  n  mean   sd median trimmed mad   min  max range skew kurtosis   se
X1    1 45 -0.07 0.35  -0.08   -0.07 0.4 -0.66 0.69  1.35 0.03       -1 0.05

Unfortunately, it doesn't seem possible to simulate correlations that follow a uniform-ish distribution with this. It seems make stronger correlations when alphad is set to very small values, but even at 1/100000000000000, the range of correlations would only go up to about 1.40.
Nonetheless, I hope this might be of some use to someone.


Como fazer uma função aguardar a resposta de outra que está dentro da primeira

Question: Tenho minha função ABAS que só pode dar o retorno quando a função confirmacao dentro dela retornar algum valor.
Poderiam me auxiliar estou com dificuldades.
ABAS[submenu]["fn_afteredit_validar_leitores"] = function(ob)
{
    var msg = "";
    if(ob.value == true)
        var msg = "Deseja incluir o leitor no nível de acesso? \r\nObs.: Será realizado o envio de lista.";
    if(ob.value == false)
        var msg = "Deseja retirar o leitor do nível de acesso? \r\nObs.: Será realizado o envio de lista.";
    confirmacao(msg, function (bt)
    {
        if(bt.answer == true)
            return true;
        else
            return false;
    });
};

Comment: não entendi exatamente o que deseja fazer, um `callback` talvez resolva, mas, não entendi o que deseja fazer!?
Comment: Poderia dar uma explicação mais detalhada?
Answer: Eu uso assim
    confirmacao(msg, function (err, bt) {
        if(err) {
         console.log('erro');
        } else {
          if(bt.answer == true)
            return true;
          else
            return false;
        }
    });

Comment: Eu acho que isso não responde, fora que sua `bt.answer` retornar um `boleano` pra que então fazer uma estrutura de decisão não está redundante?


Darken Javadoc Background Colour

Question: I'm currently using a customised dark theme for Eclipse including the UI and editor. The only problem is that the Javadoc and scrollbars aren't darkened. I don't mind having normal scrollbars, and I think that's to do with the OS theme. Is it possible to change the background colour of the Javadoc and still have it work in Eclipse?
Answer: Perhaps adjusting preferences in Eclipse manually will solve your problem?
Try opening the preferences dialog in Eclipse and navigate to General -> Appearance -> Colors and fonts. In the Colors and fonts section navigate to Java -> Javadoc view background. Then just edit it. :)
Comment: Wow, I didn't even notice that after spending an hour configuring the theme. Thank you for that!

Darken Javadoc Background Colour


Can a Feathers VideoPlayer be scaled?

Question: It would appear that the width and height of the Feathers VideoPlayer component can be set.
But when I load a video source the size always reverts to the size of the native video.
Any ideas how to fix this?
In this example the native video is 640x390 and the player is displayed at that size.
_videoPlayer = new VideoPlayer();
_videoPlayer.setSize( 640*2, 390*2 );
_videoPlayer.validate();

addChild( _videoPlayer);
_videoPlayer.videoSource = "http://www.test.com/test.mp4";
_imageLoader = new ImageLoader();
_videoPlayer.addChild( _imageLoader );
_videoPlayer.addEventListener( Event.READY, videoPlayer_readyHandler );
_videoPlayer.addEventListener( FeathersEventType.CLEAR, videoPlayer_clearHandler );

Thanks,
Mark
Answer: Setting the imageLoader size seems to be the answer.
_videoPlayer = new VideoPlayer();
_videoPlayer.setSize( 640*Main._starlingScale, 390*Main._starlingScale );
_videoPlayer.validate();

_containerSprite.addChild( _videoPlayer);
_videoPlayer.videoSource = "http://www.test.com/test.mp4";
_imageLoader = new ImageLoader();
_imageLoader.width = 640;
_imageLoader.height = 390;
_videoPlayer.addChild( _imageLoader );

//640x390 is the size of the starling display area.
Answer: By default, the VideoPlayer does not have a layout, so even if you set the width and height properties, it will do nothing to constrain the size of any children that you add, such as the ImageLoader that displays the video texture. The VideoPlayer is actually staying at its fixed size, but the ImageLoader sees that it doesn't have an explicit width/height, so it resizes itself to render the video texture at the video's native size.
You should use VideoPlayer with a layout. For instance, you might use VerticalLayout, like this:
_videoPlayer.layout = new VerticalLayout();

And then, you can pass VerticalLayoutData to the ImageLoader, and set the percentWidth and percentHeight values to 100 to have it fill the entire bounds of the VideoPlayer:
_imageLoader.layoutData = new VerticalLayoutData(100, 100);

Can a Feathers VideoPlayer be scaled?


Onde devo instanciar os objetos?

Question: Em POO há certo grau de teoria e de boas práticas para escrever as classes e as interfaces.
Elas ficam dentro de pastas com nomenclaturas consagradas. O nome da classe deve ser idêntico ao nome do arquivo. A classe deve ser precedida pelo comando package que coincide com a estrutura de pastas da própria classe.
Mas parece que não há teoria sobre como os objetos devem ser escritos e arquitetados.
É isso mesmo? Fica a gosto do programador?
Answer:
Elas ficam dentro de pastas com nomenclaturas consagradas. O nome da classe deve ser idêntico ao nome do arquivo.

Não, programação orientada a objeto nada tem a ver com isso, e não prega qualquer coisa nesse sentido. É um mito que deve ter lido em algum lugar escrito por alguém que não entende do assunto, ou então interpretou algo equivocadamente.

A classe deve ser precedida pelo comando package que coincide com a estrutura de pastas da própria classe et cetera.

Talvez esteja falando de Java, mas isso é específico da tecnologia.

Mas parece que não há teoria sobre como os objetos devem ser escritos e arquitetados.
É isso mesmo? Fica a gosto do programador?

Não é gosto do programador, é necessidade. O programador deve entender o problema como um todo e montar o que é necessário do jeito que é necessário. E isso só é possível com muita experiência. Que só se adquire errando, fazendo vários projetos, participando de equipes com gente bem capacitada (o que é raro), vendo os erros dos outros e melhorando o seu entendimento. Tem que sempre estar com a cabeça aberta pra ver coisas novas. O que é um pouco problemático hoje em dia porque as pessoas querem tudo rápido, pulam etapas e não adquirem a base necessária para pensar por si própria. Aí ela procura fórmulas mágicas para parecer que está fazendo certo.
Tem um monte de confusão de conceitos na pergunta. Objetos não são escritos, classes são escritas. Arquitetura é algo muito mais amplo que isto.
Objetos devem ser instanciados onde eles são necessários, nem antes, e obviamente nunca depois. Espero que entenda do que está falando, caso contrário, volte algumas casas no tabuleiro para aprender esses conceitos antes. Não use nomes de coisas que não sabe do que se trata. O conhecimento se constrói um conceito bem entendido de cada vez e nunca pula para um conceito que exige o entendimento de suas dependências.

Onde devo instanciar os objetos?


PHP POST not working correctly

Question: When I submit my form, I get sent to the correct file as specified in the action attribute of my form, but the PHP in the file isn't printing the variables at all... I've combed through posts of other people having the same issue, but none of their solutions fix my problem. I've stripped my code down to a simple, textbox, button, and a php file that's supposed to print the textbox value.
If it matters, I'm running this locally in chrome, not using any servers or websites yet, I'd like to get my code working locally before I upload to my server.
HTML

<form action="Submit.php" method="post">
  <input type="text" placeholder="First Name" name="firstName" id="firstName" required>

  <input type="submit" name="submitted" value="Submit">
</form>

PHP

    Name <?php echo $_POST["firstName"]; ?>

All I get when I click the button is a white page with "Name" printed.
Thanks!
Comment: You need a server to execute PHP. If you just access it as a local file, you just get the contents of the script, it won't be executed.
Comment: @Barmar I beg to differ. The question that's marked as a duplicate does mention to install a web server as well as having to point to localhost and not as a file in a web browser.
Comment: It was suggested a week or so ago, and in another question same as this, that a canonical question be made. @Barmar You and I know just as well, that this type of question gets asked more often than none. I think it would be suitable to do one up, or if one exists, to use that for any future questions such as this. A better duplicate could have been used, I agree, however and again; this type of question gets asked ever so often and will keep on getting asked.
Comment: What do you see if you `View Source`?
Comment: Exactly what I have under PHP shows up when I click view source
Comment: *not using any servers or websites yet* I hope you have a local server. Also please show us your **full** code. What are the file names (+extension)? And are the files in the same dir? Also if you use a local server do you have the files in the server dir + do you call the scripts over the server?
Comment: @Barmar that would be my problem then, I didn't know that! If you post an answer, I'll mark it best.
Comment: @Nathan Is my comment invisible or what? http://stackoverflow.com/questions/28467614/php-post-not-working-correctly#comment45260738_28467614
Comment: possible duplicate of [PHP code is not being executed (i can see it on source code of page)](http://stackoverflow.com/questions/5121495/php-code-is-not-being-executed-i-can-see-it-on-source-code-of-page)
Comment: @Rizier123 That's not a duplicate. In that question he's running Apache, it just wasn't configured properly for PHP.
Comment: The second paragraph of the question says "Apache is running properly". I don't see any answer that says to install a webserver. The accepted answer starts with "Make sure that PHP is installed" and then goes on to explain how to configure Apache to use it.
Answer: Running scripts in response to HTTP requests is something that's done by the webserver. If you just use local files, the script will simply be loaded into the browser as a text or HTML file, it won't be executed. You can't do form processing like this.
You need to run a local server, then access the form as http://localhost/form.html

PHP POST not working correctly


How to see which workstation on our network went to a certain site

Question: We had someone order a laptop from Staples.com with one of our company credit cards on Sept 25.  It showed up at our office on the 27th, nobody knew who ordered it, and then it was gone... Someone ordered and stole it.
Is there anyway I can see who visited staples.com on Sept 25, based on DNS or traffic logs of any kind?
We are a fairly small company.  We have a single active directory server which handles DHCP and DNS (Windows 2008 R2).
We have a Safe@Office 500P router.
I could also UNC to all individual workstations, if there is any sort of local DNS log on the machine itself.
Based on my preliminary checks, it looks like we're not logging anything that can help me track this down.  I was hoping someone could suggest something to prove me wrong, though :).
Thanks!
EDIT: Just some additional information I forgot to mention: we have talked to Staples.com and we got the IP Address.  It was ordered from our office.
Also, the Safe@Office 500P router does seem to do some logging, but it only seems to keep today's logs (strange right?).  We have setup a Syslog Server for it to send logs to, but it was done after-the-fact.
We found out the email address of the person who ordered it and it is a GMail.com address.  When I put that in to a Contacts List in Gmail, it will show me the first and last name which is "Rob Lowe".  There is no Rob Lowe working here.
Comment: Rob Lowe is an actor from the West Wing and Parks and Rec. I doubt he stole a laptop from you :)
Comment: (Off Topic - Apologies) I have to admit that I like Rob Lowe on Parks and Recreation. I had to watch it last night to get over the gut punch from Breaking Bad last Sunday.
Comment: @MDMarra - saying that Rob Lowe is an actor is stretching things a bit... Bryan Cranston, that's an actor.
Comment: Are you serious? Sam Seaborn was the best character on West Wing until he left! Seaborn for president!
Answer: If you're not proactively logging this, there's likely nothing useful short of going to every PC and looking at the browsing history and hoping they haven't already erased it.
How do you know they did it from on your network anyway? Chances are they would do something like this from an off-site computer anyway.
Comment: The ISP would likely only have the source IP as your public IP -- your local DNS server is likely the one that fielded the request from the offender's system.
Comment: It's a long shot, but you could also ask Staples if they can provide a user agent from their web logs -- this might narrow it down if your machines aren't totally homogenous and your router doesn't proxy this info.
Comment: Thanks MDMarra, we have talked to some folks at Staples.com and they gave us the IP Address of the person who ordered it.  It is definitely our office IP.  It was ordered by a gmail.com address.  If you put the address into the Gmail.com Contacts list, it will show the first and last name of the person.  The first/last name is "Row Lowe".
Comment: I'm assuming DNS logging on the server was never turned on?  Info on how to turn it on/check current settings here: https://support.appriver.com/KB/a669/enable-dns-request-logging-for-windows-20032008.aspx
Comment: Shoot, you are correct.  That is not enabled...  Is it likely that an ISP would log DNS?  I may check with them...
Comment: That's a good idea.  I'll have my coworker who is in contact with Staples ask that question.  Immediately after posting my last comment, I realized the ISP DNS Log wouldn't help because it would be our external IP. Argh


Preservation of exception cause when redelivering failed activemq jms messages processed by Mule ESB

Question: I have built several Mule processes that consume messages from jms queues (ActiveMQ). Whenever a Mule component throws an exception, the transaction that consumes the messages rollback and the message gets redelivered to the original queue. After a few tries, it will be sent to a dead letter queue (DLQ.queuName).
We have this working OK, but we are missing the exception thrown, either the first one or the last one, we don't care (it'll probably be the same). This is something that can be done on other brokers (like WebLogic JMS), but I've been struggling with this for a while to no avail.
Does anybody know if this is something that can be configured or do I need to build a specific Mule exception handler or policy for ActiveMQ.
TIA,
Martin
Answer: That exception is lost in ActiveMQ at the moment (don't know about Mule) but it is reported to the log as error.
It would make a good enhancement, remembering the string form of the exception in the ActiveMQConsumer and passing it back to the broker with the poison Ack that forces it to go to
the DLQ. In that way, it could be remembered as a message property in the resulting DLQ message.
How would you like to handle the exception, have it reported to a connection exception listener or have it recorded in the DLQ message?
Comment: Hi, I wanted to preserve it as custom header/property of the DLQ message. Is there some policy interface that I can implement to treat the message before it is sent to the DLQ?
Comment: Using a broker plugin, you can implement org.apache.activemq.broker.Broker#sendToDeadLetterQueue and make some modifications. There is an existing variant that discards messages, for the source is linked at the bottom of the refernece page. see: http://activemq.apache.org/message-redelivery-and-dlq-handling.html#MessageRedeliveryandDLQHandling-TheDiscardingDLQPlugin
Comment: I'll try that. So changes to the message (basically the new property) won't be lost when the tx is rollbacked? Do I need to store the transaction somewhere to later get it here or is it available somewhere? Thanks a lot. I have seen this plugin before but I thought that there might be something more straightforward.
Comment: Have implemented the feature, the exception is now captured in a message property of the DLQ message, this will be in the upcoming ActiveMQ 5.5. Details in https://issues.apache.org/jira/browse/AMQ-3236


Javascript modularising large objects

Question: I've got an object defined as a function, and it's got a lot of methods and properties in it, what are some ways to modularise this to make it more manageable?  It is possible to put functions inside the object into external files?
Edit:
Someone mentioned in one answer to include other files, but this would have an unweildy HTML page, IE:

....

Is there also any way to structure the code so only one reference is required the the javascript file?
Comment: You should generally try to have as few external scripts as possible.
Comment: What exactly "managing problems" do you have with your current code?
Comment: I have a object with a lot of methods in it, I would much prefer to have the functions split into seperate files into groups so they are a lot easier to manage than one huge file.
Comment: Is all the code needed on each web-page, or are you trying to split the code up into multiple files so that you can serve only the required functions to each web-page? btw, if you don't prefix your comment with "@Šime", I won't get a notification about your response. I believe "@Sime" works too in my case.
Answer: var myobj = {
  somefunc: function() { /* function code */ },
  prop: "some string",
  // etc.
};

and in another js file included after the above code
myobj.anotherfunc = function () { /* function code */ };

Comment: @Tom That issue has surely already been addressed on SO numerous times. Try searching for the solution.
Comment: Thanks, +1, but this would require the inclusion of a lot of script references on the base HTML page right?  Anyway to design the data to avoid that issue?  Ideally it would be nice to just include one file.
Comment: @Tom No, I don't see how that would be related.
Comment: How do you include another js file from a .js file?
Answer: You may want to check out RequireJS. It lets you define functions, modules, dependencies and load external Javascript files from Javascript (possibly async).
Answer: you could use a server side script to handle your javascripts, with jsmin you could even speed up the process (if cached).
i'm thinking of a script somthing like this:
    scripts.php
<?php

$files = array('script1.js', 'script2.js');
$output = '';

foreach($files as $file){
    $output .= file_get_contents($file);
}

header("Content-type: application/x-javascript");
echo $output;

?>

your html

you could also just include all js files inside a folder, just generate the array via folder scanning or something, i think you get the idea!
greetings Nexum
Answer: You could try this:
http://www.cryer.co.uk/resources/javascript/script17_include_js_from_js.htm
...but best practices are to just include the files you want in the main html page

Javascript modularising large objects


Sine model of height as a function of distance

Question:
I actually assumed the sine model to be:
$$y=A\sin(B(x-C))+D$$
We have: $$A=\frac{64-0}{2}=32$$
Now from the data we see that:
$$23=32\sin(40B-BC)+D$$
and $$23=32\sin(160B-BC)+D$$
$\implies$
$$160B-BC=2\pi+40B-BC$$
Which gives $B=\frac{\pi}{60}$.
But the book answer is $B=\frac{\pi}{100}$.
Am i missing something?
Answer: If $\sin (\alpha) = \sin (\beta)$, we can't conclude that $\alpha = 2\pi + \beta$.
The general solution is $\alpha = (2n+1)\pi - \beta $ or $\alpha = 2n \pi + \beta$
Now, the minimum is attained at $0$ and the maximum is attained at $100$.  We conclude that $C$ (horizontal shifting) is $50$ and $D=A=32$.
Now, let $x=100$, $64=32\sin(B(x-50))+32$,
$$\sin(50B)=1$$
We let $50B=\frac{\pi}{2}$, hence $B=\frac{\pi}{100}$.
Remark: Let's evaluate $40B-BC=\frac{\pi}{100}\left( 40-50\right)=-\frac{\pi}{10}$. Also $160B-BC=\frac{\pi}{100}\left( 160-50\right)=\frac{11\pi}{10}$, when we add them together, we get an odd multiple of $\pi$.
Comment: yes, that is right.
Comment: So is $C$ average of $x$ coordinates where the Max and Min of $y$ occurs?


Como ler arquivo .sql com Java

Question: Tenho uma consulta complexa em MySQL com sub-select e joins.
Gostaria de ler este script através de um arquivo .sql utilizando a linguagem do JPA/Hibernate (JPQL ou HQL), por exemplo getSession().createQuery(arquivo.sql);.
Ou outra forma que me permita ler o aquivo ser ter que ser chumbado o script no repositório. Isso é possível?
Answer: String sqlString = path_to_file;
String sql = IOUtils.toString(new FileInputStream(sqlString), "UTF-8");
getSession().createSQLQuery(sql);

Aproveitei e apliquei tranformers do Hibernate para tratar a informação e transformar num Dto.

Como ler arquivo .sql com Java


Kernel and Range of a linear transformation

Question: So the question is let $T:M_{2\times2}\to\Bbb R$ be defined by
$T(A) = \operatorname{tr}(A)$.
Find bases for the kernel and range of the linear transformation $T$.
Could someone explain how to solve this as I don't quite understand what the question is asking
Answer: Note that $T$ is surjective since for $a\in\Bbb R$ we have $T(A)=a$ where
$$
A=\begin{bmatrix}a & 0\\ 0 & 0\end{bmatrix}
$$
Of course, this implies $\{1\}$ is a basis for $\DeclareMathOperator{Image}{Image}\Image T$.
The Rank-Nullity theorem states
$$
\dim\ker T+\dim\Image T=\dim M_{2\times 2}
$$
Since $\Image T=\Bbb R$ and since
\begin{align*}
\dim\Bbb R &= 1 & \dim M_{2\times 2}&=4
\end{align*}
it follows that
$$
\dim\ker T=4-1=3
$$
So, to find a basis for $\ker T$, it suffices to find three linearly independent matrices in the kernel of $T$. But it can easily be checked that
\begin{align*}
\begin{bmatrix}
1 & 0 \\ 0 & -1
\end{bmatrix}
&&
\begin{bmatrix}
0 & 1 \\ 0 & 0
\end{bmatrix}
&&
\begin{bmatrix}
0 & 0 \\ 1 & 0
\end{bmatrix}
\end{align*}
are three such matrices.


RTTI Free em um objeto criado pelo Invoke

Question: Estou criando uma rotina para instanciar uma classe via RTTI e validar se essa classe implementa uma interface.
Tenho uma classe teste implementada da seguinte forma:
unit teste.classeTeste.uclasseTeste;
interface
uses
  validador;
type
  ClasseTeste = class(TInterfacedObject, IServico)
    private
    public
      function foo: string;
  end;
implementation
{ classe }
function ClasseTeste.foo: string;
begin
  Result := 'Teste foo';
end;
initialization
  ClasseTeste.ClassName;
end.

E tenho uma classe Validador implementada da seguinte forma:
unit validador;
interface
uses
  Rtti;
type
  IServico = interface
  ['{46C26FED-AEDD-456F-B182-DEECB780D264}']
  end;
  Svc = class
    public
      function Validar(pClassName: string): Boolean;
  end;
implementation
uses
  SysUtils;
{ Svc }
function Svc.Validar(pClassName: string): Boolean;
var
  lContext: TRttiContext;
  lType: TRttiInstanceType;
  lObj: TObject;
begin
  Result := False;
  lType := lContext.FindType(pClassName) as TRttiInstanceType;
  if lType = nil then
    raise Exception.Create('Classe solicitada não encontrada no sistema ou ClassNameInválido');
  lObj := lType.GetMethod('Create').Invoke(lType.MetaclassType, []).AsObject;
  try
    if not Supports(lObj, IServico) then
      raise Exception.Create('A classe serviço não implementa a interface IServico!');
  finally
    lObj.Free;
  end;
  Result := True;
end;
end.

A questão é: No método Validar, na linha
`lObj.Free`;

Ele estoura

"Invalid Pointer Operation".

Teoricamente tenho um TObject instanciado. Ele precisa ser liberado em algum momento. Como e quando libero ele?
Obs.: Estou usando o Delphi 2010.
Comment: Eu creio que é porque ele tá derivado de `interface`, e aí o Delphi por conta própria se livra da instância. Teste com o `ReportMemoryLeaksOnShutdown := True`, e não execute o `Free`. Se ao fechar a aplicação não mostrar uma mensagem com "ClasseTeste x1" então está acontecendo o que eu disse mesmo.
Comment: Tu passou `'ClasseTeste'` pra execução da tua procedure?
Comment: Na verdade, para o RTTI funcionar tenho que passar no namespace + a classe. A chamada ficou:  if objValidador.Validar('teste.classeTeste.uclasseTeste.ClasseTeste') then
    ShowMessage('Classe Valida!');
Comment: Desculpe, mas preciso aprender a usar esse editor ainda...
Comment: No Delphi 2010, o FastMM não vem embutido. Eu adicionei e rodei. Sem dar o Free ele não acusa nenhum "MemoryLeak".   Criei uma nova variavel (lob2), fiz o create e não dei o Free, e acusou o "MemoryLeak". Enfim, fiquei receoso pois lObj := lType.GetMethod('Create').Invoke(lType.MetaclassType, []).AsObject; no meu ver, me devolveria um TObject e não uma interface.
Comment: Eu realmente não sei te dar uma explicação plausível do porque pelo RTTI ele está usando gerenciamento via `interface` já que você está dizendo que quer criar um ClasseTeste e não uma IServico. Realmente, se você criar uma variável que por mais que herde uma interface, mas tu defina que a variável é ClasseTeste, não era pra ser gerenciado como interface, e aí causa uma MemoryLeak.
Comment: Ainda...no método Invoke - Invoke(lType.MetaclassType, []).AsObject - estou pedindo para ele me retornar "AsObject" :D
Answer: o problema está na linha  if not Supports(lObj, IServico) then
quando a Função Supports é chamada ela libera o lObj e como você chama o free sempre
finally
   lObj.Free;
end;

ele tenta liberar novamente, ocasionando o erro, faça o teste, implemente o método
destructor Destroy; override;

na sua classe ClasseTeste e verá que ele chama duas vezes o Free.
Comment: Desculpe, mas não consegui enxergar isso. O código da funcao Supports, em SysUtils, `Result := (Instance <> nil) and (Instance.QueryInterface(IID, Intf) = 0);`
Comment: debugando aqui seu exemplo, e implementando o destroy percebi isso


Learning of hyperparameters for Gaussian process

Question: Following the paper Practical Bayesian Optimization of Machine
Learning Algorithms. Link
It's not clear to me as to how the hyper-parameters (different from the target hyperparameters for some other method) for the Gaussian Process (GP) is been learned.
The paper mentions it as follows (Pg.4 under section 3.1):

After choosing the form of the covariance, we must also manage the
  hyperparameters that govern its behavior (Note that these
  "hyperparameters" are distinct from those being subjected to the
  overall Bayesian optimization.), as well as that of the mean function.
  For our problems of interest, typically we would have $D + 3$ Gaussian
  process hyperparameters: $D$ length scales $\theta_{1}:D$, the covariance
  amplitude $\theta$, the observation noise ν, and a constant mean m.
  The most commonly advocated approach is to use a point estimate of
  these parameters by optimizing the marginal likelihood under the
  Gaussian process, $p(y|\{x_n\}^N_{ n=1}, \theta, ν, m) = N (y | m1, \sum_{\theta} + νI)$,
  where $y = [y1, y2, · · · , y_{N}]^{T}$, and
  $\sum_{\theta}$ is the covariance matrix resulting from the N input
  points under the hyperparameters $\theta$.

Can anyone elaborate or explain in layman terms on the suggested approach of using point estimate ( and how it is done practically) mentioned in the above description?
Answer: They claim to, "use a point estimate of these parameters by optimizing the marginal likelihood under the Gaussian process."
Now the expression given indicates they believe the marginal likelihood has a Gaussian distribution $N(y|m1,\Sigma_\theta+\nu I)$. At this point you can write out the marginal likelihood (usually on the log scale) as it has the fixed form implied by the Gaussian and then optimise with respect to the unknown parameters.
The reason people use a point estimate in these circumstances is almost always the speed and tractability of calculating the estimate. The downside is that you lose notions of uncertainty about the parameters. E.g. if say $\nu$ has a shallow maximum you might underestimate the variability of some other quantities of interest.
Comment: ' $> \sum $' was a formatting error
Comment: Trimmed to reflect this.


Is it possible to calculate the cdf of this r.v.?

Question: $X$ and $Y$ are two r.v.'s with cdfs $F_{X}$ and $F_{Y}$ respectively. $Z$ is another r.v. defined as $Z=X$ with probability $a$ and $Z=Y$ with probability $1-a$ and $P(X=Y)=0$. Then can we represent the cdf of $Z$ using $F_{X}$, $F_{Y}$ and $a$?
Comment: Have you tried anything at all? Start with $P(Z \le z)$, then condition on whether your biased coin flip is heads (with prob. $a$) or tails. This is almost a one-line proof.
Comment: I tried as: $P(Z \leq z) = P(Z \leq z, Z=X) + P(Z \leq z, Z=Y) = P(X \leq z, Z=X) + P(Y \leq z, Z=Y) $ then i do not know how to proceed. Could you explain more how to use conditioning?
Answer: $$\begin{align*}
F_Z(z)
&=P(Z\leq z)\\\\
&=a\cdot P(X\leq z)+(1-a)\cdot P(Y\leq z)\\\\
&=a\cdot F_X(z)+(1-a)\cdot F_Y(z)\\\\
\end{align*}$$
Comment: Thanks. Could you explain how do you get the second "="?
Comment: We are given the probability that $Z=X$ is $a$ and the probability that $Z=Y$ is $1-a$. I would think of it as a weighted average.
Comment: Still do not understand. Could you give more rigorous reasoning or proof?
Answer: In the comments, you write

$$ \Pr (Z \le z) = \Pr ( Z\le z, Z= X) + \Pr (Z \le z, Z=Y). $$

Notice that
\begin{align*}
\Pr(Z \le z, Z=X) &= \Pr (X \le z, Z = X) \\
&= \Pr (X \le z) \cdot \Pr(Z=X) \\
&= a F_X (z).
\end{align*}
Now do the same for the term involving $Y$.
Comment: Strictly speaking all the answers I see till now make an extra assumption of independence. (eg. second line in the answer by Theoretical Econimist). The correct answer is that the distribution of $Z$ cannot be expressed in terms of those of $X$ and $Y$ and the variable $a$ in general.
Comment: @KaviRamaMurthy Isn't the assumption of independence implicit in the definition of $Z$?
Comment: Well, it depends where the OP encountered this question. In an advanced course independence assumptions have to be stated precisely.
Comment: @KaviRamaMurthy Can you give an example of (distinct) random variables $X$ and $Y$ such that I can construct $Z$ in the way described (that is, $Z=X$ with probability $a$, and $Z=Y$ otherwise) where the independence assumption would not hold?
Comment: How about this: $Y=2X$. Then $X$ and $Y$ are not independent. So  $P(X \leq z, Z=X) \neq P(X \leq z) \cdot P(Z=X) $?
Comment: @Peter $X$ and $Y$ do not have to be independent for the events $\{X\le z\}$ and $\{Z=X\}$ to be independent.


Meaning of the '.isRequired' when defining static PropTypes in React-Native

Question: static propTypes = {
  // Value to display
  value: PropTypes.string.isRequired,
  // Wheter this values was chosen by user or not
  isChosen: PropTypes.bool.isRequired,
  // Gets called when user choses this value
  onChoose: PropTypes.func.isRequired,
}

Why Should I add '.isRequired' in when defining the above PropTypes? I am a react-native beginer
Answer: If you define the PropType with isRequired, React will warn you when you use that component without passing that prop.
// -> Warning: onChoose is required but not provided
<Component value={val} isChosen={true} />

If you do not define it, the property is considered optional and no warning is displayed if you don't pass the prop.
// -> No warnings, since the props are not required.
<Component />

Answer: That's one of the React feature called built-in type-checking abilities. The .isRequired means that props element defined is required to be passed from parent component. If you don't pass it, React will give you a warning message.
Ref: https://facebook.github.io/react/docs/typechecking-with-proptypes.html


How can I retrieve current X-value in gRaphael linechart hoverColumn function?

Question: Below I've posted some code and comments for making a simple gRaphael linechart with tag hovereffects (the included script files can be found at http://raphaeljs.com and http://g.raphaeljs.com). The code works as it should (and can thus be a good example for others starting out with gRaphael linecharts), but I have a question/request regarding the text argument of the tag function:
As is, the tags will display the Y values for each point (this.values[i]) at the current column, but I would like to have both the X values and the Y values displayed (X, Y). I'm sure it's pretty simple, but so far I've not been able to figure out how to do it. Adding the comma and a space is no problem, that's just ', ' + this.values[i], but I can't figure out how to address the X values. this.attr("x") is the closest I've come so far, but that's the X coordinate on the paper, not the X value on the chart's X axis ;-)
Can anyone help me out please?
// make tags at every point on the current column
for (var i = 0; i < this.y.length; i++) {
  this.tags.push(
  // make tag (x, y, text, degree, radius)
paper.tag(this.x, this.y[i], this.values[i], 0, 4).insertBefore(this).attr([{ fill: "#ffffff" }, { fill: this.symbols[i].attr("fill") }])
);

<title></title>

Comment: Do you want the coordinates of the dots on the line chart? If yes, I can post a solution.
Comment: @Neha Choudhary Yes, I want both the X and the Y coordinate of the current dot (column) to be displayed in a tag when hoovering the line chart. My code above displays only the Y coordinate (this.values[i]). Thanks
Comment: Did you try something like `linechart.symbols.cx` and `linechart.symbols.cy`?
Comment: @Neha Choudhary I've tried now, but neither of those did work. The tag just shows undefined. This is by the way not that important to me anymore, so don't spend much time figuring this out. Thanks
Answer: Try to listing object in chrome (CTRL+SHIFT+I->Console). For example - get cy and y for every label in piechart legend.
var pieChart = paper.piechart(w, h, rad, values, {legend: legend, legendpos: "south", minPercent: 0.1, legendothers: "Others"});

for( var i = 0; i < pieChart.labels.length; i++ ) {
 var CY=pieChart.labels[i][0][0].cy.animVal.value;
 var Y=pieChart.labels[i][1][0].y.animVal[0].value;
}


Regression coefficients without intercept

Question: Could someone recommend a link or help me out here: where can I find the formula for the regression without an intercept, and how is it deriveed differently than the formula with the intercept? (matrix form)
Answer: It's the same formula you use in regression with an intercept: $\hat{\beta}=\left(X^TX\right)^{-1}X^Ty$. Just make sure there isn't a column of ones for the intercept. It's the same derivation as the formula with an intercept.
Here's a link to some notes that derive regression coefficients: Statistics 512: Applied Linear Models - Chapter 5: Linear Regression in Matrix Form [PDF].
Comment: Yup. The model without an intercept is actually a special case (nested) of the model with an intercept. Without the intercept, we are actually assuming the intercept is zero.
Comment: The residual sums of squares will not be the same unless the regression model with an intercept fits the intercept to zero.
Comment: I apologize. I think I read your initial comment incorrectly. The standard F Test for choosing between two models, one of which is nested in the other, is valid. It's no different because you aren't including an intercept.
Comment: Thank you. If you don't mind me asking another question - if I'm looking to compare variances of the same regression with and without an intercept by looking at their ratio, is there a difference in formulas?
Comment: But if I need to take a ratio of variances, doesn't that mean their formula will just be the same?
Comment: The pdf link was broken. I've updated it. Please check if the update link points to the same resources you intended it to.


Get field atributes / metadata based on the field name

Question: This is a related issue to 2sxc Get list of fields of an entity
I still have the entity entity "Cars" with two fields "Brand" and "Model".

Each field has a title, placeholder, notes, dropdown values, etc.

I need to show a details view for a single car, selected with
var thisCar = AsList(App.Data["Cars"]).Where(e => e.EntityId == 12345).First();

I now need to show some attributes/metadata of each fields. Let's say this:

EntityId: @thisCar.EntityId

(Brand label/name): @thisCar.Brand (placeholder for the Brand field)(dropdown values for the Brand field)

(Model label/name): @thisCar.Model (placeholder for the Model field)(dropdown values for the Model field)

In this case, I do not need to loop through all the attributes, but I need to access the field label, default value, placeholder, notes, dropdown values, etc, based on the field name.
How can I achieve this?
Answer: This is quite difficult to do, as the internals are not meant to be used for these kinds of scenarios.
But if you really want to do it, check out https://2sxc.org/dnn-tutorials/en/razor/tut/data-contenttype


HTTPS warning for Sites in Salesforce

Question: I am getting following error on a browser when i open Secure(HTTPS) Site URL for my sandbox org,
You attempted to reach **.cs9.force.com, but instead you actually reached a server identifying itself as .cs9.force.com. This may be caused by a misconfiguration on the server or by something more serious
The problem arises because i am using Sites for exposing WS and the HTTPS warning gives error on client side while interacting with WS.
How can i configure my org to resolve HTTPS warning message?
Answer: Because you are using sites you are able to use HTTP. There is no way to change the HTTPS Site settings. And there is no way to upload your own SSL-Certificate.
Answer: Dont know if your problem is resolved by now or not. But you need to configure your WS to accept wildcard SSL certificates. I am currently facing this issue and following link may be helpful to you.
I can't use this solution because of shared weblogic server across many applications. If you have found any solution to above, can you please comment here.

HTTPS warning for Sites in Salesforce


Removable volume dismount: drive letter remains reserved

Question: On a Windows 2008 Server regular users are allowed to mount a TrueCrypt volume running a batch as administrator. The volume is dismounted automatically on certain conditions by another batch file ran by Task Scheduler.
The volume gets inaccessible every time, but about one in four cases the drive letter remains there so next time the mount script fails.
The weird thing is that it seems random which users see this phantom drive. Sometimes everyone, sometimes only one and not even the administrator (but even then, TC ran as Admin sees it and says 'volume already mounted').
The drive letter cannot be accessed (usual 'volume may be moved' error message). Disk Manager and Mountvol does not see the drive. The only way I was able to get rid of it was rebooting the server.
Start command used by other users (Yes, I know /savecred is unsafe, but it's (hopefully) irrelevant for the problem):
<code>C:\Windows\System32\runas.exe /savecred /user:Administrator
   "cmd /c c:\users\Administrator\ws\startserver.bat"
</code>
startserver.bat:
<code>"c:\Program Files\TrueCrypt\TrueCrypt.exe" /v c:\Users\Administrator\ws\ws.dat /lz /q /m rm
</code>
Dismount batch ran as Administrator by Task Scheduler (before that, a MySQL and a Tomcat server running from the TC drive is stopped and there is a 30 seconds pause, there is no indication that any of these would fail to stop):
<code>"c:\Program Files\TrueCrypt\TrueCrypt.exe" /f /dz /w /q /s
</code>
Any tips on how to investigate this?
Answer: You may try to use diskpart to free the drive letter.
Add this line to your dismount script after truecrypt's dismount:
<code>(echo select vol X & echo remove) | diskpart > nul
</code>
Where X is your volume letter.
Comment: No luck. It says 'diskpart cannot be used right now'. Trying select vol Z from diskpart's command line results in volume does not exist error.
Comment: How about this, then: do not assign letter to your drive at mount, and use diskpart to assign/remove letter.


Evaluate $\int_{|z|=r}^{}\frac{\log\ z}{z^2+1}dz$ where $r>0$

Question: I tried to evaluate $$\int_{|z|=r}^{}\frac{\log\ z}{z^2+1}dz$$ when $r>0$. Clearly $\log z$ is not continuous at $z=-r$. So is this integral meaningful then? Since the function is continuous and bounded on  the given path except only at this point, I am thinking it should be possible to actually evaluate this. Hope someone could help me out with this confusion and help to evaluate this integral. Thanks
Answer: A branch of the logarithm is defined in a region, $G\subseteq\dot{\mathbb{C}}$, if and only if for every cycle, $\Gamma$, contained in $G$, $n(\Gamma, 0) = 0$, where $n(\Gamma,0)$ is the winding number, which represents the number of times the curve goes around $0$.
In your particular case, whichever region containing the curve you're integrating over doesn't satisfy this condition, so you can't define a branch of the logarithm.
Basically, you can't find any branch of the logarithm which is well defined in a region containing your curve.
Note: I've seen in some places the notation $\operatorname{Log}$ used to refer to the principal branch. If your adhering to this convention, then everything is ok. If you aren't then it is possible to define a branch of the logarithm in the negative real axis, as long as the condition I mentioned is satisfied.
Comment: Yes I am referring to the principal branch and know that this is integrable but I don't how that's possible
Comment: Well, as I said in my answer, this is not defined in your whole domain. To integrate the principal branch you should remove the point $-r$ from the integration. What you can do is calculate the principal value:
$$
PV-\int_{|z|=r} \frac{\operatorname{Log}z}{z^2+1}dz = \lim_{\epsilon\to \pi^-}\int_{-\epsilon}^{\epsilon}\frac{\operatorname{Log}(re^{it})}{r^2e^{2it}+1}(ire^{it})dt
$$
Comment: I see. Never seen this in any text I have read thanks alot
Comment: This can be helpful: http://en.wikipedia.org/wiki/Cauchy_principal_value


Redunduncy of Pasch's Axiom of Hilbert's Foundations of Geometry

Question: I am baffled by Hilbert's Foundations of Geometry's Axioms because the relations are not defined. They are only described, not defined. For example, the notions of 'betweeness' or 'congruency' are any relations whatsoever, provided only that they satisfy certain axioms with respect to lines and points.
I was closely examining those axioms that define the notions, waiting until I somehow persuade myself that there is only one possible interpretation as to the definitions of the relations of 'betweeness', 'congruency', 'containing', etc, but I ended up discovering that, for 'betweeness' in particular, there is another interpretation that is possible.
What I mean is, according to me, if aliens came down from the sky to examine Hilbert's Foundation of Geometry, they would not understand what exactly is meant by 'betweeness', because there is more than one interpretation that satisfies the axioms of order.
For example, let me interpret 'between' informally: that for any three points on the same line, the point at the one one end which is farther than the one at the other end from the point in between them, is the one we'll call 'between' the other two, and only that point. If the points happen to be the same distance apart from the point between them, the point in the middle will be the one 'between', as in our original understanding, and only that point. Let's assume this is what 'between' means to aliens.
Let 'between' be the new interpretation, and 'betweeness' be the original interpretation.
There is nothing wrong with our notion of 'between', in fact the original notion, of 'betweeness', is informal: that point which is bound by the other two, for example. The axioms are made to explicitly state what the properties of betweeness are, so that we aren't working with informal relations in geometry. The goal here is to check whether the axioms of betweeness are true for the alien interpretation or not. Here are the axioms...

II,1. If $A, B, C$ are points of a straight line and $B$ lies between $A$ and $C$, then $B$ lies also between
$C$ and $A$.

II,2.If $A$ and $C$ are two points of a straight line, then there exists at least one point $B$ lying
between $A$ and $C$ and at least one point $D$ so situated that $C$ lies between $A$ and $D$.

II,3.Of any three points situated on a straight line, there is always one and only one which lies
between the other two.

I am aware these are not all the axioms, but the 4th is called Hilbert's Discarded Axiom, apparantely because it was proved to be redundant(source: Wikipedia).
Also there is a 5th axiom, also known as Pasch Axiom, but I will derive it from the previous axioms down in this question, to show that it is also redundant. I am not sure why even though it is derivable from the previous axioms it is not discarded yet.
Returning to our 'between' idea, we can see that it satisfies the axioms.
Axiom II,1 directly follows from our idea of 'between' because our idea does not depend on the order. That is if $B$ is 'between' $A$ and $C$, it is also 'between' $C$ and $A$, because $B$ is simply the farthest from the other two.
Axiom II,2 is trivially true because we know we can always add a point $B$ far enough from $A$ and $C$ , so that $B$ is 'between' $A$ and $C$, and we can add a point $D$ close enough to $A$, such that $C$ is 'between' $A$ and $D$.
Axiom III,3 is true because we defined our 'between' as always exactly one point at a time: that which is farthest.
Now you can see that these are not formal proofs that the axioms follow from the idea of 'between', but it should be clear why. The goal is to show that those axioms that describe 'betweeness' can also be interpreted to describe 'between', so we can't decide what betweeness is.
Theorem 5 is not derived from Pasch's Axiom, as stated in Hilbert's Foundation of Geometry.

Theorem 5. Every straight line $a$, which lies in a plane $α$ , divides the remaining points of this plane into two regions having the following properties: Every point $A$ of the one region determines with each point $B$ of the other region a segment $AB$ containing a point of the straight line $a$. On the other hand, any two points $A,A′$ of the same region determine a segment $AA′$ containing no point of $a$

Pasch's Axiom. Let  $A,  B,  C$ be three points not lying in the same straight line and let $a$ be a straight
line lying in the plane $ABC$ and not passing through any of the points $A, B, C$.  Then,
if the straight line $a$ passes through a point of the segment $AB$, it will also pass through
either a point of the segment $BC$ or a point of the segment $AC$.

Here's the proof for Pasch's Axiom:
From the hypothesis we have that $a$ passes through a point of the segment $AB$. Therefore, the segment $AB$ contains a point from $a$.
From the hypothesis we know that $a$ does not pass through $A$ ,$B$, or $C$, therefore the points don't lie in $a$. We also have from the hypothesis that $a$ is a line lying in the plane $ABC$, and therefore by theorem 5, it divides the remaining points into two regions.
the segment $AB$ contains a point from $a$. therefore the points $A$ and $B$ are on different regions, because if they were on the same region, then from theorem 5 we would have that segment $AB$ does not contain a point from $a$, but this results in a contradiction. So $A$ and $B$ are on different regions.
By the law of excluded middle, either $C$ is in a different region from $A$ or $C$ is not in a different region from $A$. if it is the former, then segment $AC$ contains a point from $a$, by theorem 5, and therefore, $a$ passes through a point in segment $AC$ and we're done. If it is the latter, then $C$ and $A$ are on the same region. But' $C$ and $B$ can't also be on the same region, because if that was the case, $A$ and $B$ would also be on the same region, and this results in a contradiction. Therefore $C$ and $B$ are on different regions, so by theorem 5, the segment $BC$ contains a point from $a$, and therefore $a$ passes through a point in the segment $BC$. QED.
Now, my interpretation 'between' is also true for Pasch's Axiom, because I showed that the axiom is redundant. I have to admit that my interpretation does not satisfy the 4th Axiom, but the 4th axiom is redundant, so I am left even more confused. That is, if the interpretation satisfies the first 3 axioms, which satisfies Pach's Axiom, and the 4th axiom can be proven by those axioms, why doesn't my interpretation satisfy it?

So What?
You don't need to answer all of the questions, but...
Why is there another interpretation of 'betweeness' that satisfies Group II Order Axioms, and if there is only one interpretation, why doesn't the book explain how that is the case? What is the proof of the redundancy of the 4th Axiom, and simultaneously, how can the 4th Axiom be redundant when it is necessary for a single interpretation of betweeness? Why has this elementary proof of Pasch's Axiom been overlooked... No, in fact, wasn't Pasch's Axiom proven to be independent? I am surrounded by dark business and I need Sherlock Holmes to solve the problem.
Comment: The main point in Hilbert's point of view is exactly that objects and relations are not *explicitly defined*, but just described by the axioms. You can use whatever model you prefer, so long as it satisfies the axioms. Hilbert himself spoke about "beer mugs and benches" as a (not really) possible models for his axioms.
Comment: Hilbert's book is not easy reading, and he certainly did not tie up loose ends. I can recommend Hartshorne, Geometry: Euclid and Beyond. I had a bit of input to the fourth edition of M. J. Greenberg Euclidean and Non-Euclidean Geometry.  You can download a related article by Greenberg from this page: https://www.maa.org/programs/maa-awards/writing-awards/old-and-new-results-in-the-foundations-of-elementary-plane-euclidean-and-non-euclidean-geometries
Comment: Theorem 5 is equivalent to Pasch's axiom; I've seen books in which it is taken as the standard plane separation axiom in place of Pasch's axiom. Note that the first four axioms say nothing about how the betweenness relations on different lines interact, so there's no way you could get something like Theorem 5 (or Pasch) out of them.
Comment: I think the Project Gutenberg edition of Hilbert is a little corrupted (possibly it mixed up two printings, one of which included the discarded axiom and one of which didn't). It gives five linear axioms, and then says "by aid of the four linear axioms II, 1-4", but really you need all five.
Comment: I'll look into that.
Comment: Perhaps you may find something in the chapter "Ordered Geometry" in Coxeter's book Introduction To Geometry.
Answer: Contrary to what you say, Theorem 5 cannot be deduced without using Pasch's axiom.  You cite Hilbert's Foundation of Geometry, and I'm guessing you are looking at this English translation.  Unfortunately, that translation has several errors, and in particular the assertion on page 5 that the theorems rely on only axioms II,1-4 is totally wrong.  No such assertion is found in the original German and so I suspect this error (like the error discussed in A confusion about the second connection axiom of Euclidean Geometry) was introduced by the translator.
In particular, your "alien" definition of betweenness does not satisfy Theorem 5.  For instance, let $a$ be any line, and let $A$ and $B$ be two points that are very close and on the same side of $a$ (by the usual definition).  Then by the alien definition, $A$ and $B$ must be on opposite sides of $a$, since there is a point between $A$ and $B$ on $a$ (since if line $AB$ intersects $a$ at $C$, then $C$ will be the farthest away of $A, B,$ and $C$).  But now if you take a third point $C$ which is also very near $A$ and $B$, then $C$ must similarly be on the opposite side of $a$ from both $A$ and $B$.  This is impossible since there are only two sides of $a$.

As a side comment, it's a feature, not a bug, that Hilbert's axioms describe the primitive notions rather than define them.  You can think of the axioms as a set of rules that allow you to determine when a structure deserves the name "Euclidean geometry".  You must provide the definition of your structure, and then the axioms tell you whether your definition really describes a model of Euclidean geometry or not.  This approach is pervasive in modern mathematics: for instance, the axioms of a group don't tell you what the group elements or operation are; they just give conditions under which a set of elements equipped with an operation is called a "group".
Now, in the case of Hilbert's axioms, one can prove that up to isomorphism there is a unique structure which satisfies the axioms.  So, up to isomorphism, there really is only way to define a structure which follows the axioms.  But the "up to isomorphism" is crucial: the entire point of the axioms is that there are many different ways we could define what a "point" or "line" is or what "betweenness" means, and that geometry doesn't actually care exactly what definition we use.  Geometry only cares about the structural relationships between all these primitive notions, which will be preserved by an isomorphism of models of the axioms.


Showing B is not derivable from A

Question: Let's take it that under the assumption of A, ~C is true. And let's also take it that if ~C is true and if B is derivable from A, then C is true.
I want to show that B is not derivable from A. Is this possible under these assumptions? It seems like we can assume that B is derivable from A for the sake of obtaining a contradiction. So there is some proof that begins by assuming A and obtains B in a finite amount of steps. We take that derivation under which we already have A as an assumption and within it, since we know that A gives ~C, we conclude that ~C is true. Now since we also know that if ~C and if B is derivable from A, then C is true. So we get to a contradiction, and so B is not provable from A. Is this valid? Or does this only show that under the assumption that B is derivable from A, then ~A is derivable by the empty set?
Answer: Assume A $\supset$ B.  (for contradiction)
Assume ($\lnot$ C $\land$ (A $\supset$ B)) $\supset$ C. (gathered from your question)
($\lnot$ C $\land$ True) $\supset$ C.  (replacing A implies B with its truth value)
$\lnot$ C $\supset$ C.  (reducing "and True")
$\lnot$ C only implies C if $\lnot$ C is false.
We can conclude that C is true under the assumptions.  But this isn't a contradiction.  Instead, using Implication, $\lnot$ C $\supset$ C is equivalent to C $\lor$ C.  Maybe another answerer can find a contradiction but I don't.
Quoting your first paragraph, "let's also take it that if $\lnot$ C is true and if B is derivable from A, then C is true."  Let's say instead, "if $\lnot$ C is true and anything, then C is true."  It seems like you are trying to conjure up a contradiction with these C terms, then stick it onto the A and B term.  I couldn't make it work.
Is there a particular problem that inspires the question?  We should be able to work with the symbols but I am just curious.
Comment: You have that given A, ~C follows. Does this also imply that A is false, then?
Comment: I think you're right.  Sorry I missed that.  But with A false, A $\supset$ B is true (a falsehood implies anything) does that help?
Answer: I don't believe the argument you're trying to construct is valid.
As you can see below, a truth table in which $A$ is false, $B$ is true, and $C$ is true will show it is possible for your premises $A \to \neg C$ and $[\neg C \wedge (A \to B) ] \to C$ to be true and the conclusion $\neg (A \to B)$ to be false. Hence, the argument is invalid.
$
\begin{array}{llllllllllllllll}
A & \to & \neg & C & \Big| & [\neg & C & \wedge & (A & \to & B) ] & \to & C & \Big|\Big| & \neg & (A & \to & B) \\
0 & \:\color{red}{1} & 0 & 1 && \:\:0 & 1 & 0 & \:\:0 & \:1 & 1 & \:\color{red}{1} & 1 && \:\color{red}{0} & \:\:0 & \:1 & 1
\end{array}
$

Showing B is not derivable from A


How can I export a function with Moose::Exporter?

Question: I wanted to export a simple function from a base class that I use all over my sub classes without having to use $self->myfunc() all the time, just a simple func() call.
I tried doing this with the example from Moose::Exporter
but I didnt understand what
as_is     => [ 'sugar3', \&Some::Random::thing ],

was doing, as the example seems incomplete. sugar3 isnt defined anywhere, so I dunno where or how to use it. Can I call sugar3() in the subclass now? Is sugar3() some secret Moose thing?
and then the was this thing... literally
thing;

that was exported, but I have no idea what thing is doing since there is no example of it. Is this a function call?
Anyway, more to the point how do you export functions like you would normally do with Exporter, but with Moose Exporter instead, and what happens if my baseclass has 3 levels of inheritance after it, will all the sub sub classes have access to this exported function?
Comment: `$self->myfunc()` and `myfunc()` are two very different things, and you shouldn't be writing a method and then trying to export it as a sub. An exported *function* will not have access to the object - that is, it is not *object oriented*.
Comment: This isn't much of a help, but thinking about trying to mash OO inheritance and package syntax like this just gives me the heebie-jeebies. Are you sure there's not a better way to do what you're trying to do? Besides, I'm fairly certain (though I could also very easily be wrong) that a use parent 'Whatever' doesn't end up importing functions into the local package, anyway, so it doesn't sound any different than the way Exporter works.
Comment: im not trying to mash anything i just use a function call a lot to debug print and test things but I'll clean it up when im done =]. In any case I still would like to understand how this works. +1 thanks for sharing ur thoughts about this
Comment: +1 Good point. It seems like I shouldn't be trying to do this in Moose, but then why would Moose provide an exporter if we werent to export anything?
Answer: as_is => [ ... ]:
This list of function names or sub references will be exported as-is. You can identify a subroutine by reference, which is handy to re-export some other module's functions directly by reference (\&Some::Package::function).
sugar3 is the name of a sub to export.
Yes, you can call sugar3 in the subclass now if that's where you exported it to. That said, it's usually weird to export (anything but constants) to a subclass.
Yes, thing; is a sub call. Under no strict;, it could also be the same as 'thing';.
The sub classes won't have access to the sub unless it's called as a method (e.g. $o->thing; instead of thing;). It's extremely weird to export methods, though. Create a Moose::Role to give methods to a class.
Comment: @nodebunny, `$self->func()` is required for methods. I didn't say you can't export methods, I said it's extremely weird.
Comment: so is this essentially exporting -- using Roles? Is there a way not to have to type $self->func() everytime? -- not like Im going to die if I cant =P the problem is some of the older libary dont use Moose and I have to still use what they export, so why cant I do it in Moose instead?