Mixed model without fixed intercept- Numerator df for Type II and III ANOVA (lmerTest vs Proc Mixed)

Question: It comes to my attention that there is a discrepancy between lmerTest and Proc Mixed for the numerator degrees of freedom (df) for type II and III ANOVA. This only occurs for mixed model without fixed intercept.
The following is the dataset:
input9 <-structure(list(block = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 2L,
2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 4L,
4L), .Label = c("1", "2", "3", "4"), class = "factor"), a = structure(c(1L,
1L, 2L, 2L, 3L, 3L, 1L, 1L, 2L, 2L, 3L, 3L, 1L, 1L, 2L, 2L, 3L,
3L, 1L, 1L, 2L, 2L, 3L, 3L), .Label = c("1", "2", "3"), class = "factor"),
    b = structure(c(1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L,
    2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L), .Label = c("1",
    "2"), class = "factor"), y = c(56, 41, 50, 36, 39, 35, 30,
    25, 36, 28, 33, 30, 32, 24, 31, 27, 15, 19, 30, 25, 35, 30,
    17, 18)), label = "MIXED9                          ", row.names = c(NA,
-24L), class = "data.frame")

Here is the SAS code:
proc mixed data = work.input9 method = ml;
    class a b block;
    model y = a b a*b / noint solution DDFM=SATTERTHWAITE HTYPE=1,2,3;
    random block a*block;
    lsmeans a b a*b / pdiff cl;

    run;

Here is the SAS output in R:
> anova9_I_exp_t
  effect numdf             dendf            fvalue                probf
1      a     3  5.59999999999989 28.21481701385756 0.000866319611832041
2      b     1 11.99999999999997 25.85163204747710 0.000268656242407431
3    a*b     2 11.99999999999999  5.35905044510368 0.021719775545852811
> anova9_II_exp_t
  effect numdf             dendf            fvalue                probf
1      a     2  7.99999999999956  5.42609899619229 0.032427388825803982
2      b     1 11.99999999999997 25.85163204747710 0.000268656242407431
3    a*b     2 11.99999999999998  5.35905044510368 0.021719775545852839
> anova9_III_exp_t
  effect numdf             dendf            fvalue                probf
1      a     2  7.99999999999957  5.42609899619229 0.032427388825803989
2      b     1 11.99999999999997 25.85163204747711 0.000268656242407431
3    a*b     2 11.99999999999998  5.35905044510368 0.021719775545852839

Here is the R code and R output:
> results9 <- lmer(y ~ -1 + a + b + a*b + (1|block) + (1|interaction(a, block, lex.order = T)), data = input9, REML=F, control=lmerControl(optimizer="bobyqa"))
> anova9_I <- anova(results9,type="I")
> anova9_II <- anova(results9,type="II")
> anova9_III <- anova(results9,type="III")
> anova9_I
Type I Analysis of Variance Table with Satterthwaite's method
            Sum Sq        Mean Sq NumDF           DenDF  F value     Pr(>F)
a   594.2746442207 198.0915480736     3  5.600000533736 28.21482 0.00086632 ***
b   181.5000000000 181.5000000000     1 11.999999840688 25.85163 0.00026866 ***
a:b  75.2500000000  37.6250000000     2 11.999999840262  5.35905 0.02171978 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> anova9_II
Type II Analysis of Variance Table with Satterthwaite's method
            Sum Sq        Mean Sq NumDF           DenDF  F value     Pr(>F)
a   496.7108658511 165.5702886171     3  5.795106995423 23.58271 0.00118483 **
b   181.5000000000 181.5000000000     1 11.999999840688 25.85163 0.00026866 ***
a:b  75.2500000000  37.6250000000     2 11.999999840262  5.35905 0.02171978 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> anova9_III
Type III Analysis of Variance Table with Satterthwaite's method
            Sum Sq        Mean Sq NumDF           DenDF  F value     Pr(>F)
a   676.9713224343 225.6571074781     3  5.789659050233 32.14107 0.00051883 ***
b   181.5000000000 181.5000000000     1 11.999999840688 25.85163 0.00026866 ***
a:b  75.2500000000  37.6250000000     2 11.999999840262  5.35905 0.02171978 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

So the values for Type I ANOVA match, but the values for Type II and III ANOVA does not. Specifically, only the values for the first factor/effect (a) is different.
I take a closer look at the code in SAS and R(lmerTest). It turns out the hypothesis contrast matrix (L) is set up differently between SAS and R(lmerTest). For R(lmerTest), all three types of ANOVA consider a1,a2 and a3 (there are three levels in a). For SAS, Type I uses a1,a2 and a3; Type II and Type II uses a1 and a2. This causes the numerator df to be different.
lmerTest and PROC MIXED constructs L matrix differently when the fixed intercept is removed from the model. Which one is correct?
Potentially useful documents:
The paper in this link (Appendix B.2,B.3 and B.4) describes how the hypothesis contrast matrix is formulated in lmerTest. This link mentions how SAS performs Type I,II and III ANOVA.
Answer: For what it is worth, emmeans agrees with SAS on this. The Type III tests are:
> library(emmeans)
> jt3 = joint_tests(results9, mode = "satterth")
> jt3
 model term df1 df2 F.ratio p.value
 a            2   8   5.426  0.0324
 b            1  12  25.852  0.0003
 a:b          2  12   5.359  0.0217

Using a new addition not yet present in the CRAN version, we can see the associated estimable functions:
> attr(jt3, "est.fcns")
$a
     a1 a2 a3 b2 a2:b2 a3:b2
[1,] -1  1  0  0   0.5   0.0
[2,]  0 -1  1  0  -0.5   0.5

$`a:b`
     a1 a2 a3 b2 a2:b2 a3:b2
[1,]  0  0  0  0     1     0
[2,]  0  0  0  0    -1     1

$b
     a1 a2 a3 b2     a2:b2     a3:b2
[1,]  0  0  0  1 0.3333333 0.3333333

These are exactly what you would expect.
The type II tests (via jt2 = joint_tests(results9, mode = "satterth", submodel = "type2")) are identical to the type III ones here, due to the balance.
One interesting thing to me is that the Keward-Roger method yields fairly different (and more conservative) results. This is due to the adjustment that K-R applies to the covariance matrix.
> joint_tests(results9, mode = "kenward")
 model term df1   df2 F.ratio p.value
 a            2 10.67   4.070  0.0486
 b            1 16.00  19.389  0.0004
 a:b          2 16.00   4.019  0.0385

One explanation I can think of is that SAS and joint_tests() both use joint tests of estimable linear functions of the regression coefficients; hence they both explicitly address the question of estimating contrasts related to each effect. However, the appendices of the lmerTest article in JSS (linked in the OP) refer to manipulations of the columns of the design matrix. The way that R constructs the design matrix for this model, there are 3 columns associated with a and therefore the intercept effect becomes part of the a effect in its calculations. In other words, there is an implicit assumption that design-matrix columns associated with predictors are all associated with contrasts; and that isn't the case in the model without intercept.
Comment: FWIW, the `"est.fcns"` attribute is now available from the [GitHub site](https://github.com/rvlenth/emmeans)
Comment: Thanks for the through answer! The last paragraph is particularly helpful.


Can `l1` minimization be solved in Matlab using `fmincon`?

Question: Consider a MA system $$y[k] = x[k]+a_1*x[k-1] + a_2*x[k-2] + a_{9}*x[k-9] + n[k] \tag{1}$$ where $n \sim N(0,\sigma^2)$ is Additive White Gaussian Noise and $x$ is a zero mean Gaussian white process of unity variance.
In this model only 3 coefficients are non-zero whereas the rest are all zero or close to zero. I should not assume the position of the sparse coefficients in order to apply sparse estimation methods. The correct approach is to assume that there are 10 coefficients out of which few are nonzero and majority are zero.
QUESTION:
My approach is to split the objective function J into 2 parts--The first part being the OLS and the second part being the sparsification of the OLS estimate using a l1 minimization approach?
Comment: A cool package you might want to checkout is [cvx](http://cvxr.com/).
Comment: @MatthewGunn: I want to understand what is actually going on inside the l1 minimization, so was thinking if there is a step by step approach
Comment: If this question's title wasn't Matlab-specific, and made more general to the issue of selecting nonzero coefficients in an MA model, it might be reopened.
Answer: You can introduce a variable to transform a problem with an $L_1$ norm to a linear programming problem. Consider the $L_1$ minimization problem:
\begin{equation}
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $x_i$)} & \displaystyle \sum_i |x_i|\\
\mbox{subject to}  & ...
 \end{array}
\end{equation}
We can introduce variables $w_i$ and write it as a linear program:
\begin{equation}
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $x_i, w_i$)} & \displaystyle \sum_i w_i\\
 \mbox{subject to} & x_i \leq w_i \\
     & - x_i \leq w_i \\
    &...
 \end{array}
\end{equation}
If the other constraints are also linear, this can be solved with the linprog function in MATLAB.
Comment on your particular problem:
Since your problem also has quadratic terms in the objective, you would use quadprog. Using fmincon probably should work as well (though fmincon isn't specialized to your problem's structure). As I mentioned before, CVX is nice package as well. I'd check that your solution is the same using quadprog and CVX.
As I understand your problem:
\begin{equation}
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $a_i$)} & \displaystyle \sum_t \left( y_t - \mathbf{a} \cdot \mathbf{x}_t \right)^2 + \alpha \|\mathbf{a}\|_1\\
 \end{array}
\end{equation}
Which becomes:
\begin{equation}
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $a_i, w_i$)} & \displaystyle \sum_t \left( y_t - \mathbf{a} \cdot \mathbf{x}_t \right)^2 + \alpha \mathbf{1}' \mathbf{w}\\
\mbox{subject to}  & \mathbf{a} \preceq \mathbf{w} \\
& -\mathbf{a} \preceq \mathbf{w}
 \end{array}
\end{equation}
Rewriting more, the problem is equivalent to:
\begin{equation}
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $a_i, w_i$)} & \displaystyle  \mathbf{a}'X'X\mathbf{a} - 2 \mathbf{y}' X \mathbf{a} + \alpha \mathbf{1}' \mathbf{w}\\
\mbox{subject to}  & \mathbf{a} \preceq \mathbf{w} \\
& -\mathbf{a} \preceq \mathbf{w}
 \end{array}
\end{equation}
Where $X = \begin{bmatrix} \mathbf{x}_1' \\ \mathbf{x}_2' \\ \ldots \end{bmatrix}$ etc...
Comment: @SrishtiM I think your confusion is that you don't solve $\min f(x) + g(x)$ by solving $\min f(x)$ and $\min g(x)$ separately. You don't obtain your `a` from maximum likelihood and then modify that result to take into account the $L_1$ norm. You need to formulate your problem as a single, optimization problem.
Comment: @SrishtiM You need to either (1) pull out the pencil and paper and figure out what those matrices `A`, `Aeq` etc... are or (2) use a package like CVX which makes it somewhat more intuitive to write down your optimization problem.
Comment: I see, this clear a lot of my misconception. Do you have some idea or suggestions as to how I should only code for the term $\sum_{i=1}^p |a_i|$? Or if possible how to code your answer using `quadprog` or constrained minimization?I don't understand how to make the matrices `A`,`Aeq` etc in the function signature and their meaning.


Form committee of $2$ out of $5$ women and $3$ out of $6$ man with additional requirements.

Question: What is the number of ways of forming a committee that consists of $2$ out of $5$ women Ms. A, B, C, D, E, and $3$ out of $6$ man, Mr. A, B, C, D, E, F.
There are two additional requirements:

Mr. A refuses to serve on committee if Mr. B is a member
Mr. B only serve wants to serve if Ms. C is the member of the committee
Comment: How many ways to choose two women from five? Three men from six? Permutations and combinations? This is straightforward from the formula, isn't it? Where can you possibly be confused in a direct application?
Comment: There are additional requirements, where some combinations are prohibited. In the original subject, that was not clear, that's why I edited the subject a bit.
Answer:
all committees: ${6 \choose 3} * {5 \choose 2}$
remove the committees with both A and B: $4*{5 \choose 2}$
remove the committees with B and not C: ${5 \choose 2} *4$
inclusion/exclusion: add the committees with A, B, not C. (You subtracted twice and have to compensate): $4*{4 \choose 2}$
Answer: As @ астон вілла олоф мэллбэрг mentioned, this question is an application of simple combinatorics.
The number of ways of choosing  women from 5 women is $5\choose2$ while number of ways of choosing 3 men from 6 men is  $6\choose3$. So, The total number of ways are $5\choose2$$\times$$6\choose3$.
Comment: This is correct for the number of committees, but does not address the additional requirements.


Quantile Regression Pseudo R-Squared

Question: When computing the Pseudo R-squared value in quantreg in R or statsmodel in Python, what is an acceptable range to justify goodness of fit? Also, what is the functional form of Pseudo R-Squared computed in those packages?
Comment: As to the question about an acceptable range for r-square or pseudo r-square measures, there really is no such thing as a guideline for an "acceptable" range.  Expectations are really dependent on the field of study and specific application.  In a controlled chemistry experiment, you might expect an r-square of 0.99.  When dealing with biological systems or human behavior, even quite small r-squares may be encouraging.
Comment: One potential answer to the form of r-squared in quantreg: [https://stackoverflow.com/questions/19861194/extract-r2-from-quantile-regression-summary](https://stackoverflow.com/questions/19861194/extract-r2-from-quantile-regression-summary)
Comment: Another potential answer concerning r-squared in quantreg: [https://stats.stackexchange.com/questions/129200/r-squared-in-quantile-regression](https://stats.stackexchange.com/questions/129200/r-squared-in-quantile-regression)
Comment: You should also be able to use Cox and Snell or Nagelkerke or Efron pseudo r-squares.  These should be easily obtained from the model object from *quantreg*.
Answer: As far as what constitutes a good value for a performance metric, this answer explains why that is context-dependent, so there is no hard rule. This makes a lot of sense. While one might view $R^2$ -style metrics as being like letter grades in school where $R^2=0.9$ is the kind of score that seems like an $\text{A}$-grade that makes us happy while $R^2=0.4$ is the kind of score that seems like an $\text{F}$-grade that makes us sad, I remember celebrating a $41\%$ on an extremely challenging exam in graduate school that qualified as one of the higher scores in the class, so even letter grades in school are not as simple as $90/80/70/\text{etc}$.
As far as what constitutes an $R^2$-style metric for quantile regression, it depends on what you want your metric to tell you. My take on the usual $R^2$ is that it is a comparison of the performance of your model on some metric of interest (sum of squared residuals, for the usual $R^2$) to the performance of a "must-beat" model. With that in mind, the typical loss function for quantile regression, much like how the typical loss function for linear regression is the sum of squared residuals, is the quantile loss ("pinball loss" in many circles), $L_{\tau}(y, \hat y)$ in the language below.
$$
l_{\tau}(y_i, \hat y_i) = \begin{cases}
      \tau\vert y_i - \hat y_i\vert, & y_i - \hat y_i \ge 0 \\
      (1 - \tau)\vert y_i - \hat y_i\vert, & y_i - \hat y_i < 0
   \end{cases}\\
L_{\tau}(y, \hat y) = \sum_{i=1}^n l_{\tau}(y_i, \hat y_i)
$$
As I discuss here, a reasonable $R^2$-style performance metric for a quantile regression would compare the pinball loss of your model to the pinball loss of a "must-beat" model that always predicts the $\tau^{th}$ quantile of all $y$ values combined together. This is more-or-less implemented in Python using sklearn.metrics.d2_pinball_score, though I have some issues with how sklearn performs and documents the calculation. (I have strong feelings about how to calculate the $R^2$-style metrics when it comes to out-of-sample performance, and the sklearn $D^2$ function suffers from the same issues I bemoan in the link. Further, sklearn remarks that the $D^2$ is the proportion of pinball loss explained by the model. This alludes to how $R^2$ is a measure of the proportion of variance explained by a linear regression model, though this breaks down in all but the nicest of circumstances (1)(2), and I expect the same to happen for $D^2$.)
Comment: can you also frame this in terms of log-likelihood measures since quantile regression equates to MLEs from a Laplace distribution?
Comment: @FrankHarrell That makes sense to me.

Quantile Regression Pseudo R-Squared


How to solve this functional equation involving hyperbolic functions?

Question: I'm reading this (physics) book. They have the recurrence relation (book eq. 14.2.14)
$$f(K_1,0)=-\frac{1}{2}\ln\{2\sqrt{\cosh(2K_1)}\}+\frac{1}{2}f(\ln\sqrt{\cosh(2K_1)},0).\qquad(1)$$
They give the following solution (book eq. 14.2.15)
$$f(K_1,0)=-\ln(2\cosh K_1).\qquad(2)$$
I tried to check the solution was correct simply plugging (2) into (1) but end up with a $\ln(\cosh(\ln(\cosh(K_1)))$ thing.
Questions:

How do I obtain the solution (2) from (1)?
Is there an easy way to check the solution is correct?
Comment: $\cosh(\ln(x)) = \dfrac{x^{2}+1}{2x}$
Answer: Note that
\begin{align}
\cosh x = \frac{e^x+e^{-x}}{2}
\end{align}
then you see that
\begin{align}
2 \cosh( \ln( \sqrt{\cosh(2K_1)}) =&\ 2\frac{\exp\left(\ln \sqrt{\cosh(2K_1)}\right)+\exp\left(-\ln \sqrt{\cosh(2K_1)} \right)}{2}\\
=&\ 2\frac{\sqrt{\cosh 2K_1}+ (\sqrt{\cosh 2K_1})^{-1}}{2} =  \frac{\cosh 2K_1 +1}{\sqrt{\cosh 2K_1}}.
\end{align}
Then it follows
\begin{align}
-\frac{1}{2}\ln\left(2\sqrt{\cosh 2K_1} \right)-\frac{1}{2}\ln\left(\frac{\cosh 2K_1 +1}{\sqrt{\cosh 2K_1}} \right)=&\ -\frac{1}{2}\ln\left( 2\cosh(2K_1)+2\right)\\
=&\ -\frac{1}{2}\ln(4\cosh^2 K_1).
\end{align}
Comment: Nice check, thank you. Any idea on how to derive the solution from equation (1)?


Translating C++ File reading/writing to C# File reading/writing

Question: All,
I'm doing some programming exercises in C++, then attempting to translate them to C# and other languages to brush up.
This example comes from Ch. 11.2 from Nell Dale and Chip Weems' "Programming and Problem Solving with C++" (fifth edition), pages 521-522. It's a program that takes in this file (student name, GPA, four grades, and the course grade)- let's call it rec.in:

Mary Abbott
  3.4 80 90 60 99 B
Bob Baker
  4.0 90 90 90 95 A
Sally Carter
  2.0 70 70 70 65 C
Bill Dansk
  3.0 65 50 60 70 D
JoEllen Zureck
  1.9 90 95 80 99 A

And (more or less) just copies it to another file.
Here's the code to do this:
#include <iostream>
#include <string>
#include <fstream>

using namespace std;

const int MAX_STUDENTS = 150;

enum GradeType{A,B,C,D,F};

struct StudentRec
{
    string stuName;
    float gpa;
    int examScore[4];
    GradeType courseGrade;
};
StudentRec gradeBook[MAX_STUDENTS];

int main()
{
    StudentRec record[MAX_STUDENTS];
    ofstream outFile;
    outFile.open("rec.out");
    ifstream inFile;
    inFile.open("rec.in");
    for (int count = 0; count < MAX_STUDENTS; count++)
    ReadValues(inFile, record[count]);
    for (int count = 0; count <= MAX_STUDENTS; count++)
    PrintValues(outFile, record[count]);
    inFile.close();
    outFile.close();
    return 0;
}

void ReadValues(ifstream& inFile, StudentRec& record)
{
    char letter;
    char throwAway;

    getline(inFile, record.stuName);
    inFile >> record.gpa>>record.examScore[0]
    >> record.examScore[1] >> record.examScore[2]
    >> record.examScore[3] >> letter;

    inFile.get(throwAway);

    switch(letter)
    {
        case 'A' : record.courseGrade = A;
           break;
        case 'B' : record.courseGrade = B;
           break;
        case 'C' : record.courseGrade = C;
           break;
            case 'D' : record.courseGrade = D;
           break;
        case 'F' : record.courseGrade = F;
               break;
   }
}

void PrintValues(ofstream& outFile, StudentRec& record)
{
    outFile << record.stuName << endl;
    outFile << record.gpa << ' ' << record.examScore[0] << ' '
        << record.examScore[1] << ' '
        << record.examScore[2] << ' '
        << record.examScore[3] << ' ';
    switch (record.courseGrade)
    {
        case A : outFile << 'A';
        break;
        case B : outFile << 'B';
        break;
        case C : outFile << 'C';
        break;
        case D : outFile << 'D';
        break;
        case F : outFile << 'F';
        break;
    }

outFile << endl;
}

I'm starting out with a translation to C#. I've done C# work before, but not console work. As a result, it's a bit... messy... working to translate stuff. I'm a tad confused with matching the getline function of C++ in C#. There is, of course, System.IO.StreamReader, but that only reads a file a line at a time. getline does it in a stream format, as seen above. This breaks it up using the spaces/newlines between words/characters. How would I achieve the same result with C#'s library choices? I'm not exactly married to the System.IO.StreamReader solution.
Here's the beginning of what I have:
using System;
using System.IO;

namespace _112a
{
    class Program
    {
        const int MAX_STUDENTS = 150;
        enum GradeType{A,B,C,D,F};
        struct StudentRec{
            string stuName;
            float gpa;
            GradeType courseGrade;
        };

        StudentRec[] gradeBook = new StudentRec[MAX_STUDENTS];

        static void Main(string[] args)
        {
            StudentRec[] record = new StudentRec[MAX_STUDENTS];
            System.IO.StreamWriter outFile = new System.IO.StreamWriter("rec.out");
            System.IO.StreamReader inFile = new System.IO.StreamReader("rec.in");

            for (int count = 0; count < MAX_STUDENTS; count++)
            {
                ReadValues(inFile, record[count]);
            }

            for (int count = 0; count <= MAX_STUDENTS; count++)
            {
                PrintValues(outFile, record[count]);
            }

            inFile.Close();
            outFile.Close();
        }

        static void ReadValues(StreamReader inFile, StudentRec record)
        {
            char letter;
            char throwAway;

            /*can't use getline, what do?*/

            inFile.ReadLine();

            switch (letter)
            {
                case 'A':
                    record.courseGrade = (GradeType)Enum.Parse(typeof(GradeType), "A");
                    break;
                case 'B':
                    record.courseGrade = (GradeType)Enum.Parse(typeof(GradeType), "B");
                    break;
                case 'C':
                    record.courseGrade = (GradeType)Enum.Parse(typeof(GradeType), "C");
                    break;
                case 'D':
                    record.courseGrade = (GradeType)Enum.Parse(typeof(GradeType), "D");
                    break;
                case 'F':
                    record.courseGrade = (GradeType)Enum.Parse(typeof(GradeType), "F");
                    break;
            }

        }

        static void PrintValues(StreamWriter outFile, StudentRec record)
        {
            outFile.Write("{0}\n{1} {2} {3} ", record.stuName, record.examScore[0], record.examScore[1], record.examScore[2], record.examScore[3]);

            switch(record.courseGrade)
            {
                case GradeType.A:
                    outFile.Write("A\n");
                    break;
                case GradeType.B:
                    outFile.Write("B\n");
                    break;
                case GradeType.C:
                    outFile.Write("C\n");
                    break;
                case GradeType.D:
                    outFile.Write("D\n");
                    break;
                case GradeType.F:
                    outFile.Write("F\n");
            }
        }
    }
}

I have an additional question regarding the switch structure you see in the method PrintValues: in each of the cases, it's specified in the C++ file that it just takes the record.courseGrade field and compares it to the argument specified behind case. C#, being a type-strong language, won't let this happen... if you compare char 'A' to record.courseGrade, you get a type mismatch error. Alternately, leaving it without the single quotes (and thus leaving it as a record.courseGrade field) gives the error of The name 'A' does not exist in the current context. How do I best deal with this?- RESOLVED
My confusion using ReadLine() stems from how it would work. You have multiple members of the StudentRec struct being filled in this case, but they're separate entities in the struct. My fear would be doing something like:
StudentRec.stuName = inFile.ReadLine();
and having StudentRec.stuName filled with the entire line, with nothing in the other struct fields.
Furthermore, we have the issue of letter being unassigned and used as a switch; this is due to it never receiving a value from the ReadLine. However, that's an academic solution, if ReadLine does indeed split things up.
Answer: You want to use StreamReader and Writer respectively. The methods you're likely looking for are ReadLine and WriteLine, the StreamReader docs have a working example at the bottom that will suite your needs. You'll just need to change the file path and logic inside the while loop. The docs can be found here;
http://msdn.microsoft.com/en-us/library/system.io.streamreader.aspx
StreamWriter docs are here and have an example the the bottom that uses both the reader and writer;
http://msdn.microsoft.com/en-us/library/system.io.streamwriter.aspx
All of the methods have logical names and are simpler to implement than their c++ counterparts.
I believe your switch causing a compilation error because the cases need to be case "A": instead of case A:. If you put a literal there it's going to do a string comparison. It doesn't understand what an A is unless that's the name of a local variable.
EDIT: sample code to read the file;
// we need to alloc/init the streams inside for using blocks so they're properly disposed of
List<string> grades = new List<string>();
using (StreamReader inFile = new StreamReader("rec.in"))
{
     string line;
     while ((line = inFile.ReadLine()) != null)
     {
         string[] tokens = line.Split(' '); // split line on whitespace
         if (tokens.Length > 7)
             grades.Add(tokens[7]);
     }
}

using (StreamWriter outFile = new StreamWriter("rec.out"))
{
      foreach (string s in grades)
          outFile.WriteLine(s);
}

The techniques I'm using above aren't completely necessary, for example you could nest the StreamWriter using inside the StreamReaders and do the writing within that same scope. In the C# version you're looping over the contents of the file line by line. Each time we get a line we want to split it on spaces to get it's sub components. Then I add the letter grade to a list. We don't want newlines in our output strings because WriteLine does that automatically. After we're doing reading the file we loop over each string in the list and write it to the output file.
Comment: @AaronMcRuer your code could be made to work but just doesn't really look like it normally would in C#. I don't know if this is because you want the programs structure to remain the same as the C++ version or not but the code for reading/writing should be a lot different. I'll update with an example.
Comment: These output to the console. I'm looking to output to a file.
Comment: @AaronMcRuer yes, I see this now. Updated with appropriate information.
Comment: Would I have to somehow step through the stream?

I guess I'm a little unclear on how it compares to the C++ implementation. The C++ implementation is a stream, of course, so things get broken up into pieces, but the C# implementation is a functional implementation.
Comment: EDIT: adding updated code to the original question. Stand by.
Comment: Done with changes, see if that explanation changes answers.
Comment: That's probably, evan. I'm just more or less trying to stick to the original logic where I can. Of course, that can change. That's the beauty of doing these exercises just for the sake of doing them.
Answer: For the problem with switch try this:
case GradeType.A:
    outFile.Write("A\n");
    break;

You work with enumeration and must give it to the switch case.
For other problem if you want to work with the each row in file, you must assign it to a string and then process it to extract the information you need through methods such as split and so on.
string line = inFile.ReadLine();
string[] data = line.Split(' ');
record.stuName = string.Format("{0} {1}", data[0], data[1]);
record.gpa = float.Parse(data[2]);
....

Comment: This gets me to the first string ("Mary") and no further. The rest of the array is null... how would I step through? I think you've gotta be really close with your idea.
Comment: I think that the file is a record more than one line. In this case you have to use Readline times as you need. Could you post the contents of the variable line as soon as you read it.


Android LinearLayout on top of ActionBar?

Question: this seems like a very simple issue but I can't seem to figure it out.
I am adding a LinearLayout to my active Activity when a button is pressed but I want this layout to be on top of the whole activity, that includes the ActionBar.  I want the layout to work like a Dialog but I think Dialog is too heavy for what I am trying to do so I want to use a layout.
I tried applying Flags and Type when I am setting the layout's WindowManagerLayoutParams but nothing helps.
this is what I tried:
WindowManagerLayoutParams mainLayoutParams = new WindowManagerLayoutParams ();
mainLayoutParams.Height = 600;
mainLayoutParams.Width = 300;
mainLayoutParams.Gravity = GravityFlags.Center;
mainLayoutParams.Flags = WindowManagerFlags._______
mainLayoutParams.Type = WindowManagerTypes._______

I wrote "______" because I've tried ALL different ones but it makes no difference.
Please give me solution in which I can do this programmatically (not XML) since my layout is created manually.
How can I do this?
Thank you for your time.
Comment: When you say "on top", do you mean on the Z axis (higher elevation, "closer" to the user's eyes in theory), or on the Y axis (closer towards the top edge of the screen)?
Comment: I believe Z axis. I want the layout to take up 85% of the screen and rest is covered with an overlay.  So basically a standard `Dialog` or `Alert` look.  I've updated my question to show what I tried.
Answer: Not only is Dialog not a particularly "heavy" solution, it is the lightest one that I can think of. Your alternatives would be:

Rewrite your app to use Toolbar in your layouts instead of the action bar, then use a RelativeLayout or FrameLayout container around the Toolbar and main contents. You can then add your floating content to the container as a child, and later children of RelativeLayout and FrameLayout appear higher on the Z axis. However, this will cause problems on Android 5.0+ with elevation, so I would not recommend it.
You could make your floating thing be a dialog-themed activity (e.g., Theme.Holo.Dialog). When you start the activity, it will appear to float over the existing activity.
Comment: I guess my best option is go with a `Dialog`.  Another problem with that is how can I call `AnimationEnd` event on a dialog? I created XML file in my res/anim folder and applying it to dialog this way:  `myDialog = new Dialog (context, Resource.Style.myDialogAnim);`
Comment: @user3866010: The `Dialog` constructor does not take an animation resource, and I have no idea what you are trying to do. I suggest that you ask a separate Stack Overflow question where you explain what you are trying to accomplish related to dialogs and animations, in hopes that somebody can help you.
Comment: The `Dialog` constructor doesn't take an animation resource but it does take style. I created a style in my res/values/styles file that has one item: `windowAnimationStyle` and the value of that item is another style that has two items: `windowEnterAnimation` and `windowExitAnimation`.  This automatically applies animation to the dialog. But I will create a new question. Thanks
Comment: @user3866010: "The Dialog constructor doesn't take an animation resource but it does take style" -- that's not how I interpreted your previous comment, sorry. I have not played with window animations, and so I do not know if you can get your hands on the inflated animation objects to set up an event listener or not. Sorry!


How do you show whether a transformation is linear or not?

Question: Consider the transformation $T$ from $\mathbb{R^2}$ to $\mathbb{R^3}$ given by:
$$T \left( \begin{array}{ccc}
x_1  \\
x_2 \\ \end{array} \right) = x_1 \left( \begin{array}{ccc}
1  \\
2 \\
3 \end{array} \right) + x_2\left( \begin{array}{ccc}
4  \\
5 \\ 6 \end{array} \right)$$
How do we determine whether the transformation is linear, and subsequently find its matrix?
I understand that for a transformation to be linear, we need to show $T(\alpha x+ \beta y) = \alpha T(x)+\beta T(y)$. Can someone provide an instance where this would not be the case, so I can understand how this "proves" linearity?
Comment: If it is too difficult for you to do it, try to separate the proof into two parts, namely 1) $T(x+y)=T(x)+T(y)$ and 2) $T(\alpha x)=\alpha T(x)$, for all $\alpha \in \mathbb{R}$.
Answer: By definition, a transformation is linear if it is closed under vector addition and scalar multiplication. This means that for any two vectors $x$ and $y$, and any scalar $\alpha$, $$T(x+\alpha y) = T(x) + \alpha T(y).$$ A simple example of a nonlinear transformation is $T(x) = x^2.$
In this problem, you want to look for a $3\times2$ matrix that, when left-multiplied with $\begin{pmatrix} x_1\\x_2\end{pmatrix}$, will give you $$x_1\begin{pmatrix} 1\\2\\3\end{pmatrix}+x_2\begin{pmatrix} 4\\5\\6\end{pmatrix}.$$
It may be easier to see if you combine the terms on the right-hand side, $\begin{pmatrix}x_1+4x_2\\2x_1+5x_2\\3x_1+6x_2\end{pmatrix}$.
It should be clear at this point that the transformation matrix $T = \begin{pmatrix}1&4\\2&5\\3&6\end{pmatrix}$.
Answer: Just check by definition, let $\alpha, \beta \in \mathbb{R}$, $x=(x_1,x_2),y=(y_1,y_2)\in \mathbb{R^2}$, then $\alpha x+ \beta y=(\alpha x_1+\beta y_1,\alpha x_2+\beta y_2),$
$$T \left( \begin{array}{ccc}
\alpha x_1+\beta y_1  \\
\alpha x_2+\beta y_2 \\ \end{array} \right) = (\alpha x_1+\beta y_1) \left( \begin{array}{ccc}
1  \\
2 \\
3 \end{array} \right) + (\alpha x_2+\beta y_2)\left( \begin{array}{ccc}
4  \\
5 \\ 6 \end{array} \right)$$
Then can you rearrange the terms so that the RHS=$\alpha T\left(\begin{array}{ccc} x_1\\x_2\end{array}\right)+\beta T\left(\begin{array}{ccc} y_1\\y_2\end{array}\right)$
Hence it proves the linearity by definition.
Comment: Can you provide an example of a transformation that is not linear? At what point in our algebraic manipulation would we find that it was impossible to arrange the equation in that?
Comment: The transformation given in your form is always linear. For a simple nonlinear transformation, $T:\mathbb{R}\to \mathbb{R}$ given by $T(x)=x^2$. Then $T(-1)=T(1)=1$,but $T(1+(-1))=0 \neq  T(-1)+T(1)$.
Comment: @JohnBayley Please see above comment.


How to test if env variable is set in bats-assert framework?

Question: I am using BATS testing framework with bats-assert and bats-support.
I want to ensure that the user remembered to set the ELASTIC_CREDS env variable properly.  How do I do that?   Here is what I've tried:
config.bash
export SYSTEM_CREDS=myuser:mypass

Then, my checkcreds.bats test fike looks like this:
checkcreds.bats
#! ./libs/bats/bin/bats
load 'libs/bats-support/load'
load 'libs/bats-assert/load'
load 'config'

@test 'assert_system_creds()' {
  run env | grep SYSTEM_CREDS | awk -F= '{print$2}'
  assert_output --regexp '^.*:.*$'
}

Comment: I'd like to know how to do that concisely using Bats, but the link that J.Leffler posted is very helpful and may just be what I need.
Comment: See [What's a concise way to check that environment variables are set in a Unix shell script?](https://stackoverflow.com/questions/307503/whats-a-concise-way-to-check-that-environment-variables-are-set-in-a-unix-shell)  I believe this could/should be a duplicate of that.
Comment: Are you asking how to do this in Bash, or in Bats?
Comment: `grep 'x' | awk '{ y }'` can usefully be refactored to `awk '/x/ { y }'`; see also [useless use of `grep`](http://www.iki.fi/era/unix/award.html#grep)
Answer: This expression works fine:
@test 'check_env_vars()' {
  run : ${SYSTEM_CREDS?"Need to set SYSTEM_CREDS"}
  assert_success
}


build custom corpus with labels from text documents using nltk

Question: I am following the tutorial to build custom corpus with ntlk. This is an example of my code:
import pandas as pd
import numpy as np
import re
import nltk
import matplotlib.pyplot as plt

pd.options.display.max_colwidth = 200
%matplotlib inline

corpus = ['t_3_0 v_0_17 v_1_20 v_2_78 u_0_1 u_0_2 u_1_2',
          't_3_1 v_0_144 v_1_17 v_2_20 u_0_1 u_0_2 u_1_2',
          't_3_2 v_0_143 v_1_233 v_2_238 u_0_1 u_0_2 u_1_2',
          't_3_3 v_0_20 v_1_253 v_2_275 u_0_1 u_0_2 u_1_2',
          't_3_4 v_0_144 v_1_209 v_2_90 u_0_1 u_0_2 u_1_2',
          't_3_59 v_0_233 v_1_222 v_2_51 v_3_52 u_0_1 u_0_2 u_0_3',
          't_3_60 v_0_238 v_1_11 v_2_137 v_3_143 u_0_1 u_0_2 u_0_3',
          't_3_61 v_0_238 v_1_111 v_2_214 v_3_94 u_0_1 u_0_2 u_0_3',
          't_3_62 v_0_238 v_1_111 v_2_214 v_3_97 u_0_1 u_0_2 u_0_3',
          't_3_63 v_0_238 v_1_137 v_2_214 v_3_51 u_0_1 u_0_2 u_0_3'

]
labels = ['block_1', 'block_1', 'block_1', 'block_1', 'block_1',
          'block_2', 'block_2', 'block_2', 'block_2', 'block_2']

corpus = np.array(corpus)
corpus_df = pd.DataFrame({'Document': corpus,
                          'Category': labels})
corpus_df = corpus_df[['Document', 'Category']]
corpus_df

The output looks like this:
    Document                                                Category
0   t_3_0 v_0_17 v_1_20 v_2_78 u_0_1 u_0_2 u_1_2            block_1
1   t_3_1 v_0_144 v_1_17 v_2_20 u_0_1 u_0_2 u_1_2           block_1
2   t_3_2 v_0_143 v_1_233 v_2_238 u_0_1 u_0_2 u_1_2         block_1
3   t_3_3 v_0_20 v_1_253 v_2_275 u_0_1 u_0_2 u_1_2          block_1
4   t_3_4 v_0_144 v_1_209 v_2_90 u_0_1 u_0_2 u_1_2          block_1
5   t_3_59 v_0_233 v_1_222 v_2_51 v_3_52 u_0_1 u_0_2 u_0_3  block_2
6   t_3_60 v_0_238 v_1_11 v_2_137 v_3_143 u_0_1 u_0_2 u_0_3 block_2
7   t_3_61 v_0_238 v_1_111 v_2_214 v_3_94 u_0_1 u_0_2 u_0_3 block_2
8   t_3_62 v_0_238 v_1_111 v_2_214 v_3_97 u_0_1 u_0_2 u_0_3 block_2
9   t_3_63 v_0_238 v_1_137 v_2_214 v_3_51 u_0_1 u_0_2 u_0_3 block_2

What I would like to do is instead of hardcoding corpus and labels, I would prefer to read each line from txt documents and assign labels automatically using the name of the file. For example:
corpus = ['block_1.txt', 'block_2.txt', 'block_3.txt', 'block_4.txt']

labels = ['block_1', 'block_2', 'block_3', 'block_4']

corpus = np.array(corpus)
corpus_df = pd.DataFrame({'Document': corpus,
                          'Category': labels})
corpus_df = corpus_df[['Document', 'Category']]
corpus_df

The example of desirable output would look like this:
    Document                                                Category
0   t_3_0 v_0_17 v_1_20 v_2_78 u_0_1 u_0_2 u_1_2            block_1
1   t_3_1 v_0_144 v_1_17 v_2_20 u_0_1 u_0_2 u_1_2           block_1
2   t_3_2 v_0_143 v_1_233 v_2_238 u_0_1 u_0_2 u_1_2         block_1
...

I would appreciate any suggestions and help!
Thank you!
Olha
Answer: I figure out the answer. Basically, I created a list of tuples and then categorize each line.
transactions = []
with open('block_1.txt', 'r') as block1, open('block_2.txt', 'r') as block2, open('block_3.txt', 'r') as block3, open('t_block_4.txt', 'r') as block4:
    transactions = ([(transaction, 'block_1') for transaction in block1.readlines()] + [(transaction, 'block_2') for transaction in block2.readlines()] + [(transaction, 'block_3') for transaction in block3.readlines()] + [(transaction, 'block_4') for transaction in block4.readlines()])

#transactions

corpus = np.array(transactions)
corpus_df = pd.DataFrame({'Document': [i[0] for i in transactions],
                          'Category': [i[1] for i in transactions]})
corpus_df = corpus_df[['Document', 'Category']]
corpus_df


Mean Value Theorem question (show that f is constant)

Question:
Suppose that $f, g : R → R$ are functions such that
$$|f(x) − f(y)| ≤ |g(x) − g(y)| \sqrt{|x − y|}$$ for any $x, y ∈ R$.
If $g$ is differentiable with bounded derivative on all $R$, show that
$f$ is constant.

I know I am meant to be using MVT for this question, I attempted to use $g(x)$ as the function because we know it is differentiable (condition required for MVT):
$$\frac{g(b)-g(a)}{b-a}=g'(c)$$
$$|g(b)-g(a)|=|g'(c)|b-a|$$
I am lost, not sure where to start the question.
Answer: If the derivative of $g$ is bounded as $|g'(c)| \le M$ for all $c$, then
$$\left| \frac{f(x)-f(y)}{x-y} \right| \le \frac{|g(x)-g(y)|}{\sqrt{|x-y|}}
\le M \sqrt{|x-y|}$$
for any $x,y$, where the last inequality is from your application of the mean value theorem. Can you take it from here?
Comment: I am not sure how you got the denominator in $\|\frac{f(x)-f(y)}{x-y}\|$...
Comment: @JohnWarts I took the inequality given in the problem and divided both sides by $|x-y|$.
Comment: and then after this step, I can say that f'(x) is smaller than M so it must be constant since $f'(x) = \frac{f(y)-f(x)}{y-x}$
Comment: @JohnWarts Every part of your comment is wrong... First, $f'(x) \ne \frac{f(y)-f(x)}{y-x}$.  Second, we haven't concluded $f'(x)$ is smaller than $M$. Finally, even if we did show $f'(x)$ is smaller than $M$, that isn't enough to conclude that $f$ is constant.


How to debug a CLR Stored procedure in VS 2013

Question: I have tried different ways to get this done but nothing works. VS seems to ignore all my breakpoints in the .cs files when I run the test script. I have included a breakpoint in the test script as well but still its not debugging.
This is the guide from Microsoft https://msdn.microsoft.com/en-us/library/ms165051(v=vs.100).aspx which I tried, but I don't have "a Set as Default Debug Script" option when I right click on my Script file.
Comment: What version of vs do you have? Enterprise?
Comment: Yes its Enterprise
Comment: Provide a [mcve].
Answer: You might need to also:

Run Visual Studio as an Admin
Open up additional ports that are, by default, blocked by Windows Firewall
Set the project as a Startup Project.
In Visual Studio: go to "SQL Server Object Explorer", right-click on the Instance this is deploying to in the "Debug" tab of "Project Properties", and select "Allow SQL/CLR Debugging", and click the "Yes" button in the pop-up dialog.
If you want breakpoints to work, you cannot have the "Optimize Code" option checked in the "SQLCLR Build" tab of "Project Properties". By default it is unchecked for the "Debug" configuration and checked for the "Release" configuration. The configuration itself doesn't matter, just so long as that option is not checked. If it was, you need to uncheck it and re-publish and make sure that it actually updates the Assembly (sometimes it might not since there are no other changes to the Assembly).
In Visual Studio: Open a new query from "SQL Server Object Explorer" and in the drop-down attached to the Execute ">" button, select "Execute With Debugger" (which is also ALT + F5). Sometimes breakpoints aren't trapped the first time around, so just execute with debugger again.
Please note: the login that you connect to SQL Server as for debugging needs to be a sysadmin.

There might be some additional things, but I am pretty sure that you do not need a default debug script, even though some posts about this say that you do.
Please see my answer here and the links in comments on the Question:
Can't attach to SQL Server to debug SQLCLR Stored Procedure
Comment: @RukshanHassim, See this thread which shared the detailed steps: https://stackoverflow.com/questions/14180009/unable-to-debug-net-code-could-not-attach-to-process-sql-clr-remote-debugging, it seems that you refer to the old VS2010 document.
Comment: Thank you for the reply. I tried all what you mentioned but I'm still facing the same problem.
Comment: @RukshanHassim I just added steps 4, 5, and 6. Please make sure that you are also following those and try again. If I do all 6 of these steps then it works (I am using VS 2015, if that matters, though I seem to recall that these steps worked on VS 2012 and/or VS 2013).
Answer: If you have Visual Studio and SQL Server in the same machine, simply attach the sqlservr.exe to Visual Studio debugger, and execute the stored procedure. It should hit the SP method.


Prove: Suppose S is closed, non-empty then if $b = \sup\{x: x \in S\}$, $b \in S$

Question: I will try to prove the theorem in the title:

Suppose S is closed, non-empty then if $b = \sup\{x: x \in S\}$ (least upper bound), $b \in S$.

I also use the following Theorem which we proved in class: S is closed iff every Cauchy sequence in S converges to a point in S.
Either $b \in S$ and we are done, or $b \notin S$ and so $\forall \epsilon_n = \frac{1}{n}$ for $n \in \mathbb{N},$ $\exists x_n \in (b - \epsilon_n, b] \cap S$ because if not then $b - \epsilon_n$ would be the least upper bound. So, clearly $x_n \rightarrow b$ as $n \rightarrow \infty.$ Since $x_n$ converges then $x_n$ is Cauchy and in $S$, so it must converge to some point in $S$, lets say $x_o$, by the above theorem. However, limits are unique, so $b = x_o$, but $b \notin S$ and $x_o \in S$. Thus we have arrived at a contradiction and so $b \in S. $
Comment: You should probably mention that $S$ is a subset of the real numbers. There are other structures in mathematics which support suprema and/or Cauchy sequences.
Comment: Presumably it is also assumed that $b$ is finite (otherwise what about $S=\mathbb R$?)
Comment: It seems a fine proof.
Answer: Your proof looks fine.  Here is an alternative (direct) proof.  Recall the definition of the supremum is that for every $\epsilon>0$, there exists $x\in S$ with $b-x<\epsilon$.  Now define a sequence $\{x_k\}_{k\in\mathbb{N}}$ as follows.  Let $\epsilon_k=2^{-k}$ and let $x_k$ be the point in $S$ obtained by applying the definition of the supremum to $\epsilon_k$.  Then for every $k$, we have $b-x_k<2^{-k}$.
You can prove directly that $\{x_k\}$ is Cauchy, since $\sum_{k=1}^\infty 2^{-k}$ converges.  The sequence also converges to $b$.  Thus, $b\in S$!
Comment: Cauchy is overkill. You know $S$ is closed, so any sequence from $S$, like the $x_n$, converges to a point in $S$. It converges to $x$, it's constructed that way, so $x \in S$.
Comment: @HennoBrandsma I know it's overkill.  I was trying to use the op's theorem: $S$ is closed iff every Cauchy sequence in $S$ converges to a point in $S$.
Answer: You don't need Cauchy. You construct the $x_n$ which exist, as otherwise $x_n - \frac{1}{n}$ would be a smaller upperbound than $b$, which cannot be.
Now you already know that $x_n \rightarrow b$ at this point.
You know that $S$ is closed (as you're trying to use the theorem about Cauchy-sequences) but closedness already implies what you want; $b$ is a limit point (or adherence point) of $S$ by virtue of the $x_n \in S$ that converge to it. So $b \in S$ already follows.
Comment: I see. I wanted to arrive at contradiction but I guess that isn't necessary. Thank you this makes things simpler


Independence between random variables and set

Question: Let $(\Omega,\mathcal{F},\mathbb P)$ be a probability space, and $X$ is a random variable in this space.
For a set $A\in \mathcal F$, can we conclude that
"$X$ is independent with $A$" $<=>$"$\int_AXd\mathbb P=\mathbb E(X)\mathbb P(A)$"?
I mean if these two statements are equivalent?
Thank you!
Answer: If $X$ is zero mean and $A:=\{X=0\}$, then we have $0=\mathbb E[X\chi(A)]=\mathbb E[X]\mathbb P(A)$. If $\mathbb P(A)\in (0,1)$, then $A$ is not independent of itself, hence it cannot be independent of the $\sigma$-algebra generated by $X$.
Comment: thanks for your explanation !


memory available to drush

Question: If one allocates additional memory to drush in drushrc.php or drush.ini (am I correct that these are potentially valid options on a shared server?), is there a way to confirm that this amount is actually allocated when running drush?
Answer: Try:
<code>$ drush ev 'print ini_get("memory_limit");'
</code>
This should print the memory_limit value set for Drush.  -1 means no limit.

memory available to drush


Add a meta content="noindex,nofollow" tag on a node created programmatically

Question: Is it possible to add a meta content="noindex,nofollow" tag to the head on the page of a node created programmatically (and onl this node)? I have created the node like this:
function MODULENAME_install() {
  $bodytext = 'Some Text';

  $node = new stdClass();
  $node->type = "page";
  $node->title = "New title";
  $node->language = LANGUAGE_NONE;
  node_object_prepare($node);
  $node->body[$node->language][0]['value'] = $bodytext;
  $node->body[$node->language][0]['summary'] = '';
  $node->body[$node->language][0]['format'] = 'full_html';
  $node->path = array('alias' => 'new-title');
  node_save($node);
}

Comment: You can add a variable and save the node id with this field. Check the current node having same id as variable then add meta tag.
Comment: yeah but i guess that i would have to get the nid of the created node inside the install function, but the add meta tag should probably go in another function.. or am i wrong? and what function should i use to add the meta tag, and can i then use the nid from the install function in the new function. Im sorry but i am new at module development.
Comment: When you save the node the nid will be returned so based on the returned nid you could do pass this to a function that will add the meta tag.
Comment: You can use `hook_preprocess_html`.
Comment: I have added this line after node save: $new_id = $node->nid;

and then added this function but it does not work..

function MODULENAME_preprocess_html(){
    if(arg(0) == "node" && is_numeric(arg(1)) && arg(1) == $new_id){
        $noindex = array(
            '#tag' => 'meta',
            '#attributes' => array(
              'name' => 'robots',
              'content' => 'noindex',
            )
        );
        drupal_add_html_head($noindex, 'noindex');
    }
}
Answer: You can use below code for adding no-follow and no-index meta for the node. Put this code in your themes template.php file.
function MYTHEME_html_head_alter(&$head_elements) {
  if (arg(0) == 'node' && arg(1) == 'YOUR NODE NID' && !arg(2)) {
    $head_elements['MYMODULE_meta_robots'] = array(
      '#tag' => 'meta',
      '#type' => 'html_tag',
      '#attributes' => array(
        'name' => 'robots',
        'content' => 'noindex'
      )
    );
  }
}


Adobe AIR - Captive runtime

Question: I'm using Adobe Flash CS5 with Adobe Air 2.5 to build an air app.
Is there a way to make use of the Air 3 captive runtime feature? Do I need additional software to publish the Air app with the captive runtime?
Answer: Captive Runtime is not yet integrated into the Flash Pro IDE.  You'll have to package your app using the command line tools available in the free AIR SDK.
Some more info is here; basically when you compile your app be sure to specify the target parameter.  To create an Android APK, you would use this:
<code>-target apk-captive-runtime
</code>
More info on options is here
Comment: Thanks for your answer. Is there a generator available for the command line tags?

Adobe AIR - Captive runtime


What is the difference between infinitely divisible and stable law?

Question: I read somewhere that stable law is the special case of infinitely divisible. In other word, stable distribution is a special case of infinitely divisible distribution.
But I am not quite sure what differentiate both of them apart.
Answer: Every stable law is infinitely divisible, but some infinitely divisible laws are not stable.  An example of the latter is the Poisson distribution.  For each $\lambda > 0$ and each $n\in\{1,2,3,\ldots\}$, there exist independent random variables $X_1,\ldots,X_n$ each distributed as $\mathrm{Poisson}(\lambda/n)$, so that $X_1+\cdots+X_n\sim\mathrm{Poisson}(\lambda)$.  Thus each Poisson distribution is infinitely divisible.  But the Poisson distribution is not stable, since, if you add two independent identically distributed random variables $X$ and $Y$, each with a Poisson distribution, what you get does not have the same distribution as $\alpha X+\beta$ for some $\alpha,\beta\in\mathbb R$.
Comment: I know that Poisson is not stable but I do not fully understand it yet. The definition on page 7 from this pdf http://fs2.american.edu/jpnolan/www/stable/chap1.pdf.
It seems like Poisson distribution would satisfy this definition, i.e. $X_1 +···+ X_n = c_n X +d_n$,
Comment: @LiamNousa : The Poisson distribution is supported on the set $\{0,1,2,3,\ldots\}$, whereas if $X$ is Poisson-distributed then $c_n X+d_n$ is supported on the set $\{ d_n, d_n+c_n, d_n+2c_n, d_n+3c_n,\ldots \}$, so that is not a Poisson-distributed random variable. $\qquad$
Comment: @MichaelHardy Can you explain it a bit more? If $X, Y \sim\mathrm{Poisson}(\lambda)$ then $X + Y \sim\mathrm{Poisson}(2 \lambda)$. That seems to be a Poisson distribution. I don't know where I am wrong.
Comment: It might be worth noting that the Poisson distribution is discrete-stable.
Answer: If a random variable $X$ is infinitely divisible, then for each $n$ we can write $$X = Y_1 + \cdots + Y_n$$
for some i.i.d. random variables $Y_i$.  (Infinitely is a bit of a misnomer here -- all we really mean is that $n$ can be arbitrarily large.)
If $X$ is stable, then we can moreover do this in such a way that the $Y_i$ have the same distribution as $X$, up to scaling.


Comparing Graduation Rates

Question: I have multiple student graduation rates from the same school.  Each year, there are variations in the student population and each year there are different graduation rates.  Is there a way to compare the graduation rate from year to year to see the following:

Graduation rate is not changing due to random chance?
The population levels are not effecting the scores?

To expand on #2, I mean, lets say in year 1, I have 50 children, then in year 2, I have 55 children.  This means the graduation rate in year 2, each child weighs a bit less than year 1.
Thank you,
Answer: Take $p$ as the probability of a single student graduating (Bernoulli distribution), then the number of students graduating would be a binomial distribution, $Binom(N, p)$; so basically you are testing if $p$ is constant from one year to another. One way to test this is likelihood ratio test.
The reduced-model (or the null model) is the simpler model where $p$ is constant. You can easily find the estimate $\hat{p}$ under this estimation by MLE or method of moments, knowing that the mean of distribution is $p \times N$, and calculate the likelihood value using this estimate of $p$.
There would be multiple ways to define the full-model (or the alternative model); one would be to divide the sample points into halves, and take $p_0$ as the probability of graduation for the first half, and $p_1$ as the probability of graduation for the second half. Estimate each $p_i$ separately, and calculate the likelihood as the product of the likelihood of the first half with $\hat{p}_0$ and the second half with $\hat{p}_1$. (or, alternatively one can say log likelihood is the summation of log likelihood of the first half and the second half).
Finally, you can calculate the log-likelihood ratio of the two models as shown here and make inference on the full vs reduced model.
Comment: Thanks @behzad.nouri, im going to have to do some research on these methods to get them working.  I'll post my results.

Comparing Graduation Rates


How to derive the GMM log-likelihood formulation in the eigenvoice modeling technique?

Question: Given a GMM with mean $M=[M_1, M_2, ..., M_C]$ and covariance $\Sigma=[\Sigma_1, \Sigma_2, ..., \Sigma_C]$ (where $C$ is the number of mixtures),
many papers on eigenvoice modeling states that the log-likelihood of an arbitrary speech utterance $X=[X_1, X_2, ..., X_T]$ with $T$ frames is defined as
$\sum_c(-N_c\log(2\pi)^{F/2}|\Sigma_c|^{1/2}-0.5\sum_t(X_t-M_c)^T\Sigma_c^{-1}(X_t-M_c))$
where $N_c$ represents the number of frames aligned to the $c_{th}$ Gaussian mixture, and $F$ represents the dimensionality of $X_t$.
This formulation seems pretty odd to me because as far as I know, when computing the likelihood of the GMM, the weighted mixture-level likelihoods are added as follows
$\sum_c(\pi_cN(X|M_c, \Sigma_c))$
So I'm really confused why the summation on $c$ is outside the logarithm term in the first equation, and where $N_c$ came from.
Any help or opinions are appreciated.
Thanks!
PS. the paper is : P. Kenny, G. Boulianne, and P. Dumouchel, "Eigenvoice modeling with sparse training data," IEEE Transactions on Speech and Audio Processing, vol. 13, no. 3, pp. 345--354, 2005.
Comment: You'd better site the paper where you saw the formula. It seems ok, the first term is just a common denominator which is extracted from the sum on t. N_c is strange indeed.
Comment: Thanks for the reply! the paper is: P. Kenny, G. Boulianne, and P. Dumouchel, "Eigenvoice modeling with sparse training data," IEEE Transactions on Speech and Audio Processing, vol. 13, no. 3, pp. 345--354, 2005.
Answer: In the paper the text after equation is important.

where $s$ ranges over all speakers in the training set, $c$ ranges over all mixture components and for each pair $(s, c)$ the sum over $t$ extends over all frames aligned with $c$.

So actually in the formula the sum over $t$ must be written as
$$
\sum_{t \in A_c} (X_t-M_c)^T\Sigma_c^{-1}(X_t-M_c)
$$
Where $A_t$ is a set of frames aligned with $c$ and $|A_t| = N_c$. In that case the formula is correct and is basically an expansion of log of GMM probability.
Comment: Ok, then that makes this equation clear. Thanks for your help!


How to get url and description of FileFieldItemList's items in node template

Question: I have a content type called publication which results the node's twig template: node--publication.html.twig.
This content type contains a multiple file fields where the description is allowed.
I would like to share it with you, because it took a lot of time for me to find out how can I iterate through and render the items of a FileFieldItemList which would be a very easy thing without the drupal theming system.
Some of you might find useful it.
Answer: I am able to access the uri, and I can generate url from it by a Twig filter.
{{ content.field_document.0['#file'].uri.value|urlFromUri }}
{{ content.field_document.1['#file'].uri.value|urlFromUri }}
{{ content.field_document.2['#file'].uri.value|urlFromUri }}

and the description:
{{ content.field_document.0['#description'] }}
{{ content.field_document.1['#description'] }}
{{ content.field_document.2['#description'] }}

the Twig filter:
<?php

namespace Drupal\something_general\TwigExtension;

use Drupal\Core\Url;

class Custom extends \Twig_Extension
{

  /**
   * Generates a list of all Twig filters that this extension defines.
   */
  public function getFilters()
  {
    return [
      new \Twig_SimpleFilter('urlFromUri', array($this, 'UrlFromUri')),
    ];
  }

  /**
   * Gets a unique identifier for this Twig extension.
   */
  public function getName()
  {
    return 'custom.something_general.twig_extension';
  }

  /**
   * @param $uri
   */
  public static function UrlFromUri($uri)
  {
    $realpath = drupal_realpath($uri);
    $path = str_replace($_SERVER['DOCUMENT_ROOT'], '', $realpath);

    return $path;
  }

}

I am also able, to get the number of the items of the FileFieldItemList:
{% for index in 1..node.field_document.getValue|length %}
{% endfor %}

And finally the iteration in the node--publication.html.twig file:
{% for index in 1..node.field_document.getValue|length %}
  {% set document = content.field_document[ (index-1) ] %}

  {{ dump(document['#file'].uri.value|urlFromUri) }}
  {{ dump(document['#description']) }}

{% endfor %}


fullcalendar slot label in Persian

Question: I have implement fullcalendar, I have interval of 10 minutes, it works fine in English version but Persian version is not displaying interval correctly,
Please view the below screen short, if you see is not showing .30 for second slot, same issue for all slots, is any on know the fix for this?

Answer: I am able to solve my issue, thanks you this post, FullCalendar SlotDuration 24:00 and weekNumbers: true alters date formatting
<code>slotDuration: "00:10:00",
slotLabelFormat: [
'[Week] W',
        'hh:mm'
],
</code>

fullcalendar slot label in Persian


Approach to find partial sum formula

Question: The partial sum formula for $$\displaystyle \sum_{k=1}^n \dfrac{k}{(k+1)!}$$ is given by $$1-\dfrac{1}{(n+1)!}$$
I'm interested in the approach to reach there. How do you prove it without induction?
Comment: Maybe you can expand the sum and see if you can cancel out some terms.
Answer: Note that $\frac k{(k+1)!}=\frac{(k+1)-1}{(k+1)!}=\frac1{k!}-\frac1{(k+1)!}$. Therefore, your sum is a telescopic sum.
Comment: Keep practicing and you will.
Comment: I wish if I had the same eyes as you own.
Comment: @user571036 Jose's eyes are indeed excellent, but this is an old question. The neatness of the formula for the partial sums also gives us a bit of a hint as well, as you can view it as $a_1 - a_{n+1}$ for the null-convergent sequence $a_n = \frac{1}{n!}$.
Answer: Here is a probabilistic proof.  Let $[n]:=\{0,1,2,\ldots,n\}$.  Consider the set $S$ of all permutations on $[n]$.  Equip $S$ with the uniform discrete probability measure $\mathbb{P}$.  The event $E$ that a randomly picked permutation $\Sigma$ is not the identity clearly has the probability of $$1-\frac{1}{(n+1)!}\,.$$
Now, we argue this way.  For $k=1,2,\ldots,n$, let $E_k$ denote the event that $$\Sigma(0)<\Sigma(1)<\Sigma(2)<\ldots<\Sigma(k-1)\text{ but }\Sigma(k)<\Sigma(k-1)\,.$$  Then, prove that $E_1,E_2,\ldots,E_n$ are pairwise disjoint events, $\displaystyle\bigcup_{k=1}^n\,E_k=E$, and $$\mathbb{P}\left(E_k\right)=\frac{k}{(k+1)!}\,.$$

Approach to find partial sum formula


Ruby 2.0.0 - Remove the debugger gem installation

Question: I am new to Ruby. I am using Ruby 2.0.0. I want to run an existing project in Ruby 2.0.0. When I try to run "Bundle install", it is trying to install the gem debugger, even I didn't included the debugger gem in the gem file.
How can I remove the debugger gem from installing?
Comment: It could be required as dependency by another gem
Comment: Is your `Gemfile.lock` file have this gem ?
Comment: Try: "gem uninstall debugger" and then use command "bundle install".
Comment: @jonsnow, my gem file dont have the gem "debugger".
Comment: @rmagnum2002, how can I find out which gem requires "debugger"?
Comment: @test, u need gem file? or gemfile.lock file?
Comment: @Rajivgandhi you can find this in gemfile.lock
Comment: @rmagnum2002, it is not available in the gemfile.lock..
Comment: Well then, post your Gemfile here or on gist. https://gist.github.com/
Comment: @rmagnum2002, https://gist.github.com/rajivgandhi2010/b684bc6826429f966ccf
Answer: You can run gem dependency to show all gems from current Gemfile with their dependencies.
Its highly likely that debugger is required by some other gem in your case.


Slide out element to right, then back in from left

Question: Consider the following piece of HTML:

            Some content

I can make #inner slide out to the right from the screen using the following jQuery:
        $("#slide").animate(
            {
                'marginLeft':'100%'
            },400,
            function(){
                $(this).slideUp('fast');
                $(this).css("margin-right","100%");
                $(this).css("margin-left","0");
            }
        );

However, how can I make that same element, with new content (from AJAX response), slide back in from the left?
I was thinking about resetting the margins (from margin-left:100% to margin-left:0; margin-right:100%) while it was out of view and then use an animation to slide it in from the left:
        $("#slide").animate(
            {
                'marginRight':'0'
            },400,
            function(){
                $(this).slideDown('fast');
            }
        );

This slides it back into view, but not from the left of the screen. Any ideas? I got the .slideUp() from a different StackExchange question but don't know why it's needed for a horizontal slide.
Comment: Any chance you could make a fiddle?
Answer: Simply reset the margin using .css() method:

$(function() {
  $("#outer").click(function() {
    $(this).animate({
      'marginLeft': '100%',
      'opacity': 0
    }, 200, function() {
      //==================================
      //call synchronous ajax here or move this block
      //to the ajax done callback function

      //some ajax change
      $(this).css({
        "background": "blue"
      }).text("came back with new content");

      $(this).css({
        'marginLeft': '-100%'
      });
      $(this).animate({
        'marginLeft': '0',
        'opacity': 1
      }, 200);
      //=================================
    });
  });
});
#outer {
  width: 100px;
  height: 100px;
  display: block;
  background: red;
}

Click Me

Comment: Excellent, seems as though I was almost there. Thank you.
Answer: Reset the margin using .css, check it out here: https://jsfiddle.net/gdjypc7w/2/
JS
function getSummary(id) {
    $.ajax({

        type: "GET",
        url: 'Your URL',
        data: "id=" + id,
        success: function (data) {
            // the data
            $('#summary').html(data);
        }

    });

}

$("button").click(function () {
    $("#inner").animate({
        marginLeft: $("body").width()
    }, 1000, function () {
        //$("#content").text(getSummary(id); Use your id here to retrieve new data
        $("#content").text("New content");
        $("#inner").css("margin-left", "-100%");
        $("#inner").animate({
            marginLeft: "0px"
        }, 1000);
    });
});

Comment: At first, your fiddle made it slide back in from the right instead of the left, so I marked the other one as answer, but this works as well! Thanks.
Comment: No problemo, my pleasure :)


Jordan Decomposition of Dual Space of Function Space

Question: $\textbf{Background:}$
When $X$ is a locally compact Hausdorff space, there is a Jordan decomposition theorem for the dual of $C_{0}(X,\mathbb{R})$.  In particular, if $I\in C_{0}(X, \mathbb{R})^{*}$, then we may find positive linear functionals $I^{+}$ and $I^{-}$ so that $I = I^{+} - I^{-}$.
One might guess that something like this is always true.  If $B$ is a Banach space consisting of real-valued functions $f:X\rightarrow\mathbb{R}$, then we may define positive functionals of $B^{*}$ to be the functionals $I\in B^{*}$ so that if $f\ge 0$ then $I(f)\ge 0$.
$\textbf{Question:}$
(1) With $B$ as above, can you always write $I\in B^{*}$ as $I^{+} - I^{-}$?  Is there an illustrative counterexample?
(2) If this is false in general, is there an example of a more general theorem than the one given at the very beginning of this post?
$\textbf{Remark:}$ I believe that Decomposition of functionals on sobolev spaces might give a counterexample, but I have not studied the duality of Sobolev spaces.
Answer: Say $B=C^1([-1,1])$ and $If=f'(0)$.
It might be interesting to determine hypotheses on $B$ that imply every linear functional is the difference of two positive functionals.
Comment: I think I see what you meant.  If $D$ is the differentiation functional and $D = P - N$ for some $P,N$ positive, then we would have to have $P(1) = N(1)$, and for $f\ge 0$, we would have $P(f - ||f||_\infty)\le 0$, which implies $P(f)\le ||f||_\infty P(1)$, which is a contradiction, since we can make the derivative of a function large without making the function large.
Comment: When you say it "might be interesting," do you mean that you think it is interesting and also know the answer or that you think it's interesting but do not know?  My motivation for this question was looking at the space of bounded measurable functions with the uniform norm.  The continuous functions on this space is the space of finitely additive measures, which has a Jordan (but not a Hahn) decomposition theorem, I think.
Comment: So my next guess will be that we might have a Jordan decomposition when $B$ is any space of real functions with the uniform norm.  Also, thanks for your help.
Comment: @user3281410 I don't know.


Combining funtions with different types

Question: I am writing a method that is executing a variadic list of lambdas on an argument:
public class Main {
    static public<T, R> R convertBy(T data, Function... f) {
        Function composer = Function.identity();
        for (var fi: f ) {
            composer = composer.andThen(fi);
        }

        return (R)composer.apply(data);
    }

     public static void main(String[] args) {
        Function<String, String> l1 = x -> x.toLowerCase(Locale.ROOT);
        Function<String, Integer> l2 = String::length;

        Integer ret = convertBy("Lorem ipsum", l1, l2);
      }
}

Is there a way to use templates and not raw Functions?
By that I mean some kind of way to express this transformation using templates: T -> ... -> R. Or at least get rid of the casting to R in return.
Comment: Logically, you can't chain an arbitrarily-long list of functions unless their input and output types are the same. That is, T must equal R.
Comment: Honestly, an easier way of doing this is to construct a single function using `l1.andThen(l2)`.
Comment: You could probably model this with a heterogeneous list of some sort, but that is very cumbersome in Java (or maybe it's not even possible to achieve with full type safety?). You could do it in Scala 3 though: https://www.scala-lang.org/2021/02/26/tuples-bring-generic-programming-to-scala-3.html. Just chaining `andThen`-calls is much easier :)
Comment: @AndyTurner before calling convertBy?
Integer ret = convertBy("Lorem ipsum", l1.andThen(l2));
This is unfortunately not possible in this case, it must accept a variadic list of functions.
Answer: You can't do this with a variadic argument, because you then need all of the functions to be of the same type: that means that all of the functions have to have the same input and output type, in order that the output of one call can be passed as the input to the next.
In other words, R must be equal to T.
static public<T> T convertBy(T data, Function<T, T>... fs) { ... }

You can pretend to support variadic arguments by providing a number of overloads, for the 1, 2, 3, ... N cases:
static public<T0, T1> T1 convertBy(T0 data, Function<T0, T1> f0) { ... }
static public<T0, T1, T2> T2 convertBy(T0 data, Function<T0, T1> f0, Function<T1, T2> f1) { ... }
static public<T0, T1, T2, T3> T3 convertBy(T0 data, Function<T0, T1> f0, Function<T1, T2> f1, Function<T2, T3> f2) { ... }

Which obviously involves a fair bit of code duplication. (There is prior art for this, for example in the List.of methods)
But other than that, there is no good way of expressing it in a type-safe way.
However, because these aren't actually variadic methods, you can't call them with an array of functions. Given that, it doesn't make much sense to have all of these overloads, because you can just use andThen to chain the functions; and then just use the apply method:
T result = f0.andThen(f1).andThen(f2).apply(data);

Combining funtions with different types


tomcat undeploys application after deleting war file

Question: I am having two web applications one running on tomcat7.0.57 and one on tomcat6.0.3 on amazon EC2 server.My application gets undeployed when following is the case:
-upload .war file
-restart tomcat
-delete war file
Initially on WHM server war file was no more needed after tomcat restart.Please help.
Comment: You need to stop Tomcat before deleting the war file. then after deleting the war file restart tomcat.
Answer: This is correct and expected behavior of Tomcat: WAR file is primary source of application code and if it gets changed or removed, this is treated as request to redeploy or undeploy the application.
If you stop tomcat before deleting WAR file and start it only after that, the application will not be undeployed.
Ideally you should not have the need to delete the WAR file: if you change content of the unpacked web application and don't want it to be redeployed from WAR on next Tomcat restart, probably, it's better to separate dynamic content and move it to some other folder or database.


Pegar o subdomínio com PHP

Question: Como faço para pegar o nome do subdomínio com PHP? Tentei da forma abaixo, mas ele me retorna o domínio principal:
$caminhoAbsoluto = $http . $_SERVER['SERVER_NAME'] . "/";

Por exemplo, tenho o seguinte subdomínio: http://subdiretorio.site.com.br, com o código acima, está trazendo da seguinte forma:
 http://site.com.br

e gostaria de pegar:
http://subdiretorio.site.com.br

Comment: Acredito que a variável `$_SERVER['HTTP_HOST']` retorne ela completa, não?
Answer: Se o virtualhost estiver bem configurado então SERVER_NAME deve funcionar bem, mas creio que o seu caso seja outro meio, como proxy-reverso ou então os hosts para suas aplicações são resolvidos pelo header Host: no HTTP.
Ou seja o valor de SERVER_NAME não é alterado, pois as variáveis SERVER_ geralmente são valores fixos de configurações.
Então para resolver isto você terá que usar o HTTP_HOST, pois as as variáveis com prefixo HTTP_ se baseiam nos headers.
O SERVER_NAME provavelmente não esta "configurado" para cada subdomínio especifico, ou seja o processo é dinâmico e não "fixo" dentro do VirtualHost, por exemplo se cada subdomínio estivesse definido com ServerName e ServerAlias o VirtualHost iria funciona corretamente, exemplo em Apache:
<VirtualHost *:80>
    ServerName dominio.com
    ServerAlias foo.dominio.com
    ServerAlias bar.dominio.com
    ServerAlias baz.dominio.com

    DocumentRoot /public
</VirtualHost>

Mas provavelmente suas configurações são "dinâmicas", ou seja, os seus subdomínios seja resolvidos pelo header mesmo, então neste caso a variável SERVER_NAME só é populado com o alias do host principal, já que as demais não devem existir nas configurações, então o jeito é usar $_SERVER['HTTP_HOST'] que vai pegar o valor de Host:, como no exemplo quando solicitar a página no seu navegador isto vai ocorrer:
GET /foo/bar/baz HTTP/1.1
Host: subdominio.dominio.com
Connection: keep-alive

Então o valor de Host: que é no exemplo subdominio.dominio.com, será populado para o $_SERVER['HTTP_HOST'], ficando assim:
$caminhoAbsoluto = $http . $_SERVER['HTTP_HOST'] . "/";

Nota: suponho que o $http você já tenha, para saber se é HTTP ou HTTPS, se não tiver ainda faça isto apenas:
 $http = !empty($_SERVER['HTTPS']) && $_SERVER['HTTPS'] == 'on' ? 'https://' : 'http://';

 $caminhoAbsoluto = $http . $_SERVER['HTTP_HOST'] . "/";

Answer: Utilize a variável $_SERVER['HTTP_HOST'], ela irá retornar o host completo, com subdomínio inclusive. Logo se usar o comando a seguir, terá o resultado que esperava:
echo 'http://' . $_SERVER['HTTP_HOST']; // Será impresso: http://subdiretorio.site.com.br

E para extrair somente o subdomínio da string, utilize o simples trecho de código a seguir:
echo explode('.', $_SERVER['HTTP_HOST'])[0]; // Será impresso: subdiretorio

Answer: Seria algo do tipo:
echo $caminhoAbsoluto = $_SERVER['HTTP_HOST'] . "/";

se quiseres apenas o subdominio usa o array_shift:
$var = explode('.', $_SERVER['HTTP_HOST']);
echo array_shift($var);

Comment: Caro 13dev, esse uso de explode pode até funcionar no php5.3, 5.4 e cia, mas no php7 com certeza vai falhar, pois `array_shift` espera uma referencia, causando algo como `PHP Notice:  Only variables should be passed by reference`, claro que no geral o PHP ignora e é só um Notice, mas ainda sim o objetivo é usar referencia e não valores em funções que esperam **referencias**: http://php.net/manual/pt_BR/language.references.php
Comment: nem tinha notado, mas já editei a resposta, obrigado pelo aviso!
Answer: Experimente essas funções para ver se resolve o teu problema:
<?php

// COMO COMPLEMENTO PARA PESQUISA DE PROTOCOLO HTTP OU HTTPS
// Apenas para fins de conhecimento
if( !function_exists('server_protocol') ){
    function server_protocol() {
        $protocol = $_SERVER['SERVER_PROTOCOL'];
        if ( ! in_array( $protocol, array( 'HTTP/1.1', 'HTTP/2', 'HTTP/2.0' ) ) ) {
            $protocol = 'HTTP/1.0';
        }
        return $protocol;
    }
}

echo server_protocol() . "
";

#################################################################

// ENDERECO COMPLETO DO SITE COM PROTOCOLO, ENDERECO e /
if( !function_exists('urlbase') ) {
    function urlbase() {
        $protocol = (!empty($_SERVER['HTTPS']) && $_SERVER['HTTPS'] !== 'off' || $_SERVER['SERVER_PORT'] == 443) ? "https://" : "http://";
        $query = $_SERVER['PHP_SELF'];
        $path = pathinfo( $query );
        $url = $protocol . $_SERVER['HTTP_HOST'] . "/";
        return $url;
    }
}

echo urlbase() . "
";

#################################################################

?>

Comment: Serve para qualquer diretório, sendo ele também um subdiretório, retornará a URL completa.

Pegar o subdomínio com PHP


Spring Restful service - replace queryString with dynamic values

Question: I am trying to consume restful web service using RestTemplate class.
Sample code:
String quote = restTemplate.getForObject("http://musicbrainz.org/ws/2/artist/f27ec8db-af05-4f36-916e-3d57f91ecf5e?&fmt=json&inc=url-rels+release-groups", String.class);
            log.info(quote.toString());

In this queryString i want to dynamically pass f27ec8db-af05-4f36-916e-3d57f91ecf5e. How do i do that?
Answer: Any url variable can be wrapped in {} and passed in as a variable. If you have more than one it's replace them in order.
If you need a more complicated logic use UriComponentsBuilder to build your url.
String quote = restTemplate.getForObject("http://musicbrainz.org/ws/2/artist/{dynamic-param}?&fmt=json&inc=url-rels+release-groups", String.class, "f27ec8db-af05-4f36-916e-3d57f91ecf5e");


Surface integral question help.

Question: If $S$ is the surface of the sphere $x^2 + y^2 + z^2 = a^2$, compute the value of the surface integral
 $$\iint_S xz\,{\rm d}y\,{\rm d}z + yz\,{\rm d}z\,{\rm d}x + x^2\,{\rm d}x\,{\rm d}y$$
Answer is given as zero
Comment: Are you aware of the [divergence theorem](http://en.wikipedia.org/wiki/Divergence_theorem)?
Comment: Have you tried using it for this problem?  Where are you stuck?
Comment: I am little confuse with notations dydz , dzdy and dxdy.Usually they have only dx,dy and dz.
Answer: Clarification of the notation: what they're asking for is the surface integral of the vector-field given by
$$
F(x,y,z) = \langle xz, yz, x^2 \rangle
$$
over the surface $x^2 + y^2 + z^2 = a^2$.  If you use the divergence theorem, this amounts to calculating
$$
\iiint_V  (\nabla \cdot F)\,dx\,dy\,dz
$$
Comment: Now I have got it.I was just confused with those notations.Thank you.

Surface integral question help.


Is there a way to get the optimal cutoff points based on probability of topic models and the outcomes?

Question: I have topic models probability obtained using LDA topic models method. I'd like to use these probabilities for 5 topics to predict an outcome (0/1). I'd like to find an optimum cutoff point of each topic to the outcome (0/1). I was thinking about ROC but it seems like it may not work since I don't have the predicted outcome. Are there any methods that allows me to find the cutoff points for each variable, or does it make sense to do it?
Background

We collected open ended responses from participants and performed the topic modeling.
we collected outcome of choice from these participants e.g., intent to leave, performance, etc
hypothesis is that the response the participants enter is indicative of the outcome.

The question is: at what points in each of these topic model probabilities the outcome value changes e.g., from 0-1 or 1-0.
Comment: Please provide more information, in particular what you mean by "I was thinking about ROC but it seems like it may not work since I don't have the predicted outcome." Do you know the actual `outcome` that you're trying to be able to predict for each case? Are the 5 topics mutually exclusive and comprehensive, so that for any one case the probabilities add up to 1? How does the `outcome` differ from the topic probabilities that you've already modeled?
Comment: @EdM, really good questions. I've updated my question to give a brief background. The topics are not mutually exclusive or independent. Hence my problem with using classification. But I've been trying to find a way to analyze these kinds of data. One thing I'm curious about is to see if it's possible to get the optimum cutoff points where if the topic probabilities for one topic is higher or lower than the cut point, then gives the outcome. Much like sensitive/specificity.
Answer: It's seldom that there is a single "point" at which the "outcome value changes." There's usually a gradation with respect to a predictor in terms of the probability of a particular outcome.
So what usually works best for a binary outcome is a probability model for the outcome that takes as much information into account without overfitting. Furthermore, with binary outcomes there's a particular risk of omitted-variable bias; if you leave out any predictor associated with outcome it can make it harder to identify other predictors associated with outcome.
In that context, looking for separate cutoffs for each of the predictors based on ROC is throwing away the detailed information about how probability of outcome changes with each topic's probability. It's also treating the topics separately instead of together.
So instead of looking at your topic probabilities separately with respect to outcome, it would seem to make the most sense to combine them into a single model. A simple model might be a logistic regression model that includes each topic probability as a predictor, which in R might be written for a 0/1 outcome like:
glm(outcome ~ T1 + T2 + T3 + T4 + T5, family = "binomial")

where T1 etc represent the probabilities found for the corresponding topic mappings. That provides a probability model for outcome that uses all the probability information about each of the topics together at once. In principle that could be extended to a mixed model in which you allow for differences among individuals.
Then you can get an ROC that combines information from all of your topic mappings at once. You then use your knowledge of the subject matter and the goals of your project (e.g., how much more costly are false negatives than false positives for you) if you need to choose a probability cutoff that best represents the "point" at which the "outcome value changes."
Comment: @Kuni that's a standard issue in regression. Predictors, like your topic probabilities, are often correlated. That can make the precision of individual regression-coefficient estimates worse than you might want, but it doesn't necessarily make the regression model overall much worse at prediction. The big problem is if the set of predictors is close to linearly dependent, which is why I asked earlier whether all the probabilities add up to one. If they do, or come close to that, you might want to omit one predictor or use a penalized approach like ridge regression to handle the problem.
Comment: Got it! super helpful. Thank you
Comment: thank you for an awesome answer. It is really helpful. One question: can I do logistic regression even if the topic probabilities may not be independent?


Shortest distance between z axis and the line $x+y+2z=3, 2x+3y+4z+4=0$

Question: z axis is : $x=0=y$
$x + B(y) = 0$          -->            (1)
$x+y+2z-3+ A (2x+3y+4z+4 ) = 0$              -->  (2)
These are two planes constructed through the two lines. We find the parallel planes so that it is easier to calculate the distance after that.
Therefore, for planes to be parallel :
$\frac{1+2A}{1} $  = $\frac{1+3A}{B}$ = $\frac{2+4A}{0}$
But now no values of A,B will satisfy this.
How to proceed further ?
Comment: possibly duplicate for: http://math.stackexchange.com/questions/13734/how-to-find-shortest-distance-between-two-skew-lines-in-3d?rq=1
Comment: Yes it is. But applying the method to this does not give the answer!
Answer: Solving the given equations for $x$ and $y$ in function of $z$ gives $x=13-2z$, $y=-10$. It follows that $$\sqrt{x^2+y^2}=\sqrt{(13-2z)^2+100}$$ is minimal when $z={13\over2}$, and the minimal value is $10$.
Comment: I think it's easier to solve for $z$ in terms of $x$ (one still has $y=-10$). Then $x^2+y^2=x^2+100$ which is obviously minimal when $x=0$. (Though in either case the labor is minimal.)
Answer: $${1+2A\over 1}={1+3A\over B}={1+4A\over 0}$$ Has the solution $$A=-{1\over 4},B={1/4\over 1/2}={1\over 2}$$ As far as 3d geometry is concerned!
Comment: Are you suggesting that $\frac{0}{0}$ is defined and is equal to $\frac{1}{2}$..?
Comment: I am sorry. I had written the wrong equation from above given equations. Edited!
Comment: Can you please resolve it. Sorry for the confusion.
Answer: The intersection of the given planes is the line
\begin{align*}
\mathbf{r} = 13\mathbf{i} - 10\mathbf{j} + t(-2\mathbf{i}+\mathbf{k})
\end{align*}
The vector perpendicular to $\mathbf{k}$ (vector along $z$ axis) and the above line is $\mathbf{k} \times (-2\mathbf{i}+\mathbf{k}) = -2\mathbf{j}$. Hence the shortest distance is given by the projection of the line joining $(0,0,0)$ and $(13, -10, 0)$ on the vector $\mathbf{j}$ and this is 10.
Comment: Hey! thanks for the answer. If possible, can you also solve by above method ?
Comment: When $A = -\frac{1}{2}$ we get the plane $y-10 = 0$ that contains the given line. Note that $y=0$ is also a plane containing the $z$ axis and parallel to this plane. Hence the shortest distance is 10.
Comment: Thanks! It works this way.
Comment: For A = -1/2, we get B as infinity. Which means (1) is reduced to $y=0$ by dividing by B. Thanks! This is the solution I needed.


Calculating the distribution of the sum of the squares of the predictors in linear regression

Question: I'm calculating the distribution of the sum of the squares of the components of the MLE $\hat{\beta}$ in linear regression with normal errors. We are assuming that $\beta = 0$. The distribution of the MLE is therefore $\hat{\beta} \sim N(0, \sigma^2 (X^TX)^{-1})$ where $X$ is the design matrix of the model and $\sigma^2$ is the variance of the normally distributed errors.
I want to show that the distribution of $\frac{||\hat{\beta}||^2}{\sigma^2}$ is $\sum_{i=1}^{n-p}\lambda^{-1}W_i$, where the $W_i$ are $\chi_1^2$ random variables, and the $\lambda_i$ are the eigenvalues of $X$ (in this problem we are given that $X$ is full rank and has an eigendecomposition into $X = UDV^T$).
I've already calculated up to $\hat{\beta} \sim N(0, \sigma^2(X^TX)^{-1}) = N(0, VD^{-2}V^T)$ but I don't know how to proceed to find the distribution of $||\hat{\beta}||^2 / \sigma^2$, could anyone help with this?
Answer: To be accurate, "$X = UDV^T$" is not the "eigendecomposition", but "singular value decomposition (SVD)" of $X$. And with this SVD of $X$, it must be made clear that the size of $D$ is $n \times p$ (assuming $X$ is $n \times p$), and $U$ and $V$ are order $n$ and order $p$ orthogonal matrix respectively.  So the notation "$D^{-2}$" does not make sense (unless you are using the "Thin SVD" in which $U$ is $n \times p$ and $D$ is $p \times p$ as opposed to the common "Full SVD" as above).  The correct way is expressing $D$ as $D = \begin{bmatrix}\Lambda \\ 0\end{bmatrix}$, where $\Lambda = \operatorname{diag}(\sigma_1, \ldots, \sigma_p)$ is an order $p$ diagonal matrix with $\sigma_1 \geq \cdots \geq \sigma_p > 0$ (which are called singular values of $X$.  The relationship between the eigenvalues $\lambda_j$,
$1 \leq j \leq p$ of the matrix $X^TX$ and $\sigma_j$ are given by $\lambda_j = \sigma_j^2$). Therefore, the true distribution of $\hat{\beta}$ is
\begin{align}
\hat{\beta} \sim N_p(0, \sigma^2V\Lambda^{-2}V^T),
\end{align}
which implies that $\sigma^{-1}\hat{\beta} \overset{d}{=} A\xi$, where $\xi \sim N_p(0, I_{(p)})$ and $A$ satisfies $A^2 = V\Lambda^{-2}V^T$.  In addition, since $V$ is an order $p$ orthogonal matrix, $V^T\xi := W \sim N_p(0, I_{(p)})$. It then follows that
\begin{align}
\sigma^{-2}\|\hat{\beta}\|^2 \overset{d}{=} \xi^TA^2\xi = \xi^TV\Lambda^{-2}V^T\xi =  W^T\Lambda^{-2}W = \sum_{j = 1}^p\sigma_j^{-2}W_j^2 = \sum_{j = 1}^p\lambda_j^{-1}W_j^2.
\end{align}
Therefore, the result you presented is incorrect in that:

There are $p$ summands, not $n - p$ summands. To see this more clearly, just consider the simplest $X = \begin{bmatrix}I_{(p)} \\ 0\end{bmatrix}$, for which case $\hat{\beta} \sim N_p(0, \sigma^2I_{(p)})$.
$\lambda_j$ is the eigenvalue of $X^TX$, not of $X$. In fact, as a rectangular matrix, it is impossible for $X$ to have eigenvalues.
Comment: @whuber I am not really "criticizing/distracting" OP.  IMO, I have to clarify all the dimensions and basic concepts first before answering this question correctly. For the dimensionality of $D$ (either $n \times p$ in my answer or $p \times p$ as you suggested), I believe I follow the widely-accepted SVD formulation (https://en.wikipedia.org/wiki/Singular_value_decomposition).  I do have seen some author made $D$ to be $p \times p$, but in that case the dimension of $U$ needs to be adapted accordingly.  But I see your point and will edit the wording a bit.
Comment: (+1) But the complaints about notation are distracting. There is a standard formulation of SVD in which "$D^{-2}$" is often used to refer to the $p\times p$ (not $n\times n$) matrix whose *nonzero* diagonal elements are $d_i^{-2},$ where $d_i$ are the nonzero diagonal elements of $D.$  Although that needs clarifying, it could be done a little less critically.  It might also be worth pointing out that you used the full-rank assumption about $X$ when you concluded there are $p$ terms in the sum.
Comment: As for the full-rank assumption, it was clearly mentioned in OP's question.
Comment: Right: my point was only that it could be helpful to point out *how* you used the full-rank assumption in this solution.  My comment about being critical originates from a general sense of negativity in the post, beginning with stating what $X=UDV^\prime$ is *not* and culminating with a summary of what is *incorrect.* The same points can be made in the same (mathematical and statistical) ways from a positive perspective. Just start off by saying what the SVD really is and end by summarizing what the correct result is. IMHO, such a positive approach is less likely to be misread.
Answer: If you know the mean μ=0 and variance (let's call it Σ) of β, then you can compute the expected value and variance at least.
The expected value is tr(Σ).
The variance is 2 tr(Σ ⋅ Σ).
https://en.wikipedia.org/wiki/Quadratic_form_(statistics)?wprov=sfti1
This distribution is a Generalized Chi ² distribution
https://en.wikipedia.org/wiki/Generalized_chi-squared_distribution?wprov=sfti1
It may be worth considering if you really want the sum of squares ( β' β ) , or if you'd be just as happy with β' Σ⁻¹ β.  That is a lot easier.  It follows a standard Chi² distribution I think.


Inserting checkbox values into MySQL Database with PHP

Question: everyone.
I know this is a far treated subject, but I've looked everywhere and  cannot get it solved.
I have a series of checkboxes in my webpage and need to insert the values into a database.
For what I've read, it's supposed to be done this way:

        Salón Completo con:

            <input name="equipSalon[]" type="checkbox" id="equipSalon[]" value="mesa"  />
            mesas

            <input name="equipSalon[]" type="checkbox" id="equipSalon[]" value="sillas"  />
            sillas

            <input name="equipSalon[]" type="checkbox" id="equipSalon[]" value="sofa"  />
            sofá

And then, in the PHP script that the submit button takes to:
 $equipSalonF=mysql_real_escape_string(implode(',', $_POST['equipSalon']));
mysql_query("INSERT INTO markers (ciudad,
                            zona,address,name,
                            telefono,email,piso,
                            tipo,erasmus,nhabitaciones,
                            plazas,equipHabita,nbanos,
                            salon,cocina,electrodomesticos,
                            garaje,internet,calefaccion,
                            sexo,precio,superficie,otros,
                            fecha,lat,lng)
        VALUES ('{$_POST['ciudad']}','{$_POST['zona']}',
                '{$_POST['address']}','{$_POST['name']}','{$_POST['telefono']}',
                '{$_POST['email']}','{$_POST['piso']}','{$_POST['tipo']}',
                '{$_POST['erasmus']}','{$_POST['nhabitaciones']}',
                '{$_POST['plazas']}','{$equipHabitaF}',
                '{$_POST['nbanos']}','{$equipSalonF}',
                '{$equipCocinaF}','{$equipElectroF}','{$_POST['garaje']}',
                '{$_POST['internet']}','{$_POST['calefaccion']}',
                '{$_POST['sexo']}','{$_POST['precio']}',
                '{$_POST['superficie']}','{$_POST['otrosF']}',
                '{$_POST['fecha']}','{$_POST['lat']}',
                '{$_POST['lng']}'",$link);

I have more than this checkboxes and this doesn't work, just says "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 21"
If anyone wants to look at the full index.html and php files, you can have them here: http://paste.ideaslabs.com/show/8qEHDnsZdr the form starts at line 147. And the PHP File in here: http://paste.ideaslabs.com/show/IJFLSHM7Ka
Thanks in advance.
Comment: Except you should remove the ID attributes. You only need to do that for the name attribute.
Comment: if you really want to see what is happening under the hood, dump the POST variable:  \n"; ?>   ... then use view source to see the debug in comments
Answer: Checkboxes are only posted when they are ticked. So if the checkbox is not ticked, is will not appear in $_POST. Also you should generally not give any values to checkboxes. Instead use names to distinguish between them.
In the database I usually represent checkboxes with tinyints and store 1 for checked and 0 for unchecked.
// If your checkbox name is foo, this will convert it
// into a value that can be stored in the database
$foo = isset($_POST['foo']) ? 1 : 0;

Also note that html requires ids to be unique. So you can't have multiple elements with the same id. And you should sanitize your inputs to prevent sql injection. Use mysql_real_escape_string() on user inputs that go in the database.
Update
The main issue is that the query is missing a ')' at the last line.
The query should look like
$query = "INSERT INTO markers (ciudad,
                        zona,address,name,
                        telefono,email,piso,
                        tipo,erasmus,nhabitaciones,
                        plazas,equipHabita,nbanos,
                        salon,cocina,electrodomesticos,
                        garaje,internet,calefaccion,
                        sexo,precio,superficie,otros,
                        fecha,lat,lng)
    VALUES ('{$_POST['ciudad']}','{$_POST['zona']}',
            '{$_POST['address']}','{$_POST['name']}','{$_POST['telefono']}',
            '{$_POST['email']}','{$_POST['piso']}','{$_POST['tipo']}',
            '{$_POST['erasmus']}','{$_POST['nhabitaciones']}',
            '{$_POST['plazas']}','{$equipHabitaF}',
            '{$_POST['nbanos']}','{$equipSalonF}',
            '{$equipCocinaF}','{$equipElectroF}','{$_POST['garaje']}',
            '{$_POST['internet']}','{$_POST['calefaccion']}',
            '{$_POST['sexo']}','{$_POST['precio']}',
            '{$_POST['superficie']}','{$_POST['otrosF']}',
            '{$_POST['fecha']}','{$_POST['lat']}',
            '{$_POST['lng']}')";
mysql_query($query, $link);

Note the closing ')' on the last line of the query. Also note that creating the query like this, in a variable, allows you to output the created query so you can see what exactly is sent to MySQL and you also can test your query in a different environment (ie in phpmyadmin or any other database administration tool).
Comment: You may be able to get a proper value in your `$_POST` for a checked checkbox, but if the checkbox is not checked then it will never be posted. So you have to implement logic to see which values have been posted and use that to determine which values have not been posted.
Comment: Thanks for the answer, but I'm trying to store the values of the checkboxes, not if they're checked or not.

I want to make a string with the values of the checkboxes and store them into de MySQL DB. ¿Is this possible?
Comment: You should more carefully look the code he's posting. He's quoting the $_POST values, so there shouldn't be any problems, no matter whether the checkboxes are checked or not.
Comment: @periklis, quoting values is good, but inputs also need to be escaped to prevent sql injection.
Comment: I totally agree with you, but what does that have to do with the user's question?
Comment: @periklis, escaping does not solve the problem, but imho it should be mentioned if it is not done. And you might want to check my updated answer.
Comment: @Arjan Thanks a lot!

And yes, I will escape inputs, this is just testing.