Self-studying the normal linear model: what can I do with it?

Question: I am currently self-studying the normal linear model ($X_i \sim N(A \beta, \sigma^2I)$), and have learned about estimating $\beta$ and $\sigma^2$, testing certain types of hypothesis (although not too complicated ones), creating confidence intervals, and making predictions (my study has been mathemathically rigorous, so even though most of these things are explained in basic courses, my understanding is fairly deep).
My question is, what comes "next" within this framework? Should I move on to other parts of statistics, focus on other models with different distributions, or is there still more depth in the normal setup? I'm having a hard time figuring out what more I can get out of it other than the things I already mentioned.
Comment: You could focus on generalized linear models now
Comment: Welcome to CV. In one sense, statistics is like karate training...you're always reviewing the basics and that is the foundation for higher levels of achievement. One piece that may expand your horizons is this *Edge* article about Richard Nisbett's *Crusade Against Multiple Regression*.  https://www.edge.org/conversation/richard_nisbett-the-crusade-against-multiple-regression-analysis
Answer: This is by no means authoritative, but hopefully it gives you somewhere to start.
You've studied the model $Y = X\beta + \varepsilon$ with $\varepsilon \sim \mathcal N(0, \sigma^2 I)$.
There are many extensions of this model. Some are:

Explore different assumptions on $\varepsilon$. Maybe you only assume $E(\varepsilon) = 0$, $Var(\varepsilon) = D$. What happens if $D \neq \sigma^2 I_n$? If $D$ is diagonal you can get weighted least squares; if $D$ is not even diagonal then you can use generalized least squares. Also, what happens if $\varepsilon$ has specified but not normal distributions (Laplace, for example)? This turns out to be closely related to considering loss functions other than squared loss.
Consider other distributions of $Y_i$. This can lead to GLMs
Explore penalized regressions. This is a big area. I'd start with the lasso and ridge regression.
Ridge regression and the lasso involve tuning parameters. This leads you to learning about cross validation. Or maybe you'd prefer to use information criteria like the AIC and BIC. There's a lot to study here, such as where exactly they come from and asymptotic relationships with different kinds of cross validations. Significance testing also gets tricky. Maybe you'll need to use the bootstrap more often.
Ridge regression and the lasso also have Bayesian interpretations. This can lead you to going full Bayesian and investigating the effects of various priors and multilevel models. This might require the use of MCMC which is a big thing in and of itself.
Very closely related to the previous point is the delightful world of mixed effects models, which is also very closely related to generalized least squares
Maybe you've got continuous predictors and want to use polynomials in them, but you don't like global polynomials. You can consider basis expansion methods like splines. I'd suggest you take a look at the Elements of Statistical Learning (available free online).
From here you could study kernel methods in general; this includes methods like the support vector machine.

Now you've got both simple interpretable tools like iid normal-errors linear regression, nonparametric flexible methods like SVM, and many things in between.


Least-squares solution to a linear matrix equation

Question: Let $\\A$ be a matrix of size $\\(m, n)$, $\\b$ a column vector of size $\\m$, $\\x$ a column vector of size $\\n$ and $\\a$ a real number.
If $\begin{bmatrix}
x \\
a
\end{bmatrix}$ is the least-squares solution to the linear matrix equation:
$$
\begin{bmatrix}
A &  \begin{matrix}
-1 \\
-1 \\
... \\
-1
\end{matrix}
\end{bmatrix}
\begin{bmatrix}
x \\
a
\end{bmatrix} = b
$$
does it mean that the arithmetic mean of $\\A x - b$ equals $\\a$?
As background information, I used that result to answer this post on stackoverflow.
Comment: Is $b$ a column vector whose entries are all the scalar value $b$?
Comment: It is a column vector, but the values can be different.
Answer: Yes, this is actually a restatement of the following well-known fact: if the design matrix contains a column of all ones (or equivalently, all negative-ones), then the sum of the residuals is zero. For a short proof,
let $e$ denote the column of all ones, $X = [A, -e]$, by the nature of least squares, the residual vector $b - (Ax - ae)$ is perpendicular to the linear space spanned by the columns of $X$. In particular, it is perpendicular to the vector $e$, thus,
$$0 = e^T(b - Ax + ae) = an - e^T(Ax - b)$$
or $a = e^T(Ax - b)/n$, which is your assertion.
Answer: Another way to derive the answer based upon the calculus of minimization is given in Show that the least squares line must pass through the center of mass.


Question in the proof of Lusin's theorem.

Question: I want to prove Lusin's theorem.

Let $E\subset\mathbb R$ be a measurable set and $f:E\to\mathbb R$ be a measurable function. Then, for each $\epsilon$, there exists $F\subset E$ s.t. $F$ is closed about relative topology from $E$, $m^\ast(E\setminus F)<\epsilon$, $f|_F$ is continuous. ($m^\ast$ is Lebesgue outer measure.)

I've already understand the case $m^\ast(E)<\infty$, and I'm trying general case.
Hint is given : Consider $E_k:=E\cap[-k,k]$.

Let $E_k:=E\cap[-k,k]$.
For each $k\in\mathbb N$, $m^\ast(E_k)<\infty$ and $f|_{E_k}$ is measurable thus there is $F_k\subset E_k$ s.t. $F_k$ is closed about relative topology from $E_k$, $m^\ast(E_k\setminus F_k)<\epsilon/2^k$, $f|_{F_k}$ is continuous.
$F:=\cup_{k=1}^\infty F_k$, $E:=\cup_{k=1}^\infty E_k$.
Then $F\subset E$.
Since $E\setminus F\subset\cup_{k=1}^\infty (E_k\setminus F_k)$, I get $m^\ast(E\setminus F)\leqq\sum_k m^\ast(E_k\setminus F_k)\leqq\epsilon.$
Maybe what I did so far is correct, but this is not sufficient yet : I have to check
(i) $F$ is closed about relative topology from $E$.
(ii) $f|_F$ is continuous.
For (i), each $F_k$ is closed in $E_k$ but I don't know why $F=\cup_k F_k$ is closed in $E$. $F$ has to be written as $F=E\cap C$, where $C$ in closed in $\mathbb R$. I haven't found such $C$.
For (ii), each $f|_{F_k}$ is continuous but I don't think this guarantees the continuity of $f|_F$. I think that "each $f|_{A_k}$ is continuous $\Rightarrow$ $f|_{\cup_{k=1}^\infty}A_k$ is continuous" doesn't generally hold. (Maybe, I have to show $x_j\to x$ $\Rightarrow$ $f|_F(x_j)\to f|_F(x)$, but I don't know how.)
Postscript
Hint says $E_k:=E\cap[-k,k]$ but perhaps $E_k$ has to be defined s.t. $\{E_k\}$ is disjoint : e.g., $E_k=E\cap(k,k+1]$ or, $E_1=E\cap[-1,1]$, $E_k=E\cap([-k,k]\setminus[1-k,k-1])$ for $k\geqq 2.$
Answer: EDIT: I see your postscript. I think defining $E_k=E\cap[k,k+1]$ is a much better idea, personally.

It is important to note that, with no loss of generality, $F_{n+1}\cap[-n,n]=F_n$ can be enforced for all $n\ge1$. This ensures $F$ is the colimit of $F_1\hookrightarrow F_2\hookrightarrow\cdots$ which is what we need for $f$ to be continuous, and also for $F$ to be closed. If you aren't comfortable with that language, ignore it!
$F$ is closed because $F\cap[-n,n]$ is a finite union of $F_1,F_2,\cdots,F_n$ which is closed, for every $n$. It follows $F\cap K$ is closed for every compact subset $K$ of $\Bbb R$ hence $F$ is closed, as $\Bbb R$ is compactly generated.
That's just one way to see it though. We can prove it more directly - suppose $(x_i)_{i\ge1}$ is a sequence of elements of $K$ converging to some $x\in\Bbb R$. If $n=\lceil x\rceil+1$, I know there is some $N$ that $i\ge N$ implies $x_i\in[-n,n]$, hence $x_i\in F_n$ for all $i\ge N$. Considering the sequence $y_j:=x_{j+N}$, we know $(y_j)_{j\ge1}$ is a sequence of elements of $F_n$ converging to $x$. Because this set is closed, I know $x\in F_n\subseteq F$. Thus, $x\in F$ - by the sequential criterion for closure of a set in a metric space, it follows $F$ is closed.
Why is $f$ continuous on $F$? Fix an $x\in F$ and $U$ an open neighbourhood of $f(x)$. I know $x\in F_n$ for some $n$, by definition of union. We may take $n>\lceil x\rceil+1$. Now, continuity of $f$ on $F_n$ implies there is some $1>\delta>0$ that $|y-x|<\delta$ and $y\in F_n$ implies $f(y)\in U$. If $y\in F$ and $|y-x|<\delta<1$, we know $y\in[-n,n]$ hence $y\in F_n$ holds, so this condition extends to: $y\in F$ and $|y-x|<\delta$ implies $f(y)\in U$. Since $U$ is arbitrary, we have $f$ continuous at $x$. Since $x$ is arbitrary, $f$ is continuous over all of $F$.
Comment: I understand there is $N$ s.t. $i\geqq N \Rightarrow x_i\in[-n,n]$, but why does this imply $x_i\in\cup_{k=1}^n F_k$ for $i\geqq N$ ?
Comment: @da We know two things. $x_i\in[-n,n]$ and $x_i\in F$. So, $x_i\in F_k$ for some $k$, $x_i\in F_k\cap[-n,n]$. By my simplifying assumption (which is possible to achieve wlog and is necessary for bits of the proof to work) I know $F_k\cap[-n,n]\subseteq F_n$ for any $k$. So $x_i\in F_n$ for all $i\ge N$ (I know I wrote 'union' in the post, that was before I realise the simplifying assumption is quite important: I'll edit in that change now)
Comment: I put the simplifying assumptions after that paragraph where I really should have written it in before, to emphasise - that's fixed now
Comment: My $F_n$ only satisfies $m^\ast(F_n)<\infty$ and $F_n\subset_{\rm{closed}} E_n$. I cannot see why we can enforce $F_{n+1}\cap[-n,n]=F_n$.
Comment: @daYou can pick the $F_n$ in this way. Do so by induction, say we have chosen the $F_n$ in this way up to step $N$. In picking $F_{N+1}$, I can notice that $E_{N+1}\setminus\mathrm{int}F_n$ is a measure space on which $f$ is measurable, so by the finite Lusin theorem there is a relatively closed $K$ on which $f$ is continuous and $E_{N+1}\setminus K$ has small measure. $K$ is going to be relatively closed in $E_{N+1}$ too, and $F_{N+1}:=K\cup F_N$ is relatively closed in $E_{N+1}$, satisfies that $f$ is continuous on it, and $E_{N+1}\setminus F_{N+1}$ can be made to have small measure
Comment: Does such $F_N$ satisfiy $F_{N+1}\cap[-N,N]=F_N$? I have $F_{N+1}\cap[-N,N]=(K\cup F_N)\cap[-N,N]=(K\cap[-N,N])\cup(F_N\cap[-N,N])$, but does this become $F_N$ ?
Comment: @da Sure, $K\cap[-N,N]$ could be nonempty. But that would mean $A:=F_N\cup(K\cap[-N,N])$ is a relatively closed subset of $E_N$, with the complement having small measure, and $f$ shall be continuous on it. So we may as well have taken $F_N=A$ in the first place. That is, we can take the $F_i$ to be maximal relatively closed subsets in this regard. So we can arrange it to be the case that $K\cap[-N,N]$ is empty.


Attempt to invoke virtual method 'java.util.ArrayList com.orbotix.async.DeviceSensorAsyncMessage.getAsyncData()' on a null object reference

Question: I am new to Android programming (just started for the sake of my Final Year Project) and I am having trouble with my current task on hand. I am trying to access my Sphero's IMU and output the data into a text file. Streaming the data onto my android device was not much of an issue after I learn how to create an android app and playing around with the Samples I got from https://github.com/orbotix/Sphero-Android-SDK.
However I had been stucked on the output data to text phase which I could not seems to solve. I manage to figure out how to output text file for string messages and arrays with DataOutputStream and other outputstream method(buffer, bytes etc). But when it comes to outputting data from the sphero accelerometer streaming, I could not output any file as the app keep crashing on error:
Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'java.util.ArrayList com.orbotix.async.DeviceSensorAsyncMessage.getAsyncData()' on a null object reference
Here are the output part which I believe is causing the problem:
public void write_data() {
    double accelX = ((DeviceSensorAsyncMessage) asyncMessage).getAsyncData().get(0).getAccelerometerData().getFilteredAcceleration().x;
    ArrayList arrayList = new ArrayList();
    while (running == true) {
        arrayList.add(accelX);

        Toast.makeText(getApplicationContext(), "File Open", Toast.LENGTH_LONG).show();
        int sz = arrayList.size();
        for (int i = 0; i < sz; i++) {
            try {
                DataOutputStream dos = new DataOutputStream(new FileOutputStream(output_file, true));
                dos.writeUTF("" + arrayList);
                dos.writeChars("\n");
            } catch (IOException e) {
                Toast.makeText(getApplicationContext(), "Unable to copy data", Toast.LENGTH_LONG).show();
                //e.printStackTrace();
            }
        }
        try {
            dos.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

I am not sure as to the fact that the output method is wrong as I still do not understand the NullPointException Error from past issues. Can someone please advice. Thank you in advance and I would really appreciate any advice.
Answer: For those who happened to be seeking the same answer, I am not good at the explanation of coding as I am new but I managed to resolve the issues by restructuring actions.
Here is the new code that I wrote which finally work as I desired:
public void handleAsyncMessage(AsyncMessage asyncMessage, Robot robot) {
    if( asyncMessage == null )
        return;

    //Check the asyncMessage type to see if it is a DeviceSensor message
    if( asyncMessage instanceof DeviceSensorAsyncMessage ) {
        DeviceSensorAsyncMessage message = (DeviceSensorAsyncMessage) asyncMessage;

        if( message.getAsyncData() == null
                || message.getAsyncData().isEmpty()
                || message.getAsyncData().get( 0 ) == null )
            return;

        //Retrieve DeviceSensorsData from the async message
        DeviceSensorsData data = message.getAsyncData().get( 0 );

        //Extract the accelerometer data from the sensor data
        displayAccelerometer(data.getAccelerometerData());

        //Extract attitude data (yaw, roll, pitch) from the sensor data
        displayAttitude(data.getAttitudeData());

        //Extract gyroscope data from the sensor data
        displayGyroscope( data.getGyroData() );

        writeAccelerometer(data.getAccelerometerData());
    }
}

public void writeAccelerometer(AccelerometerData accelerometer) {
    if (running == true) {
        try {
            dos.writeBytes(String.format("%.4f", accelerometer.getFilteredAcceleration().x));
            dos.writeChars("\n");
            dos.writeBytes(String.format("%.4f", accelerometer.getFilteredAcceleration().y));
            dos.writeChars("\n");
            dos.writeBytes(String.format("%.4f", accelerometer.getFilteredAcceleration().z));
            dos.writeChars("\n");
            System.out.println("Write Successful");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

That is all I have to share and hope it helps.


For what $\alpha \in \mathbb R$ is $\int_{\{\mathbf{|x|} \in \mathbb{R}^d : \mathbf{|x|} \le 1\}} \mathbf{|x|^{- \alpha}} dm(\mathbf{x})$ finite?

Question: I'm studying the Lebesgue integral and have run into trouble with this problem. The integral I'm trying to work with is:
$\int_{\{\mathbf{|x|} \in \mathbb{R}^d : \mathbf{|x|} \le 1\}} \mathbf{|x|^{- \alpha}} dm(\mathbf{x})$
I had tried this and managed to do it for $d =1,2$ by drawing and visualizing it and feel the answer is $\alpha \lt d$. I feel the proof might be inductive but am unable to prove it that way. Another approach, hinted at by a friend I'm working with is to perhaps convert the problem to generalized polar coordinates but I am unable to do it that way. Is there some other, more simple way to do it that I might not be seeing?
Thanks.
Answer: This well explained in the following theorem in Folland's Real Analysis


I'm getting this error while running the following code on sql developer ORA-01008: not all variables bound

Question: DECLARE
    v_start_date INT := 20220302;
    objtosearch varchar2(50) := '%data%';
BEGIN
    WHILE v_start_date <= 20230331 LOOP
        EXECUTE IMMEDIATE ( '
    insert into bazey.data_usage
        SELECT
            ft.id_date,
            vs.nr_orgn,
            SUM(qt_upld_trfc + qt_dwld_trfc) usage_mbs
        FROM
                 (
                SELECT
                    *
                FROM
                    fct_trf
                WHERE
                        ft.id_date = '
                            || v_start_date
                            || '
                    AND x_file_name =: objtosearch
            ) ft
            JOIN voice_data_sample vs ON substr(ft.nr_orgn, - 9, 9) = vs.nr_orgn
        GROUP BY
            ft.id_date,
            vs.nr_orgn
    ' );
        COMMIT;
        v_start_date := to_number(to_char(to_date(v_start_date, 'YYYYMMDD') + 1, 'YYYYMMDD'));

    END LOOP;
END;

Answer:  AND x_file_name =: objtosearch

here you have a : bind variable marker, though you should remove the space between that and the name. When you have binds inside EXECUTE IMMEDIATE, to actually bind it you have to specify the variable using the USING clause. It won't match automatically simply because the name is the same. And if you do this for objtosearch, you really should do the same to start_date:
 EXECUTE IMMEDIATE '
    insert into bazey.data_usage
        SELECT
            ft.id_date,
            vs.nr_orgn,
            SUM(qt_upld_trfc + qt_dwld_trfc) usage_mbs
        FROM
                 (
                SELECT
                    *
                FROM
                    fct_trf
                WHERE   ft.id_date = :start_date
                    AND x_file_name =:objtosearch
            ) ft
            JOIN voice_data_sample vs ON substr(ft.nr_orgn, - 9, 9) = vs.nr_orgn
        GROUP BY
            ft.id_date,
            vs.nr_orgn
    ' USING var_start_date,objtosearch;

If I may suggest, however, you really have no reason at all use to EXECUTE IMMEDIATE at all here. We use dynamic SQL only if the object names (column names, table names) or large structural items (join clauses, query blocks, etc.) need to be programmatically set and can change. If all that's changing is data values you are looking for, don't use dynamic SQL, use normal SQL:
  insert into bazey.data_usage
            SELECT
                ft.id_date,
                vs.nr_orgn,
                SUM(qt_upld_trfc + qt_dwld_trfc) usage_mbs
            FROM
                     (
                    SELECT
                        *
                    FROM
                        fct_trf
                    WHERE
                            ft.id_date = v_start_date
                        AND x_file_name = objtosearch
                ) ft
                JOIN voice_data_sample vs ON substr(ft.nr_orgn, - 9, 9) = vs.nr_orgn
            GROUP BY
                ft.id_date,
                vs.nr_orgn;


ActionFilter to check for ModelState.IsValid is never hit

Question: I wrote an ActionFilter to check if my modelstate is valid (so I can log the issue):
public class ValidationFilter : ActionFilterAttribute
{
    public override void OnActionExecuting(ActionExecutingContext context)
    { ... }
}

My breakpoint is never hit, I do get an output in Postman:

{
      "errors": {
          "RequiredText": [
              "Test - RequiredText is required"
          ]
      },
      "title": "One or more validation errors occurred.",
      "status": 400,
      "traceId": "0HLOBA7E4R7SL:00000002"
  }

I'm asuming that .NET Core does the validation beforehand.
Is there a way to turn this off so I my Attribute is used?
Comment: One of the ways to turn off default validation would be to remove validation attributes from your model. Then, validate on your own in your filter.
Answer: I think you must disable the default behavior by setting SuppressModelStateInvalidFilter option to true. You can set this option to true in the ConfigureServices method. Like,
public void ConfigureServices(IServiceCollection services)
{
    services.Configure<ApiBehaviorOptions>(options =>
    {
        options.SuppressModelStateInvalidFilter = true;
    });
}


RMSE or MAPE? which one to choose for accuracy?

Question: I have a weekly times series for which I would like to find the best fit model. So far I've tried arima, Harmonic regression with arima error, neural network and in the end I would like to decide which one has been better fitted to my raw data. The time series look like this, with heavy seasonal and cyclic pattern:

I also put the Ljung-Box test and the plot of predicted values here for each:
# Arima
fit <- training %>% auto.arima(lambda = 0)
fit %>%
  checkresiduals()

    Ljung-Box test

data:  Residuals from ARIMA(3,1,0)
Q* = 23.619, df = 23, p-value = 0.4252

Model df: 3.   Total lags used: 26

#Harmonic regression with arima error
fit2 <- auto.arima(training, lambda = 0, seasonal = TRUE, xreg = fourier(training, K = 4))
fit2 %>%
  checkresiduals()

    Ljung-Box test

data:  Residuals from Regression with ARIMA(2,1,1) errors
Q* = 21.642, df = 15, p-value = 0.1175

Model df: 11.   Total lags used: 26

#Neural Network
fit3 <- nnetar(training, lambda = 0)
fit3

They all seem fine based on Ljung-Box test but they somehow failed to capture the wiggly form of the time-series here which I don't know how important it is. But my main question is when I check the accuracy if I choose RMSE I have to pick the harmonic regression one and if I choose MAPE I have to pick neural network model. And I would also like to know why RMSE and MAPE values are so different here.
# Arima
accuracy(forecast(fit, h = 16), test)
                    ME     RMSE      MAE       MPE     MAPE      MASE       ACF1 Theil's U
Training set  1.948693 27.56683 19.09467 -4.402578 25.87164 0.5790763 0.21495069        NA
Test set     43.293579 61.02374 46.31065 32.745528 39.26652 1.4044442 0.09636865  1.158678

# Harmonic Regression
accuracy(forecast(h = 16, fit2, xreg = fourier(training, K = 4)), test)
                    ME    RMSE      MAE         MPE      MAPE      MASE      ACF1 Theil's U
Training set  4.323546 24.4800 16.05035   -1.464388  21.89874 0.4867525 0.1751586        NA
Test set     -2.495049 42.1323 33.03114 -171.095704 194.16485 1.0017220 0.2349017  4.288442

# Neural Network
accuracy(forecast(fit3, h = 16), test)

                    ME     RMSE      MAE       MPE     MAPE      MASE      ACF1 Theil's U
Training set  3.414448 22.63083 14.31504 -2.615375 16.93870 0.4341265 0.2253450        NA
Test set     40.095160 58.90628 44.16645 28.908539 37.72779 1.3394181 0.1107563  1.119875

Thank you very much for your help, I really appreciate it in advance.
Comment: Phrases like "best fit" and "accuracy" are not well-defined *until  you say what you want them to mean*. Once you do that properly, your loss function should become clear enough to operationalize. That won't solve the lack of fit problem you perceive with the models you have there, of course, because that's got little to do with the specific loss.
Comment: Be aware that it is hard to forecast the stock market. You will probably not be able to find a simple model that forecasts the VIX with any precision.
Comment: And if you do figure out how to predict the VIX, please tell me (and only me) how to do so :p
Comment: @Dave I will have a look and will be in touch hahaha
Comment: Thank you very much for your comment. Since I am quite naive in this field, can you please elaborate how I could improve my question? or perhaps if you have any recommendations on improving models I would appreciate it.
Comment: @AlexbGoode No it's not stock market, just some sales forecasting lol :)
Answer: Use the RMSE.
Note that the (R)MSE and the MAPE will be minimized by quite different point forecasts (see my answer at Higher RMSE lower MAPE). You should first decide which functional of the unknown future distribution you want to elicit, then choose the corresponding error measure.
However, note that an ARIMA model will output a conditional expectation forecast, i.e., the functional that optimizes (R)MSE. It makes little sense to train a model to minimize the (R)MSE, then to assess its forecasts with a different error measure (Kolassa, 2020, IJF). If you truly want to find a MAPE-optimal forecast, you should also use the MAPE to fit your model. I am not aware of any off-the-shelf forecasting software that does this (if you use an ML pipeline, you may be able to specify any fitting criterion and choose the MAPE), and I have major doubts as to the usefulness of a MAPE-minimal forecast.
Comment: Ah, thank you for the upvote there then, I presume. You might find my IJF paper useful, it's quite short and accessible. Feel free to request it from me on ResearchGate, if you don't have access to the IJF.
Comment: Great! Can you please add a few words in your LinkedIn invitation and refer to this thread? I get a lot of invitations from people I do not know at all, most of which I ignore... Also, [here is a thread on the MAPE that may be interesting](https://stats.stackexchange.com/q/299712/1352), though I'll admit that I keep on saying the same things over and over...
Comment: "It makes little sense to train a model to minimize the (R)MSE, then to assess its forecasts with a different error measure" - is this time-series specific? I was under the impression that in ML one often trains using a computationally/numerically well-behaved loss function but uses another, more interesting, error measure on the holdout data. (Crossing fingers that the two agree to some extent!)
Comment: @AlbertoSantini: I would say that this is bad practice in either case. If my teacher tells my kids they should study lessons 1-5, and then the test is on lessons 6-10, the kids will be justifiably upset - so why should we do the analogue in statistics and ML? (Exception: I'm good with using a smooth approximation to a non-differentiable loss function.) I think this problem is more prevalent in classical forecasting, where most models implicitly yield expectation forecasts - in ML, it's more common to specify your own objective function.
Comment: Thank you very much Mr. Kolassa for your answer. I already visited your other answered today but thought I could ask to be sure that it also the case with my time-series. It's a general rule though.
Comment: I actually doubt it before I post this question, since I saw MR. Rami Krispin used `MAPE` in his book to compare arima and harmonic regression and he based his judgement on `MAPE` so I thought I would ask to be sure. Glad to learn now thanks to 2 very good answers.
Comment: In fact it's a great honor for me to have you here, sure I will take a look and I would also be glad to follow you on linkedin. You are quite an inspiration person to new learners :)
Comment: Oh I am sorry, I just send my request and didn't add any notes. Yes I understand I also answer R questions on stackoverflow and sometimes we have to repeat, but for me at least it helps to review some materials. Thank you very much for your answer and so glad to have met you :)
Answer: Based on visual inspection your time series might have a discernable wave every 1 to 3 months.  If you include longer period terms in your model this will create an extrapolated mean estimate and prediction interval that is more "wiggly."  It's not critical, but it might improve the point predictions.  As you currently have it the vast majority of the observations in the test set fall within the prediction intervals.
MAPE stands for mean absolute percentage error.  It is the average multiplicative effect between each estimated mean and the observed outcome.  RMSE stands for root mean squared error, i.e. standard deviation.  While they both summarize the variability of the observations around the mean, they are not in the same scale so don't expect the values to be similar.  I suggest using RMSE as this is the basis for how the model is fit to the data.
Comment: Thank you very much for your answer Mr. Johnson. I understand totally. I increased the value fo `K` in harmonic regression as I learned from Mr. Hyndman as the line gets more wiggly by increasing `K`, but after a certain level the model fails the `Ljung-Box` test.


Makefile line `test -f `

Question: I have a Makefile with the following line:
 $(CXX) $(CXXFLAGS) $(INCL) -c -o $@ `test -f '$<' || echo '$(SRCDIR)/'`$<

Can you explain me what this line does?
Note: The Makefile is taken from here: https://projects.coin-or.org/BuildTools/wiki/user-examples
That page specifies this:

The generic compilation rules (e.g., .c.o) might look somewhat
  complicated. But with this, the Makefile can also be used by people
  who work with the MS compilers (e.g., cl) under Cygwin, since those
  compilers don't understand the UNIX-type directory paths. In such a
  case, the CYGPATH_W will have been set to "cygpath -w" by configure,
  otherwise it is just "echo".

I am not going to work with MS compilers.
Answer: The test -f thing is a Unix-ism that will return true if the specified file exists and false if it does not. "$<" is the source file for a make rule. So test -f '$<' || echo '$(SRCDIR)/' will check for the existence of the source file and if it does not exist, will echo $(SRCDIR)/ followed by the name of the file. That will be the input to the actual compiler.
It's basically saying "if file X exists in the local directory, then compile it. Otherwise (that's what the || does), compile the file called X located in the directory $(SRCDIR)". This way you can put all your source in a subdirectory and then override certain files by putting new copies of them in the same directory as the Makefile.
Comment: Okay, so let's suppose all the source files are located in the same directory the Makefile is, if I understand well I could replace the line by `$(CXX) $(CXXFLAGS) $(INCL) -c -o $@ '$<'`.
Comment: One more question: Are the ` 's necessary?
Comment: Yes to both of your questions. The backtick tells the shell "execute this command and then insert the result (i.e. stdout from the command) here."

Makefile line `test -f `


Submodularity of function defined on graph

Question: I have a bipartite graph $(V_1\cup V_2, E)$ on which we define $\sigma: V_1\rightarrow\mathbb{Z}$ such that $\sigma(W)=|W|-|\Gamma W|$, where $\Gamma$ gives us the points (from $V_2$, since its a bipartite graph) adjacent to at least one point is $W$.
Now I am asked to prove $\sigma(W_1\cap W_2) + \sigma(W_1\cup W_2) \geq \sigma(W_1) + \sigma(W_2)$.
I have done the following:
$\sigma(W_1\cap W_2) + \sigma(W_1\cup W_2)$
$= |W_1 \cap W_2| - |\Gamma (W_1\cap W_2)| + |W_1 \cup W_2| - |\Gamma (W_1\cup W_2)|$
$= |W_1| + |W_2| - |\Gamma (W_1\cap W_2)| - |\Gamma (W_1\cup W_2)|$
using that $|W_1 \cap W_2| + |W_1 \cup W_2| = |W_1| + |W_2|$ for this last equality.
Now I think that all I have left to prove is that $- |\Gamma (W_1\cap W_2)| - |\Gamma (W_1\cup W_2)| \geq - |\Gamma (W_1)| - |\Gamma (W_2)|$, but I'm not sure how to go about this.
Answer: First note that if $A\subseteq B$ then $\Gamma (A)\subseteq\Gamma(B)$. This gives us $$\Gamma (W_1\cap W_2)\subseteq \Gamma(W_1)\cap \Gamma(W_2)$$ and $$\Gamma(W_1)\cup\Gamma(W_2)\subseteq \Gamma(W_1\cup W_2).$$ But if $v\in \Gamma(W_1\cup W_2)$, then $v$ is adjacent to at least one vertex in $W_1\cup W_2$ and hence $v$ is adjacent to a vertex in $W_1$ which implies $v\in \Gamma (W_1)$, OR $v$ is adjacent to a vertex in $W_2$ which implies $v\in \Gamma(W_2)$. This gives $v\in \Gamma(W_1)\cup\Gamma (W_2)$ and therefore $$\Gamma(W_1)\cup\Gamma(W_2)=\Gamma(W_1\cup W_2).$$
Hence $$|\Gamma(W_1)|+|\Gamma(W_2)|=|\Gamma(W_1)\cup\Gamma(W_2)|+|\Gamma(W_1)\cap\Gamma(W_2)|$$$$=|\Gamma(W_1\cup W_2)|+|\Gamma(W_1)\cap\Gamma(W_2)|\geq |\Gamma(W_1\cup W_2)|+|\Gamma(W_1\cap W_2)|.$$


Nic teaming server 2012 r2

Question: I've been asked to build a server and team the nics, my understanding is the nics should each be dhcp enabled and the team should be configured with a static ip. Would this be correct and also what items should be enabled on the adapter .. Should it only have "network adapter multiplexor protocol" checked? Or should it have IPv4 and link later topology checked
Thanks in advance :)
Comment: Once the individual NICS are in the Team you can't do anything with them can you? certainly settings get ignored anyway.
Comment: "dhcp enabled" or "the team should be configured with a static ip' - which one is it, it's one or the other but can't be both - really it's should be static - and you should ask someone who's done this before to do it, don't mean to be rude but this isn't something to learn in production.
Comment: I'it's fine, no offence taken, I've done this before but always left the nics as dhcp and assigned a static ip to the team, but a colleague has suggested I've done it wrong and the nics should all have their own static ip, however I don't see the function that would serve as once they become part of a team their independent ip is disregarded, hope that makes sense and thanks for the quick response
Comment: That makes no sense at all, sorry
Comment: Lol ok simplified .. Nic 1 and mic 2 both assigned by dhcp .. Combined to make a team called team 1, then team 1 is assigned a static ip .. My question is do nic 1 and nic 2 need to have static ips configured prior to being added to team 1
Comment: No, there (in the eyes of the OS) different connections. You can do but you may as well spend the time making the team and setting the static on that.
Comment: Your colleague is totally wrong. You should not assign different IP addresses to each NIC in the team. After you create the team, you assign IP and VLAN settings to the virtual team interface. The phrase "DHCP enabled" is confusing in this context because in Windows that's the same thing as "not configured". Probably saying "not configured" would make the question clearer.
Answer: Your colleague is totally wrong. You should not assign different IP addresses to each NIC in the team. After you create the team, you assign IP and VLAN settings to the virtual team interface.
Regarding advanced layer 2 features like TCP Chimney Offload and RSS, etc., it depends on the NIC manufacturer whether those features are compatible with Windows NIC teaming and whether they should be configured at all. See here.
If in doubt, leave everything default and/or unconfigured on the NICs to be teamed. Also, I agree with Chopper that if you are building one or more Hyper-V hosts for production purposes for the first time, you should ideally both do a lot of research and consult with or hire a professional, as well as do a lot testing and practice builds. You can put a retail trial of Server 2012 R2 on a desktop computer with an extra NIC or two added in and experiment with settings and practice. You really want to understand how your specific server interfaces and switches work together and separately, as well as understand Hyper-V virtual networking.

Nic teaming server 2012 r2


nodejs and kafka keyed partitioner

Question: I am trying to use keyed partitioning with kafka Highlevelproducer, on a 4 partitions topic
I have the code below:
<code>   var kafka = require('kafka-node'),
      HighLevelProducer = kafka.HighLevelProducer,
      client = new kafka.Client(Host+":"+Port,client_id),
      producer = new HighLevelProducer(client,{partitionerType: 3});
.
.
   theKey = theKey+1;
   if ( theKey > Nb_key ) {
      theKey = 0;
   }
   var payloads = [
        { topic: Topic, key: theKey, messages: JSON.stringify({"hello": "world", "Timestamp": +timestamp}) }
      ];
      producer.send(payloads, function (err, data) {
      });
</code>
I checked that messages have the correct key value, but still, all messages are sent to partition 1.
Do you see anything wrong with what I am doing ?
Thanks
Answer: Looking at the code, I think I understood the problem.
The baseProducer is doing a hash on the key as a string, and the result is always the same, so always same partition.
I wrote my own custom partitioner, doing a simple modulo on the key with the nb of partitions, it works perfectly.
See code below:
<code>   var MyPartitioner = function (partitions, key) {
   key = key || '0';
   var index = parseInt(key) % partitions.length;
   return partitions[index];
   };
   var kafka = require('kafka-node'),
      HighLevelProducer = kafka.HighLevelProducer,
      client = new kafka.Client(Host+":"+Port,client_id),
      producer = new HighLevelProducer(client,{requireAcks: 0,partitionerType: 4},MyPartitioner );
      producer.on('ready', function () {
      timer.setInterval(send_data, '',SendIntervalInUsecs+'u');
   });
</code>

nodejs and kafka keyed partitioner


Log4J able to recover from disk full?

Question: We have several java application server running here, with several apps. They all log with Log4J into the same file system, which we created only for that reason.
From time to time it happens that the file system runs out of space and the app gets
<code>log4j:ERROR Failed to flush writer,
java.io.IOException
</code>
Unfortunately Log4J does not recover from this error, so that even after space is freed in the file system, no more logs are written from that app. Are there any options, programming-wise or setting-wise, to get Log4J going again, besides restarting the app?
Answer: I didn't test this, but the website of logback  states:

Graceful recovery from I/O failures
Logback's FileAppender and all its sub-classes, including
  RollingFileAppender, can gracefully recover from I/O failures. Thus,
  if a file server fails temporarily, you no longer need to restart your
  application just to get logging working again. As soon as the file
  server comes back up, the relevant logback appender will transparently
  and quickly recover from the previous error condition.

I assume the same would be true for the above situation.
Answer: What do you see is an acceptable outcome here? I'd consider writing a new Appender that wraps whichever appender is accessing the disk, and tries to do something sensible when it detects IOExceptions. Maybe get it to wrap the underlying Appenders write methods in a try-catch block, and send you or a sysadmin an email.
Comment: If you're using the OnlyOnceErrorHandler, you might consider using the FallbackErrorHandler (http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/varia/FallbackErrorHandler.html) instead. That way you can specify a second appender (maybe an email appender) when the first can't write any more. That'll handle a lot of what wrapping would do.
Comment: From what I gather from http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/helpers/OnlyOnceErrorHandler.html the error is thrown once and after that it's over. What I would wish for is that, although the error is thrown only once, the appender continues to try to write to that file system... maybe with 5 minute pauses... it seems like a wrapper around the appender is a viable idea
Answer: limit the size of your logs and try using a custom appender to archive logs to a backup machine with lots of disk space.

Log4J able to recover from disk full?


Probability - Find $P(X_n \ge max(X_1, X_2, \dots, X_{n-1}))$

Question: We have a set $X_1, X_2$, . . . , $X_n$ iid standard normal random variables. How would I go about finding $P(X_n \ge max(X_1, X_2, \dots , X_{n-1}))$
Comment: In random sampling from _any_ continuous distribution, the probability that the last of $n$ randomly chosen observations will be their maximum is $1/n.$ // Your question is not exactly clear to me, so that may not be what you're asking.
Comment: Welcome to CV. Since you're new here, you may want to take our tour, which has information for new users.  Since this looks like homework (apologies if it's not), please add the [self-study] tag and read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. If this is self-study rather than homework, let us know, and... it's still a good idea to show us what you've tried.
Comment: Also answered at https://stats.stackexchange.com/q/133616/119261.
Answer: The event $X_n \geqslant \max(X_1,...,X_{n-1})$ is equivalent to $X_n = \max (X_1,...,X_n)$ so you are really just asking for the probability that the $n$th sample value is the maximum.  You have two conditions to get this: the first is that the values are IID, and the second is that they come from a continuous distribution.  Think about what each of these things implies, and you should be able to find the resulting probability (which is extremely simple).


Cannot see extended User model field in admin interface

Question: I have added a field to my User model called extra, as follows:-
class ExtendedUser(models.Model):
user = models.OneToOneField(User)
extra = models.CharField(max_length=30, null=True)

# when a user is created, make sure an ExtendedUser is created too
def create_extended_user(sender, instance, created, *args, **kwargs):
    if created:
        extended_user = ExtendedUser(user=instance)
        extended_user.save()
post_save.connect(create_extended_user, sender=User)

In my admin.py, I try to ensure that I can edit this extra field when I'm editing a user:-
class ExtendedUserInline(admin.TabularInline):
    model = ExtendedUser
    fk_name = 'user'
    max_num = 1

class ExtendedUserAdmin(UserAdmin):
    inlines = [ExtendedUserInline]

admin.site.unregister(User)
admin.site.register(User, ExtendedUserAdmin)

However, I can't see the extra field in the admin page for User.
What have I done wrong?
Comment: nope, that doesn't work either.
Comment: Adding an extra comma doesn't make any difference.
Answer: Have you restarted the development server since editing admin.py?
See also this very similar thread.
Comment: Yup, restarted the dev server - makes no difference.
Answer: Your first block of code looks ok to me, although you might want to use the decorators for readability. Here's an updated example.
class ExtendedUser(models.Model):
user = models.OneToOneField(User)
extra = models.CharField(max_length=30, null=True)

# when a user is created, make sure an ExtendedUser is created too
@receiver(post_save, sender=User)
def create_extended_user(sender, instance, created, *args, **kwargs):
        if created:
            extended_user = ExtendedUser(user=instance)
            extended_user.save()

Updated Answer:
You need to set AUTH_PROFILE_MODULE = 'app_name.ExtendedUser'
Here's a post that did pretty much the same as you.  Some of the comments below are worth reading.
Comment: Sadly, that doesn't work.

Surely the UserAdmin is the wrong class name here anyway? The one I used was ExtendedUserAdmin - as I was extending the usual UserAdmin.
Comment: You're right.  Answer deleted.  Going to try replicating with your code.
Comment: That doesn't seem to do it either. Still can't see the extra field in the admin interface.


How to select_related on reverse foreign key?

Question:
Possible Duplicate:
A left outer reverse select_related in Django?

A BlogPost has many Comments. I want to get a list of BlogPosts and all their comments.
Thus, I have
BlogPost.objects.filter(my_filter).select_related()

But the ForeignKey is on the Comment, not the BlogPost, so the select_related() doesn't prefetch any comments. Is there a way to get this to work?
I can't reverse the query (Comment.objects...) because then the other objects that the select_related() does fetch wouldn't work. I need it to work both ways.
Answer: Why won't you fetch the comments and then use regroup template tag to display them:
# Select all Comments with BlogPost data - one query
comments = Comment.objects.select_related('blog_post').order_by('-blog_post').all()

Then in template:
{% regroup comments by blog_post as posts %}
{% for blog_post in posts %}

Blog post {{ blog_post.title }}

    {% for comment in blog_post.comments %}
    ...
    {% endfor %}

{% endfor %}

Comment: Another problem with this solution is that it won't get blog posts with no Comments
Comment: I think because of my last paragraph. `BlogPost` has other foreign keys that `select_related` *does* work on, and I want it to continue working on those. By fetching the comments first you're just shifting the problem.


Sampling probability distribution-how many samples?

Question: Simple question.
I have a rv $X$ whose mean and variance I know but am interested in the true distribution, $P(X \le a)$ which I do not know. So I resort to sampling $X$. I do this $N$ times and create a approximate pmf for $X$. I want to know how "good" this is. How do I quantify this?
Okay, more realistically I am interested in this specific value $P(X \le k)$. Say the true answer to this is $\alpha$ which I do not know. But using my $N$ samples I get an answer to this as $\beta$. Now how close is $\alpha$ to $\beta$ probabilistically?
Comment: If you are only interested in $\alpha$ then you are sampling from a Bernoulli distribution with probability of success $\alpha$. You can construct a confidence interval for $\alpha$
Answer: The Central Limit Theorem has something to say about your last question. Specifically, the usual way of estimating such as $P\left(X \leq k \right)$ consists of using the empirical distribution function (ECDF) and computing
$$F_n(k) = \sum_{i=1}^n \frac{1}{n} \mathbf{1} \left(X_i \leq k \right)$$
where we just count the number of observations $X_1, \ldots, X_n$ that fall short of $k$ and divide by the total number observations. That is, our most intuitive estimator for this probability is the corresponding sample proportion.
It turns out that this estimator has fantastic statistical properties. It can be shown that it converges, as $n \to \infty$, to $P\left(X\leq k \right)$ with probability one and that it does so uniformly for all values of $k$. Since $F_n(k)$ is also a sum of iid terms, one can further apply the CLT to make probabilistic statements about the distance from the true number. It's not very difficult to show that
$$F_n(k) \xrightarrow{D} \mathcal{N} \left( P(X\leq k), \frac{ P(X\leq k)\left(1-P(X\leq k\right)} {n}  \right) $$
By using the properties of the Normal distribution you can make probabilistic statements about the quantity $|F_n(k)-P(X\leq k)|$ after plugging in $F_n(k)$ in the variance of the asymptotic distribution.
Comment: @bissi No, the mean of this absolute value is not zero. The variable $F_n(k)-P(X\leq k)$ (without the absolute value) is normally distributed with mean zero and the variance you say. Based on this distribution you can compute the probability that this difference is, say less than 0.5 by $$ P\left(
|F_n(k)-P(X\leq k)| \leq 0.5   \right) = P\left( -0.5 \leq
F_n(k)-P(X\leq k) \leq 0.5  \right) $$ using the normal distribution. This is what you had in mind, right?
Comment: Asymptotically, yes, but it's the rate of convergence that's essential to the question, no?
Comment: @A.Webb The OP is interested in making probabilistic statements, the classical CLT is all that he needs for this.
Comment: @JohnK how do I get the mean and variance of this normal distribution as I do not know the value $P(X \le k)$?
Comment: @bissi For the variance you use the estimate that you have for $P(X\leq k )$, namely $F_n(k)$. It can be shown that the asymptotic normal distribution still holds after you plug-in the estimate. $F_n(k)-P(X\leq k)$ has by definition mean zero, so you don't have to worry about that. Just find the density of the the absolute value of a normal variable.
Comment: @JohnK got it. So for completeness, the mean of this absolute value is 0 and the variance is actually simply $b(1-b)/n$ where $b$ is P(X\le k)$
Comment: @JohnK This leads to a chicken and egg problem, as the probability that this difference is less than 0.5 is a normal distribution with mean 0 but the variance is unknown, as the $b$ is unknown in the first place...
Comment: @bissi No it doesn't. Like I said in my previous comment if you replace the unknown $b = P(X\leq k)$ with $F_n(k)$ in the variance, you have a legitimate way of computing such probabilities.
Comment: @JohnK. I don't get it. $P(−0.5≤F_n(k)−P(X≤k)≤0.5)= 2 \mathbb{N}(0,
b(1-b)/N)[0.5]$
Answer: As noted in earlier answers, the empirical frequency$$\frac{1}{N}\sum_{i=1}^N \mathbb{I}_{(-\infty,x]}(X_i) \stackrel{\Delta}{=} \frac{1}{N} S_N$$is associated with a Binomial variate $S_N\sim\text{Bin}(N,p)$ for which a conservative confidence interval can be constructed. There are many proposals for this construction. Including one by George Casella.
For instance, Jeffreys' confidence interval on $p$ with coverage $1-2\alpha$ is given by
<code>N=50;SN=13;alpha=.05
qbeta(alpha,.5+SN,.5+N-SN)
[1] 0.1268967
qbeta(1-alpha,.5+SN,.5+N-SN)
[1] 0.4364774
</code>
Comment: @JohnK: pardon my French, I often confuse ) with ] given that in French open intervals are denoted as ]a,b[.
Comment: It's interesting that your sample CDF is left-continuous.
Answer: If it's just P(X≤k) you need for a given k, then it becomes a relatively simple problem. You can treat the problem as sampling from a binomial random variable Y~ Bin(N,p) where you are interested in estimating the probability of success p (or what you call α). For each sampled value x out of the distribution of X, a success will be x≤k and the estimate for p (or α) will be y/N where y is the number of successes (number of sampled x's which are ≤k). There is no definite answer to how close the estimate is, it's up to you to determine. You can either construct a confidence interval for p (with, say, 0.95 confidence) and see if the interval is narrow enough for you. Conversely, you can decide what the margin of error for p you are allowing is (for example +-0.02), and determine if the confidence level obtained is high enough. A third way is to to determine both the margin of error and the confidence level and solve for the sample size N.  I just noticed that the title of your question is about the needed sample size - this does not appear in the question itself. For determining N you need to have both a margin of error (E) and a confidence level CL (which determines Z as the value of the standard Normal distribution with probability (1-CL)/2 to the right of it) and then N=0.25(Z/E)²
The 0.25 is conservative - assuming you don't know anything about p (or α).
Comment: I understand everything up till $y/N$ being an estimate for the quantity I am seeking. Now the question is what is the CI of this$y/N$ given the number of samples I have. So your "third way" This I don't understand from your answer.. What is say the 95%CI around my estimate as a function of N?
Comment: The first article that came up in a Google search is http://www.dummies.com/education/math/statistics/how-to-determine-the-confidence-interval-for-a-population-proportion/     -   I hope you can follow it.


C# - Iterate through all members of a group using graph.api graphClient.Groups. Compiler error

Question: I'm trying to follow this C# example at this link which Iterate over all the messages.
https://learn.microsoft.com/en-us/graph/sdks/paging?tabs=csharp
I have applied that paging to something similar to instead iterate through members that belong to a group and the code is throwing an error on the return true; line of all places.  Can anyone help me understand why I get the error:
Error   CS0029  Cannot implicitly convert type 'bool' to 'System.Threading.Tasks.Task'
Here is the code:

                var members = await
                 graphClient.Groups[groupObjectId].Members.GetAsync(requestConfiguration =>
                {
                    requestConfiguration.QueryParameters.Top = 999;

                });
                var pageIterator = PageIterator<Group, GroupCollectionResponse>
                    .CreatePageIterator(
                    graphClient,
                    members,
                    // Callback executed for each item in
                    // the collection
                    (m) =>
                    {
                        Console.WriteLine($"Group ID: {m.Id}");
                        Console.WriteLine($"Group Name: {m.DisplayName}");
                        Console.WriteLine($"Group Description: {m.Description}");
                        Console.WriteLine($"Group Created Date Time: {m.CreatedDateTime}");

                        //count++;
                        return true;
                    },
                    // Used to configure subsequent page
                    // requests
                    (req) =>
                    {
                        // Re-add the header to subsequent requests
                        req.Headers.Add("Prefer", "outlook.body-content-type=\"text\"");
                        return req;
                    }
                );

                await pageIterator.IterateAsync();

Answer: I think the issue is that the query returns DirectoryObjectCollectionResponse not GroupCollectionResponse. So, you need to modify the PageIterator and for each item check if the item is either User or Group
You don't need to configure subsequent page request (adding header req.Headers.Add("Prefer", "outlook.body-content-type=\"text\"") makes sense only for messages as it is in the example)
var pageIterator = PageIterator<DirectoryObject, DirectoryObjectCollectionResponse>
            .CreatePageIterator(
            graphClient,
            members,
            // Callback executed for each item in
            // the collection
            (m) =>
            {
                if (m is User user)
                {
                    Console.WriteLine($"User ID: {user.Id}");
                    Console.WriteLine($"User Name: {user.DisplayName}");
                    Console.WriteLine($"User Created Date Time: {user.CreatedDateTime}");
                }
                else if (m is Group group)
                {
                    Console.WriteLine($"Group ID: {group.Id}");
                    Console.WriteLine($"Group Name: {group.DisplayName}");
                    Console.WriteLine($"Group Description: {group.Description}");
                    Console.WriteLine($"Group Created Date Time: {group.CreatedDateTime}");
                }

                //count++;
                return true;
            }
        );

You can even cast the response to the group
var groups = await graphClient.Groups[groupObjectId].Members.GraphGroup.GetAsync(requestConfiguration =>
             {
                 requestConfiguration.QueryParameters.Top = 999;

             });
        var pageIterator = PageIterator<Group, GroupCollectionResponse>
        .CreatePageIterator(
        graphClient,
        groups,
        // Callback executed for each item in
        // the collection
        (m) =>
        {
            Console.WriteLine($"Group ID: {m.Id}");
            Console.WriteLine($"Group Name: {m.DisplayName}");
            Console.WriteLine($"Group Description: {m.Description}");
            Console.WriteLine($"Group Created Date Time: {m.CreatedDateTime}");

            //count++;
            return true;
        });

Comment: How did you know that?  This is the first time I've tried to create code to do this and also use the graph api calls.  How did you know to use DirectoryObjectCollectionResponse vs GroupCollectionResponse.
Comment: @bmickey I've tried the same code in Visual Studio and found out that graphClient.Groups[groupObjectId].Members.GetAsync returns DirectoryObjectCollectionResponse
Comment: You are correct, I just didn't understand that.  I really appreciate your help!


Are castanets used in tango music?

Question: I'm working on a piece for Pierrot ensemble (fl, cl, vln, vlc, pa, perc).  While it's not trying to be a strict or authentic tango, it uses several familiar rhythmic patterns and "sounds like" one.  I like the timbre of castanets and am considering using them.  I read that castanets are much more common to flamenco, but I also read that tango and flamenco are fairly closely related.
How out-of-place would it be to use castanets in a tango?  Would it jar the listener out of the music too much?
Answer: Castanets are not used in an Orchestra Týpica in Argentina. However, they are common in German and American tango orchestras. Tango grew out of a combination of ballads and Bizet's "Habanera" from "Carmen." Flamenco styles are more closely related to Spanish (and Moorish and perhaps Gypsy) ballad styles. Dancing is a later addition.
Broadway tangos (like "Hernando's Hideaway) seem to always use castanets. I'd suggest trying it with and without castanets and see which sounds better (rather than the most authentic).
Comment: You're right; I made the correction.
Comment: Piazzolla recordings seem to use guiro very prominently (https://youtu.be/_WIaHQwlfsw, https://youtu.be/F-XblSpna3M) as well as claves or two-pitch wood blocks. I also associate the *cajon* with tango, but I think that's more of a flamenco or Peruvian tradition.

Are castanets used in tango music?


Generalized Fibonacci Sequence Question

Question: The Fibonacci Sequence is defined as the recurrence
$a(n)=a_{n-1}+a_{n-2}$ where $a(0)=0$ and $a(1)=1$. Today, I was bored so I considered the sequence $a(n)=\sqrt{a_{n-1}}+\sqrt{a_{n-2}}$. Ten minutes and a C program later, I discovered something:
$$\lim_{n \to \infty}{a_n}=4$$
In addition, it doesn't matter what $a(0)$ and $a(1)$ are! It always converges. (Conjecture)
More experimenting lead to a more generalized result:
Consider $a(n)=a_{n-1}^{\frac{1}{k}}+a_{n-2}^{\frac{1}{k}}$ where $a(0)$ and $a(1)$ are any two positive real numbers and k is a positive integer greater than 1. This implies $$\lim_{n \to \infty}{a_n}=2^{\frac{k}{k-1}}$$
I thought that this result was pretty funny so I came here. Can someone shine some light on this discovery and prove this conjecture right? (I'm pretty sure it's not wrong)
Comment: Of course, what is straightforward if tedious for Qiaochu could be devious and nerve-wracking for others.
Comment: If a limit $L \ge 0$ exists it satisfies $L = L^{1/k} + L^{1/k}$, so either $L = 0$ or $L = 2^{ \frac{k}{k-1} }$. I think it should not be hard to show that the sequence is bounded away from $0$ if it is ever positive, so it remains to show that the limit exists and this should be a relatively straightforward if tedious argument.
Comment: [This](http://math.stackexchange.com/questions/18409/about-the-positive-sequence-a-n2-sqrta-n1-sqrta-n) is being offered on the margin to this question.
Comment: Cool question, I must say!
Comment: Indeed. It sounds easy enough to argue that the sequence is bounded, but rather non-straightforward to show that it necessarily converges towards a limit. A priori one could think it might also end up converging towards a _cycle_, or even in chaotic behavior.
Comment: I wonder now, whether other generalized Fibonacci sequences have similar behavior / related convergences.
Answer: It'll make things easier for me to write $a_{n+2} = a_{n+1}^p + a_n^p$ ($0 < p < 1$), so that the expected limit satisfies $L = 2L^p$, hence $L = 2^{ \frac{1}{1-p} }$. Let $b_n = 2^{ - \frac{1}{1-p} } a_n$; then
$$b_{n+2} = \frac{b_n^p + b_{n+1}^p}{2}$$
and we want to show that $b_n \to 1$ if $b_0, b_1$ are positive. The above implies
$$\text{min}(b_n^p, b_{n+1}^p) \le b_{n+2} \le \text{max}(b_n^p, b_{n+1}^p)$$
and by induction we find that
$$\text{min}_{0 \le i \le 1, \le \lfloor \frac{n}{2} \rfloor \le j \le n}(b_i^{p^j}) \le b_{n+1} \le \text{max}_{0 \le i \le 1, \lfloor \frac{n}{2} \rfloor \le j \le n}(b_i^{p^j})).$$
(The indices may be slightly off.) The conclusion follows from the simple fact that if $x > 0$ then $x^{p^n} \to 1$.
Comment: Ah so assuming they get closer and closer as you go to infinity they would be equal so adding them would be same as multiplying by two ($L=2L^p$). The third step was a little difficult to follow, but now I understand. Best answer so far! :)
Answer: As Qiaochu said, the statement $\lim\limits_{n \to \infty}{a_n}=2^{\frac{k}{k-1}}$ requires further justification. However, it can be proven as follows:
Intuitive Explanation:
We will show that there are three possible behaviors for the sequence $a_n,n\in\mathbb{N}$. First, eventually all terms will be less than $2^{\frac{k}{k-1}}$ and the sequence will be increasing. Second, eventually all terms will be greater than $2^{\frac{k}{k-1}}$ and the sequence will be increasing. Either of these means the limit exists, and using lemmas 1 and 3 we see that it cannot be $0$ so must be $2^{\frac{k}{k-1}}$. Third, it will alternate between being above $2^{\frac{k}{k-1}}$ one term and below $2^{\frac{k}{k-1}}$ the next, getting arbitrarily close to $2^{\frac{k}{k-1}}$.
Lemma 1: For some $n\in\mathbb{N}$, $a_n \geq 1$.
Proof: Suppose that this is not the case, so we have some least upper bound $B<1$ for the sequence $a_n,n\in\mathbb{N}$. Let $a_n$ be such that $a_n>B/2$. Clearly each term is positive and less than $1$ meaning $a_i^{1/k}>a_i$, so $a_{n+1}=a_n^{1/k}+a_{n-1}^{1/k}>a_n^{1/k}>a_n>B/2$ and thus $a_{n+2}=a_{n+1}^{1/k}+a_{n}^{1/k}>a_{n+1}+a_n>B/2+B/2=B$, contradicting $B$ being an upper bound.
Lemma 2: If $a_n > 2^{\frac{k}{k-1}}$, then $a_{n+1}< \max\{a_n,a_{n-1}\}$.
Proof: Note $x>2^{\frac{k}{k-1}}\implies x^{1/k}<\frac{x}{2}$, so if $a_n > 2^{\frac{k}{k-1}}$, $a_{n+1}< 2\max\{a_n,a_{n-1}\}^{1/k}<\max\{a_n,a_{n-1}\}$.
Lemma 3: If $a_0=a_1=1$, the sequence $a_n,n\in\mathbb{N}$ is increasing and has limit $2^{\frac{k}{k-1}}$.
Proof: We shall induct on $n$ from $2^{\frac{k}{k-1}}\geq a_2 = 2\geq a_1\geq a_0$. If $2^{\frac{k}{k-1}}\geq a_n\geq a_{n-1}\geq a_{n-2}$ then $a_{n+1}\geq 2a_n^{1/k}\geq 2\frac{a_n}{2}=a_n$, while $a_{n+1}\leq (2^{\frac{k}{k-1}})^{1/k}+(2^{\frac{k}{k-1}})^{1/k}=(2^{\frac{k}{k-1}})^{1/k}$, thus the sequence is increasing and bounded above, so must have a limit. This limit is either $0$ or $2^{\frac{k}{k-1}}$, and since $a_n\geq 1,\forall n\in \mathbb{N}$ it must be $2^{\frac{k}{k-1}}$
Proof of Theorem: By lemma 1, we have some $a_n>1$. We must also have $a_{n+1}>1$, so applying lemma 3 in light of the fact that $a_{n-1}^{1/k}+a_{n-2}^{1/k}$ is increasing in both $a_{n-1},a_{n-2}$ gives us that the limit inferior $I$ is at least $2^\frac{k}{k-1}$. Suppose we have some $a_{i-1},a_i>2^\frac{k}{k-1}$, so by lemma 2 $a_{i+1}< \max\{a_i,a_{i-1}\}$, but at the same time $a_{i+1}>(2^\frac{k}{k-1})^{1/k}+(2^\frac{k}{k-1})^{1/k}=2^\frac{k}{k-1}$ thus by induction the sequence $a_n,n\in\mathbb{N}$ is eventually decreasing and bounded below by $2^\frac{k}{k-1}$. Then it has a limit, and this limit must be $2^\frac{k}{k-1}$. If no such $a_{i-1},a_i$ exist, then we either have some $a_{i-1},a_i<2^\frac{k}{k-1}$ or all $a_{i-1},a_i$ are on different sides of $2^\frac{k}{k-1}$. In the first case, an argument analogous to the proof of lemma 3 shows that the sequence is has limit $2^\frac{k}{k-1}$. In the second, we can let $a_i>2^\frac{k}{k-1}$, so every term in the sequence $a_i,a_{i+2},\ldots$ is greater than $2^\frac{k}{k-1}$ while every term in $a_{i+1},a_{i+3},\ldots$ is less than $2^\frac{k}{k-1}$. By lemma 2 the first sequence is decreasing, so the limit superior $S$ of $a_n,n\in\mathbb{N}$ exists. It cannot be strictly greater than $2^\frac{k}{k-1}$, as if $S=2^\frac{k}{k-1}+\epsilon/2$ for $\epsilon>0$ we have some $a_{i+2j}<2^\frac{k}{k-1}+\epsilon$ so $a_{i+2j+2}=a_{i+2j}^{1/k}+a_{i+2j+1}^{1/k}<2^\frac{k}{k-1}/2+\epsilon/2+2^\frac{k}{k-1}/2 = S$, which contradicts the fact that the sequence $a_i,a_{i+2},\ldots$ is decreasing. Hence $I \geq 2^\frac{k}{k-1}\geq S$, so $I = S = 2^\frac{k}{k-1}$ thus $\lim\limits_{n \to \infty}{a_n}=2^{\frac{k}{k-1}}$.
Comment: @itdoesntwork Does that help?
Comment: Great answer, but a little to complicated for my simple mind :)
Comment: I'll edit to add an intuitive explanation.

Generalized Fibonacci Sequence Question


Turn off log4j2 startup debug logging

Question: How do I turn off the debug logging that log4j2 spits out when it initialises itself?
These sort of things:
2014-10-22 11:16:45,505 DEBUG Building Plugin[name=filter, class=org.apache.logging.log4j.core.filter.ThresholdFilter]. Searching for builder factory method...
2014-10-22 11:16:45,505 DEBUG No builder factory method found in class org.apache.logging.log4j.core.filter.ThresholdFilter. Going to try finding a factory method instead.
2014-10-22 11:16:45,505 DEBUG Still building Plugin[name=filter, class=org.apache.logging.log4j.core.filter.ThresholdFilter]. Searching for factory method...
I'm calling log4j2 from a simple Java application via eclipse. By that I mean:
public static void main(String[] args) { ... }
The loggers portion of the xml config is:
 <Loggers>
    <Root level="trace"/>

    <logger name="audit">
        <appender-ref ref="AUDITOUT"/>
        <appender-ref ref="DEBUGOUT"/>

        <appender-ref ref="ORACLEOUTINFO"/>
        <appender-ref ref="ORACLEOUTWARN"/>
        <appender-ref ref="ORACLEOUTERROR"/>
        <appender-ref ref="ORACLEOUTFATAL"/>
    </logger>

    <logger name="org.apache.log4j">
        <appender-ref ref="FILEOUT"/>
        <appender-ref ref="STDOUT"/>

        <appender-ref ref="ORACLEOUTWARN"/>
        <appender-ref ref="ORACLEOUTERROR"/>
        <appender-ref ref="ORACLEOUTFATAL"/>
    </logger>

    <logger name="jh.Runner2">
        <appender-ref ref="DEBUGOUT"/>
        <appender-ref ref="STDOUT"/>

        <appender-ref ref="ORACLEOUTTRACE"/>
        <appender-ref ref="ORACLEOUTDEBUG"/>
        <appender-ref ref="ORACLEOUTINFO"/>
        <appender-ref ref="ORACLEOUTWARN"/>
        <appender-ref ref="ORACLEOUTERROR"/>
        <appender-ref ref="ORACLEOUTFATAL"/>
    </logger>
 </Loggers>

Appenders:
 <Appenders>
    <Console name="STDOUT">
        <ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
        <PatternLayout pattern="%d %p [%t] - %m%n"/>
    </Console>

    <RollingFile name="FILEOUT" fileName="${log-path}/mainlog2.log" filePattern="${log-path}/mainlog-%d{yyyy-MM-dd}.log">
        <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
        <PatternLayout pattern="%d %p [%t] - %m%n"/>
        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
    </RollingFile>

    <RollingFile name="DEBUGOUT" fileName="${log-path}/debuglog2.log" filePattern="${log-path}/debuglog-%d{yyyy-MM-dd}.log">
        <ThresholdFilter level="DEBUG" onMatch="ACCEPT" onMismatch="DENY"/>
        <PatternLayout pattern="%d %p %C [%t] - %m%n"/>
        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
    </RollingFile>

    <RollingFile name="AUDITOUT" fileName="${log-path}/audit2.log" filePattern="${log-path}/auditlog-%d{yyyy-MM-dd}.log">
        <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
        <PatternLayout pattern="%d %p [%t] - %m%n"/>
        <Policies>
            <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
        </Policies>
    </RollingFile>

    <JDBC name="ORACLEOUTTRACE" tableName="J0T_EVENT">
        <Filters>
            <ThresholdFilter level="DEBUG" onMatch="DENY" onMismatch="NEUTRAL"/>
            <ThresholdFilter level="TRACE" onMatch="ACCEPT" onMismatch="DENY"/>
        </Filters>
        <ConnectionFactory class="com.xxxxxxxxx.db.LoggingDBConnectionFactory" method="getDatabaseConnection" />
        <Column name="EVENT_ID" literal="${nextEventID}"/>
        <Column name="APP_NM" literal="${appName}"/>
        <Column name="CREATE_DT" isEventTimestamp="true"/>
        <Column name="EVENT_TYPE_ID" literal="1"/>
        <Column name="EVENT_TXT" pattern="${eventTxtPattern}" isUnicode="false"/>
    </JDBC>

etc.
Comment: can you paste your appenders?
Answer: Your configuration starts with
<Configuration status="debug" ...

Change this to
<Configuration status="warn" ...

and you will only see WARN-level internal log4j messages (which is probably what you want).

Turn off log4j2 startup debug logging


Bind data in to TextInput from an array in react native

Question: I have an array of data. Like
product = [
  {
    name: 'item 1',
    quantity: 10,
  },
  {
    name: 'item 2',
    quantity: 12,
  },
  {
    name: 'item 3',
    quantity: 14,
  },
]

I want to show this quantity field in TextInput.
So how can I do that and also manage the onChangeText event?
Comment: So the user is looking at one of these items, and there's a text input that comes up with the corresponding quantity, which the user can edit and save? It helps if you can have a go yourself and share the code so people can see where you've got stuck. At the moment it just feels like you're asking others to do your work for you...
Answer: Check this complete example.
import React, { Component } from "react";
import { View, TextInput } from "react-native";

export default class Example extends Component {
  state = {
    product: [
      {
        name: "item 1",
        quantity: 10
      },
      {
        name: "item 2",
        quantity: 12
      },
      {
        name: "item 3",
        quantity: 14
      }
    ]
  };

  onChangeText = ( item, text ) => {
      let updateProducts = [...this.state.product]
      let index = this.state.product.findIndex(obj => obj.name == item.name)
      updateProducts[index].quantity = text;
      this.setState({
        product: updateProducts
      })
  }

  render() {
    return (
      <View>
        {this.state.product.map(item => (
          <TextInput
            style={{ height: 40, borderWidth: 1, margin: 5, width: "80%", padding: 5 }}
            onChangeText={(text) => this.onChangeText(item, text)}
            value={item.quantity}
          />
        ))}
      </View>
    );
  }
}

Hope this helps you. Feel free for doubts.
Answer: First I recommend for you You should read array and list from react native documentation then you should read TextInput properties finally :)
Here is working example
https://snack.expo.io/HJm8EOPr8 - react native
 export default class App extends React.Component {
   constructor(props) {
    super(props)
    this.state = {
        product : [
  {
    name: 'item 1',
    quantity: 10,
  },
  {
    name: 'item 2',
    quantity: 12,
  },
  {
    name: 'item 3',
    quantity: 14,
  },
]
    }
  }
  render() {
    return (
      <View style={styles.container}>
        {this.state.product.map(item => (
              <TextInput  value={item.quantity}  />
        ))}
      </View>
    );
  }
}


What is the type of the objects returned by ISelectionContainer.GetObjects?

Question: I'm writing an extension for VS2010, and my package implements IVsSelectionEvents to track the selection in the solution explorer:
    public int OnSelectionChanged(
        IVsHierarchy pHierOld,
        uint itemidOld,
        IVsMultiItemSelect pMISOld,
        ISelectionContainer pSCOld,
        IVsHierarchy pHierNew,
        uint itemidNew,
        IVsMultiItemSelect pMISNew,
        ISelectionContainer pSCNew)
    {
        if (pSCNew != null)
        {
            uint count;
            pSCNew.CountObjects((uint) Constants.GETOBJS_SELECTED, out count);
            object[] selectedObjects = new object[count];
            pSCNew.GetObjects((uint) Constants.GETOBJS_SELECTED, count, selectedObjects);

            // What is the type of this object?
            var selectedItem = selectedObjects[0] as ???;

        }
        return VSConstants.S_OK;
    }

GetObjects fills an array of objects, but I don't know what is the actual type of these objects. GetType returns System.__ComObject, which is not very helpful. I assume these objects implement a known interface, but how do I know which one? This doesn't seem to be documented anywhere... Currently I'm using dynamic to access the properties of the object, but it's not ideal, and anyway I don't know which methods are exposed by this object (I can only see the properties in the Watch window)
Does anyone know which interface is implemented by project items in the solution explorer?
Answer: OK, so it seems that the items returned by GetObjects are of type FileProperties. But anyway, I found a much easier approach, using EnvDTE instead... An EnvDTE.ProjectItem object can be retrieved from the hierarchy and item id with this method:
    private static ProjectItem GetProjectItem(IVsHierarchy hierarchy, uint itemId)
    {
        object o;
        if (hierarchy.GetProperty(itemId, (int)__VSHPROPID.VSHPROPID_ExtObject, out o) == VSConstants.S_OK)
        {
            return o as ProjectItem;
        }
        return null;
    }


How to down pick a chord with skipped strings?

Question: I'm following a tab and I encountered what is pictured below. How do I prevent the 4th string from ringing when I play this?
Answer: You could use the pick to sound the fifth string, and the fingers to pluck the top three strings. Some people call this hybrid picking. You might pick the fifth string and drag the nail of your middle finger across the top three strings at the same time.
Another alternative would be to get one of your fretting-hand fingers (probably your second finger if you fret the note with the first) in there to mute the fourth string. Then you can drag the pick across all of the strings.
I find myself using all of these techniques (but the first and the last the most often), and the larger context of what I am playing determines what happens when I play.
Comment: Probably playing a normal Am7 chord, but muting instead of fretting the 4th string, is the practical way to think of executing the mute.
Comment: @MichaelCurtis -- exactly right; your suggestion (in the deleted answer) to just play an **Am7** was a good one, and in general it's great advice when you encounter something that you can't quite play to play something sensible that's close. Even thought that doesn't _exactly_ answer OP's question, it is useful advice, and I was about to upvote it. If you want to undelete that answer, I'll still upvote; if not, I may incorporate that advice into my answer ;)
Comment: I wasn't really sure if I had something worthy of an answer, but I'll undelete.
Comment: Doesn't that arrow direction negate your first para? Since it looks like a beginner's chord, really there should be a little kiss on the 4th string, indicating muted.
Comment: @Tim do you mean an X on the 4th string to mean 'mute' (...little kiss on the 4th string)
Comment: @MichaelCurtis - that's about all it could be. **X**
Answer: As tabbed this looks like a Am7 chord with the 4th string omitted. If you play E on the 4th string, 2nd fret it will make an octave with the open E string then you can play all 5 strings.
Of course that isn't what the tab says to play, so if you want to be faithful to the tab, you have to use one of @DavidBowling's excellent suggestions.
It depends on the music and the nature of the tab (is it someone's arrangement of a song?) to decide if adding a note is OK or not.
Comment: First thought is to question the tab...   As I do so many.


MusES musical representation system -- questions on the "algebra of pitch-classes" and some conventions

Question: This may be pretty niche... For a project I'm working on I've been reading
this paper on MusES; my goal is to find a simple musical representation system (or build my own) for a program I'm working on.
On the second page they talk about dealing with the algebra of pitch-classes, and state

There is a non-trivial algebra of alterations, which includes the following pseudo-equations:
    # o b = b o # = identity

    x o natural = natural, for any x in (#, b, natural)

This algebra is non trivial because not everything is allowed, at least in the classical theory, e.g. triple sharps.

First, is it safe to assume that o is basically function composition? It also seems that the convention (a o b)(x) := b(a(x)), somewhat common in group theory, since otherwise the second equation makes no sense. Am I understanding this notation correctly?
Second, why limit this algebra to things that are denotable in regular music theory? I'm a bit confused why they would couch this in the abstractions offered by an algebra, where the # and b operations can easily be represented as +1 and -1 functions, and where natural is the zero function, if they are going to insist on conforming to a notational system not built for such abstraction. I get that triple sharps don't show up on the page, but they certainly exist, even if we don't use them. Is there some larger context to this that I'm missing? I'm coming from more of a math/comp sci background so it is entirely possible that this is a convention in music theory that has some nice properties that I'm not aware of.
Comment: From my general experience (and a math degree) the quality of the "pseudo-mathematics" in a lot of modern music theory is abysmal, *judged as mathematics*. But I don't have a (publishable!) opinion on whether this is deliberate obfuscation of the trivially obvious, or just plain mathematical incompetence.
Comment: Out of curiosity, is there a reason you're staying away from [pitch-class set theory](http://openmusictheory.com/setClassAndPrimeForm1.html)? It sounds like this is a more complicated version of it.
Comment: Not really. I'm trying to find a simple music representation system that I can use to implement some of Fux's counterpoint as a system of constraints. I've been googling "music representation systems" and this is one of the few things I've found that isn't behind a pay wall.
Comment: I'm not sure if this trivial observation blows away the basic axioms of the referenced paper - but *in any tuning system other than ET* F sharp is NOT the same note a G flat - they don't even have the same pitch. And unless your theory can deal with that, it won't "understand" tonal harmony very well - 12ET intonation is only an *approximation* to the Platonic "key system" underpinning tonal harmony IMO! Why else would Bach have written a prelude in Eb minor followed by a fugue in D# minor, in the Well-Tempered Clavier, unless *he* understood that fact?
Comment: … and as an aside, the Eb minor prelude includes a structurally important chord of Fb major, while the D# minor fugue does the same thing with an E# major chord. Just happenstance? I think not...
Comment: @alephzero Yeah, this seems like some mathematical hackery tbh. I was trying not to come out and say it in my post cuz I didn't want to get a bunch of angry theorists jumping down my throat...
Comment: @alephzero WRT tuning systems: I think of this as a basic modulus operation. There is a lot of stuff that can be gleaned from representing the integers mod n. Certainly we don't want to get rid of non-equal temperments in actual music, just as we don't want to eschew n for zero in actual math. I don't think that this invalidates a paper, though it certainly does point out a short coming. Still, this system, and systems like it, are built with an eye towards abstraction and computing, and this necessitates certain simplifications.
Comment: Also, I'm curious as to what your thinking is on why Bach chose to follow the Ebm prelude with the D#m prelude. I'm not up enough on my temperments to really grok the difference
Answer: I've never heard of this system, but it has a lot of similarities to other concepts I've come across.
The second question is much easier to explain than the first and maybe it will clear up what's. When talking about modifying a single note from it's base position, yes putting sharp as +1, flat as -1, and natural as 0 totally makes sense and I've done it in projects I've worked on in the past. There is a problem with that logic which is sharps and flats aren't additive, they override each other instead so assigning +1 to a sharp and -1 to a flat may lead to the wrong conclusion. In the same vein of thought, even the same symbols aren't additive! For example if you see a flat in the key signature and have a flat next to the note on the staff, the note is flat not double flat.
Questions on the site about these consents:

Are accidentals in the key signature and measue additive?
Do accidentals override key signature and previous accidentals?

Now back to the first. The best I can come up with for algebraic expressions they are rotations of the 7 PCs across one of the other 5 or else I can't really make sense. The author mention 7 PCs in correctional to the 7 notes A-G bring the total to 35 due to the 5 accidentals double flats to double sharps. So he mentions the sets like this:

Abb Bbb Cbb Dbb Ebb Fbb Gbb
Ab  Bb  Cb  Db  Eb  Fb  Gb
A   B   C   D   E   F   G
A#  B#  C#  D#  E#  F#  G#
A## B## C## D## E## F## G##

How the pitch equivalents though line up is shown below:

    0   1   2   3   4   5   6   7   8   9   10  11
0   A   A#  A##
1   Bbb Bb  B   B#  B##
2       Cbb Cb  C   C#  C##
3               Dbb Db  D   D#  D##
4                       Ebb Eb  E   E#  E##
5                           Fbb Fb  F   F#  F##
6   Gb                              Gbb Gb  G   G#
0                                           Abb Ab

The transformation only really makes sense on the first chart when you look how the vertical rotations happen across the same letter names. The second chart, saying # and b always have some kind of identity is wrong  as seen in row 7 and 8 where sharps are not inverses for flats and vice versa as you get into double sharp and double flat territory.
In closing, I will note the the last 3 bullet bullet points of section 2.1 which you are quoting are incorrect (or at least very misguided). Triple flats and triple sharps exits and the key of G# Major does exist. While they are at the extreams of musical knowledge they are well defined concepts.
Comment: Great response. I agree that in standard notation a flat on a C# gives a Cb, in the paper they are defining algebraic operations `b` and `#` which, by their axioms of `# o b = b o # = natural`, codify the inverse property. In the paper they seem to sharpen and flatten rather than yield a sharp or flat note. But, as you say, this goes against standard notation.
Comment: Viewing a pitch class as a tuple (base-note, accidental), with base-note in A, B, ..., G and accidental an integer, this additive style notation is I think the same (up to some modulus) of your rotational notation.
Comment: I'm not sure about the second axiom, but if I'm not mistaken I think that it is saying that `natural` just sends (base-note, accidental) to (base-note, 0), where any non-zero accidental is cleared?


Quando coloco uma tela principal, esta tela principal desativa a calculadora na segunda tela

Question: Fiz o App para somar dois numeros, porem quando coloco uma tela inicial apenas com um botao para chamar a tela de soma, os calculos da tela de soma ficam inativos, como resolver?Obrigado
Tela MainActivity.java
package br.com.frutasalmeidasantos.www.testefinal;

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    public void chamasolucao (View v) { setContentView(R.layout.activity_solucao); }
}

Solucao.java
package br.com.frutasalmeidasantos.www.testefinal;

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;
import android.widget.Toast;

public class Solucao extends AppCompatActivity {
    EditText campo1;
    EditText campo2;
    Button btcalc;
    TextView resposta;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_solucao);

        campo1 = (EditText) findViewById(R.id.campo1);
        campo2 = (EditText) findViewById(R.id.campo2);
        btcalc = (Button) findViewById(R.id.btcalc);
        resposta = (TextView) findViewById(R.id.resposta);

        btcalc.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                String scampo1 = campo1.getText().toString();
                String scampo2 = campo2.getText().toString();

                if (scampo1.isEmpty() || scampo2.isEmpty()){
                    Toast.makeText(Solucao.this, "", Toast.LENGTH_SHORT).show();
                    return;
                }

                int ncampo1 = Integer.parseInt(scampo1);
                int ncampo2 = Integer.parseInt(scampo2);

                int nresposta = ncampo1 + ncampo2;

                resposta.setText(""+ncampo1+ "+" +ncampo2+ "=" +nresposta);
            }
        });
    }
}

Manifest
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="br.com.frutasalmeidasantos.www.testefinal">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <activity android:name=".Solucao"></activity>
    </application>

</manifest>

Comment: Obrigado @ramaral, funcionou!
Comment: Altere o  método `chamasolucao()` para `public void chamasolucao (View v) { startActivity(new Intent(this, Solucao.class)); }`
Answer: Para solucionar o seu problema faça o seguinte:
Na linha public class MainActivity extends AppCompatActivity { adicione implements View.OnClickListener deixando a linha dessa forma:
public class MainMenuActivity extends AppCompatActivity implements View.OnClickListener{
Abaixo do onCreate você cria o método public void onClick(View v){}
O resultado será esse (MainActivity):
package br.com.frutasalmeidasantos.www.testefinal;

import android.content.Intent;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;

public class MainMenuActivity extends AppCompatActivity implements View.OnClickListener{

    private Button btnSolucao;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main_menu);

        btnSolucao=(Button)findViewById(R.id.btnSolucao);
        btnSolucao.setOnClickListener(this);

    }

    public void onClick(View v){

        Intent it = new Intent(this, Solucao.class);
        startActivity(it);

    }

}

Comment: Fiz exatamente como sugeriu, ainda nao funcionou! Alguma sugestao?
Comment: @GilbertoAlmeida vc nn cria um novo metodo, você coloco o implements na sua public class antes do onCreate
Comment: Boa noite! vc pode me enviar o exemplo como no modelo acima? Sou iniciante e apesar de ter chegado ate aqui, agora travei! consigo executar a splash screen e a calculadora separada por exemplo, mas nao consigo fazer a calculadora funcionar quando tenho duas activities, ela fica com o botal calcular nulo.
Comment: Favor não utilizar snippet se não for um código executável no site, como em javascript, css e html.
Comment: @diegofm ??? nn entendi


Truth Conditions for Modal Logic

Question: I know that
$$
1)\space w ⊩\diamond P\iff there\space is\space some\space worlds\space w' \space such\space that\space wRw': w ⊩ P
$$
$$
2)\space w ⊩\square P\iff for\space all\space worlds\space w' \space such\space that\space wRw': w ⊩ P
$$
Now suppose I have a world e which doesn't 'see' any other world. Is it then the case that:

$$
w ⊩\diamond P
$$
  $$
w ⊮\square P
$$

The reason I think so is that (1) doesn't hold for e since it doesn't see any worlds. But (2) seems to me to hold because it is a "for all"-statement. So, at least to me, (2) holds, since it is true that for all worlds e sees (there are none), those force P.
However, I have two questions about this.
Couldn't the same reasoning be applied to $$\square ¬P  $$
Thus, we would have $$e ⊩ \square P\space and\space e ⊩ \square ¬P $$
This, seems to me to be very strange. But, then again, I'm new to Modal Logic.
Secondly, is it not necessary that we know, at least that $$ e ⊩ P $$ if we are going to even start talking about P being necessary at e?.
Thanks.
Answer: The arguments you've given for (1) and (2) seem to say that $w⊩□P$ and $w⊮⋄P$, which is the opposite of what you've actually put in the displayed formula. (The argument works, though -- if there is no $w'$ with $wRw'$, then any property at all holds "for all $w'$ with $wRw'$".)
You are also correct that if there is no $w'$ with $wRw'$, then both $w⊩□P$ and $w⊩□¬P$ hold. This is why we might want to assume that, at least, $wRw$ for all $w$. Your second question has the same answer: the properties we want our logical system to have will put constraints on the relations $R$ that we are willing to consider.
Comment: Sorry! Yes, I meant w⊩□P and w ⊮ ⋄P ... I'm tired after a whole day of logic.

Thank you!

Truth Conditions for Modal Logic


Android Kotlin: unable to save InputStream as file

Question: I am building an Android application using Kotlin. In my application, I am trying to save InputStream as a File. But it is not working.
This is the function to save InputStream as File
fun copyStreamToFile(inputStream: InputStream) {
        var outputFile = File("testing-again.png")
        inputStream.use { input ->
            val outputStream = FileOutputStream(outputFile)
            outputStream.use { output ->
                val buffer = ByteArray(4 * 1024) // buffer size
                while (true) {
                    val byteCount = input.read(buffer)
                    if (byteCount < 0) break
                    output.write(buffer, 0, byteCount)
                }
                output.flush()
            }
        }
    }

It is giving me the following error
Process: com.forkthecoop.com22222, PID: 16949
    java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=546, result=-1, data=Intent { dat=content://com.google.android.apps.photos.contentprovider/-1/1/content://media/external/images/media/31/ORIGINAL/NONE/image/jpeg/323542094 flg=0x1 clip={text/uri-list {...}} }} to activity {com.forkthecoop.com22222/com.forkthecoop.com22222.SendOrdinaryMessageActivity}: java.io.FileNotFoundException: testing-again.png: open failed: EROFS (Read-only file system)
        at android.app.ActivityThread.deliverResults(ActivityThread.java:5015)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:5056)
        at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:51)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7656)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
     Caused by: java.io.FileNotFoundException: testing-again.png: open failed: EROFS (Read-only file system)
        at libcore.io.IoBridge.open(IoBridge.java:492)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:236)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:186)
        at com.forkthecoop.com22222.PartialMediaUploadPreview.copyStreamToFile(PartialMediaUploadPreview.kt:171)
        at com.forkthecoop.com22222.PartialMediaUploadPreview.handleOnActivityResult(PartialMediaUploadPreview.kt:146)
        at com.forkthecoop.com22222.SendOrdinaryMessageActivity.onActivityResult(SendOrdinaryMessageActivity.kt:208)
        at android.app.Activity.dispatchActivityResult(Activity.java:8310)
        at android.app.ActivityThread.deliverResults(ActivityThread.java:5008)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:5056)
        at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:51)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7656)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
     Caused by: android.system.ErrnoException: open failed: EROFS (Read-only file system)
        at libcore.io.Linux.open(Native Method)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
        at libcore.io.BlockGuardOs.open(BlockGuardOs.java:254)
        at libcore.io.ForwardingOs.open(ForwardingOs.java:166)
        at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:7542)
        at libcore.io.IoBridge.open(IoBridge.java:478)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:236)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:186)
        at com.forkthecoop.com22222.PartialMediaUploadPreview.copyStreamToFile(PartialMediaUploadPreview.kt:171)
        at com.forkthecoop.com22222.PartialMediaUploadPreview.handleOnActivityResult(PartialMediaUploadPreview.kt:146)
        at com.forkthecoop.com22222.SendOrdinaryMessageActivity.onActivityResult(SendOrdinaryMessageActivity.kt:208)
        at android.app.Activity.dispatchActivityResult(Activity.java:8310)
        at android.app.ActivityThread.deliverResults(ActivityThread.java:5008)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:5056)
        at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:51)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7656)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
2021-03-14 15:56:54.505 518-2903/system_process E/WifiThreadRunner: WifiThreadRunner.call() timed out!
    java.lang.Throwable: Stack trace:
        at com.android.server.wifi.WifiThreadRunner.call(WifiThreadRunner.java:80)
        at com.android.server.wifi.WifiServiceImpl.startScan(WifiServiceImpl.java:497)
        at android.net.wifi.IWifiManager$Stub.onTransact(IWifiManager.java:718)
        at android.os.Binder.execTransactInternal(Binder.java:1159)
        at android.os.Binder.execTransact(Binder.java:1123)

My application has both read and write permissions to the Storage. It is just not working. What is wrong with my code and how can I fix it?
Comment: I am not sure what the error is. From that error, `Caused by: android.system.ErrnoException: open failed: EROFS (Read-only file system)` and `Failure delivering result ResultInfo{who=null, request=546, result=-1, data=Intent { dat=content://com.google.android.apps.photos.contentprovider/-1/1/content://media/external/images/media/31/ORIGINAL/NONE/image/jpeg/323542094 flg=0x1 clip={text/uri-list {...}} }} to activity {com.forkthecoop.com22222/com.forkthecoop.com22222.SendOrdinaryMessageActivity}: java.io.FileNotFoundException: testing-again.png: open failed: EROFS (Read-only file system)`
Comment: So, I think you are having the error cause you don't have permission to do something on that path/directory.
Comment: `var outputFile = File("testing-again.png")` Supply s full path. Not only a file name.
Comment: it's because you try to save your image in this path : `/testing-again.png` and it's a system file
Answer:
FileNotFoundException: testing-again.png: open failed: EROFS
(Read-only file system)

You cannot save a file in the system folder so you must specify the path and save your image for example i saved the image in the DCIM folder below :
fun copyStreamToFile(inputStream: InputStream) {
    val storeDirectory = this.getExternalFilesDir(Environment.DIRECTORY_DCIM) // DCIM folder
    val outputFile = File(storeDirectory, "testing-again.png")
    inputStream.use { input ->
        val outputStream = FileOutputStream(outputFile)
        outputStream.use { output ->
            val buffer = ByteArray(4 * 1024) // buffer size
            while (true) {
                val byteCount = input.read(buffer)
                if (byteCount < 0) break
                output.write(buffer, 0, byteCount)
            }
            output.flush()
        }
    }
}

You will find your image
in this path : /storage/emulated/0/Android/data/Your_Package_Name/files/DCIM
Comment: I just sent a message in the chat room. Please have a look.
Comment: I am getting the following error when I changed the path as in your code.
Comment: 2021-03-14 17:19:14.300 18978-18978/com.forkthecoop.com22222 E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.forkthecoop.com22222, PID: 18978
    java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=546, result=-1, data=Intent { dat=content://com.google.android.apps.photos.contentprovider/-1/1/content://media/external/images/media/31/ORIGINAL/NONE/image/jpeg/1216397895 flg=0x1 clip=
Comment: is that the same mistake you mentioned on your question!?
Comment: `java.io.FileNotFoundException: testing-again.png: open failed: EROFS (Read-only file system)` ?
Comment: No, I used the full path as in your code. val storeDirectory = this.getExternalFilesDir(Environment.DIRECTORY_DCIM) // DCIM folder
    val outputFile = File(storeDirectory, "testing-again.png")
Comment: please show me the next of the `error jpeg/1216397895 flg=0x1 clip=........`
Comment: Hello, testing it again. It is not throwing the error now. But it is not saving the file in the DCIM folder. The folder is basically empty.
Comment: Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/229901/discussion-between-shay-kin-and-wai-yan-hein).