Analytical solution to linear-regression coefficient estimates

Question: I'm trying to understand matrix notation, and working with vectors and matrices.
Right now I'd like to understand how the vector of coefficient estimates $\hat{\beta}$ in multiple regression is computed.
The basic equation seems to be
$$
\frac{d}{d\boldsymbol{\beta}} (\boldsymbol{y}-\boldsymbol{X\beta})'(\boldsymbol{y}-\boldsymbol{X\beta}) = 0 \>.
$$
Now how would I solve for a vector $\beta$ here?
Edit: Wait, I'm stuck. I'm here now and don't know how to continue:
$    \frac{d}{d{\beta}}
\left(
\left(\begin{smallmatrix}
 y_1 \\
 y_2 \\
 \vdots \\
 y_n
\end{smallmatrix}\right)
-
\left(\begin{smallmatrix}
 1 & x_{11} & x_{12} & \dots & x_{1p} \\
 1 & x_{21} & x_{22} & \dots & x_{2p} \\
 \vdots & & & & \vdots \\
 1 & x_{n1} & x_{n2} & \dots & x_{np} \\
\end{smallmatrix}\right)
\left(\begin{smallmatrix}
 \beta_0 \\
 \beta_1 \\
 \vdots \\
 \beta_p
\end{smallmatrix}\right)
\right) '
\left(
\left(\begin{smallmatrix}
 y_1 \\
 y_2 \\
 \vdots \\
 y_n
\end{smallmatrix}\right)
-
\left(\begin{smallmatrix}
 1 & x_{11} & x_{12} & \dots & x_{1p} \\
 1 & x_{21} & x_{22} & \dots & x_{2p} \\
 \vdots &  & & & \vdots \\
 1 & x_{n1} & x_{n2} & \dots & x_{np} \\
\end{smallmatrix}\right)
\left(\begin{smallmatrix}
 \beta_0 \\
 \beta_1 \\
 \vdots \\
 \beta_p
\end{smallmatrix}\right)
\right)
$
$
\frac{d}{d{\beta}} \sum_{i=1}^n \left( y_i - \begin{pmatrix} 1 & x_{i1} & x_{i2} & \dots & x_{ip} \end{pmatrix}
\begin{pmatrix}
 \beta_0 \\
 \beta_1 \\
 \vdots \\
 \beta_p
\end{pmatrix} \right)^2$
With $x_{i0} = 1$ for all $i$ being the intercept:
$
\frac{d}{d{\beta}} \sum_{i=1}^n \left( y_i -
\sum_{k=0}^p x_{ik} \beta_k
 \right)^2
$
Can you point me in the right direction?
Comment: @GaBorgulya, thanks for the edit, did not know about `smallmatrix`, so did not try to edit, since usual solution of breaking the formula in several lines would not have worked here.
Answer: We have
$\frac{d}{d\beta} (y - X \beta)' (y - X\beta) = -2 X' (y - X \beta)$.
It can be shown by writing the equation explicitly with components. For example, write $(\beta_{1}, \ldots, \beta_{p})'$ instead of $\beta$. Then take derivatives with respect to $\beta_{1}$, $\beta_{2}$, ..., $\beta_{p}$ and stack everything to get the answer. For a quick and easy illustration, you can start with $p = 2$.
With experience one develops general rules, some of which are given, e.g., in that document.
Edit to guide for the added part of the question
With $p = 2$, we have
$(y - X \beta)'(y - X \beta) = (y_1 - x_{11} \beta_1 - x_{12} \beta_2)^2 +
(y_2 - x_{21}\beta_1 - x_{22} \beta_2)^2$
The derivative with respect to $\beta_1$ is
$-2x_{11}(y_1 - x_{11} \beta_1 - x_{12} \beta_2)-2x_{21}(y_2 - x_{21}\beta_1 - x_{22} \beta_2)$
Similarly, the derivative with respect to $\beta_2$ is
$-2x_{12}(y_1 - x_{11} \beta_1 - x_{12} \beta_2)-2x_{22}(y_2 - x_{21}\beta_1 - x_{22} \beta_2)$
Hence, the derivative with respect to $\beta = (\beta_1, \beta_2)'$ is
$
\left(
\begin{array}{c}
-2x_{11}(y_1 - x_{11} \beta_1 - x_{12} \beta_2)-2x_{21}(y_2 - x_{21}\beta_1 - x_{22} \beta_2) \\
-2x_{12}(y_1 - x_{11} \beta_1 - x_{12} \beta_2)-2x_{22}(y_2 - x_{21}\beta_1 - x_{22} \beta_2)
\end{array}
\right)
$
Now, observe you can rewrite the last expression as
$-2\left(
\begin{array}{cc}
x_{11} & x_{21} \\
x_{12} & x_{22}
\end{array}
\right)\left(
\begin{array}{c}
y_{1} - x_{11}\beta_{1} - x_{12}\beta_2 \\
y_{2} - x_{21}\beta_{1} - x_{22}\beta_2
\end{array}
\right) = -2 X' (y - X \beta)$
Of course, everything is done in the same way for a larger $p$.
Comment: Awesome, I was looking for exactly that type of pdf. Thanks a ton!
Comment: Oh, I thought I could do it myself now, but I can't. Can you tell me if my steps are right or if I should take "another way" to solve this?
Comment: @Alexx Hardt: My first equation in the edit is the same as your very last equation in the particular case where p = 2. So, you can mimic my calculations for components 3, 4, ..., p.
Comment: Thanks again :) I think I'll actually use all three suggestions. I'm building a .pdf which explains and sums up basic stats matrix algebra, because I somehow never wanted to learn it when I learned it in my classes. To solve it with three different ways will help me understand it better, I hope.
Comment: Oh, but this is for p=2 and n=2, right? I'll write it down with n=3 I think.
Comment: @Alexx Hardt: Yes, n=2 as well in my answer. You can also have a look at http://www.amazon.com/Matrix-Algebra-Useful-Statistics-Probability/dp/0471866814. This book is a standard in statistics I think.
Answer: You can also use formulas from Matrix cookbook. We have
$$(y-X\beta)'(y-X\beta)=y'y-\beta'X'y-y'X\beta+\beta'X'X\beta$$
Now take derivatives of each term. You might want to notice that $\beta'X'y=y'X\beta$. The derivative of term $y'y$ with respect to $\beta$ is zero. The remaining term
$$\beta'X'X\beta-2y'X\beta$$
is of form of function
$$f(x)=x'Ax+b'x,$$
in formula (88) in the book in page 11,  with $x=\beta$, $A=X'X$ and $b=-2X'y$. The derivative is given in the formula (89):
$$\frac{\partial f}{\partial x}=(A+A')x+b$$
so
$$\frac{\partial}{\partial \beta}(y-X\beta)'(y-X\beta)=(X'X+(X'X)')\beta-2X'y$$
Now since $(X'X)'=X'X$ we get the desired solution:
$$X'X\beta=X'y$$
Comment: @ocram, thanks. I would not call it ingenious, it is a standard application of existing formulas. You just need to know the formulas :)
Comment: +1 mpiktas: Your solution is more ingenious than mine and I think it should be used in more complex practical situations.
Answer: Here is a technique for minimizing the sum of squares in regression that actually has applications to more general settings and which I find useful.
Let's try to avoid vector-matrix calculus altogether.
Suppose we are interested in minimizing
$$
\newcommand{\err}{\mathcal{E}}\newcommand{\my}{\mathbf{y}}\newcommand{\mX}{\mathbf{X}}\newcommand{\bhat}{\hat{\beta}}\newcommand{\reals}{\mathbb{R}}
\err = (\my - \mX \beta)^T (\my - \mX \beta) = \|\my - \mX \beta\|_2^2 \> ,
$$
where $\my \in \reals^n$, $\mX \in \reals^{n\times p}$ and $\beta \in \reals^p$. We assume for simplicity that $p \leq n$ and $\mathrm{rank}(\mX) = p$.
For any $\bhat \in \reals^p$, we get
$$
\err = \|\my - \mX \bhat + \mX \bhat - \mX \beta\|_2^2 = \|\my - \mX \bhat\|_2^2 + \|\mX(\beta-\bhat)\|_2^2 - 2(\beta - \bhat)^T \mX^T (\my - \mX \bhat) \>.
$$
If we can choose (find!) a vector $\bhat$ such that the last term on the right-hand side is zero for every $\beta$, then we would be done, since that would imply that $\min_\beta \err \geq \|\my - \mX \bhat\|_2^2$.
But, $(\beta - \bhat)^T \mX^T (\my - \mX \bhat) = 0$ for all $\beta$ if and only if $\mX^T (\my - \mX \bhat) = 0$ and this last equation is true if and only if $\mX^T \mX \bhat = \mX^T \my$. So $\err$ is minimized by taking $\bhat = (\mX^T \mX)^{-1} \mX^T \my$.

While this may seem like a "trick" to avoid calculus, it actually has wider application and there is some interesting geometry at play.
One example where this technique makes a derivation much simpler than any matrix-vector calculus approach is when we generalize to the matrix case. Let $\newcommand{\mY}{\mathbf{Y}}\newcommand{\mB}{\mathbf{B}}\mY \in \reals^{n \times p}$, $\mX \in \reals^{n \times q}$ and $\mB \in \reals^{q \times p}$. Suppose we wish to minimize
$$
\err = \mathrm{tr}( (\mY - \mX \mB) \Sigma^{-1} (\mY - \mX \mB)^T )
$$
over the entire matrix $\mB$ of parameters. Here $\Sigma$ is a covariance matrix.
An entirely analogous approach to the above quickly establishes that the minimum of $\err$ is attained by taking
$$
\hat{\mB} = (\mX^T \mX)^{-1} \mX^T \mY \>.
$$
That is, in a regression setting where the response is a vector with covariance $\Sigma$ and the observations are independent, then the OLS estimate is attained by doing $p$ separate linear regressions on the components of the response.
Comment: @DWin, yes, the rendering is a bit funky. I thought you might have intended the comment for another post since this one has no votes (up or down) and so the comment seemed to be out of place. Cheers.
Comment: @cardinal +1, useful trick. This question turned out to be a pretty good reference.
Comment: Fortunately the forum rules allow adding +1 to every answer. Thanks for the education, guys!
Comment: @DWin, did you mean to post this under the comments to the question?
Comment: I suppose I could have. I had sequentially gone through the question and then all the answers (after the processing of the MathML stopped jerking around) and found each of the answers informative. I just dropped my comment on yours because it was where I stopped reading.
Comment: Just more interface funkiness I guess. I did click it and it did turn color but it didn't "take". Fixed (I hope).
Comment: @DWin: I see. Yes, I believe the interface may limit the number of votes you can submit within a certain time span. So, if you went through and voted on each in rapid succession, it may have simply blocked later ones. That's probably to keep people from writing programs to automate voting.
Comment: @mpiktas, yes, I agree that this question has turned out better than I would have initially guessed.
Answer: One way which may help you understand is to not use matrix algebra, and differentiate with each respect to each component, and then "store" the results in a column vector.  So we have:
$$\frac{\partial}{\partial \beta_{k}}\sum_{i=1}^{N}\left(Y_{i}-\sum_{j=1}^{p}X_{ij}\beta_{j}\right)^{2}=0$$
Now you have $p$ of these equations, one for each beta.  This is a simple application of the chain rule:
$$\sum_{i=1}^{N}2\left(Y_{i}-\sum_{j=1}^{p}X_{ij}\beta_{j}\right)^{1}\left(\frac{\partial}{\partial \beta_{k}}\left[Y_{i}-\sum_{j=1}^{p}X_{ij}\beta_{j}\right]\right)=0$$
$$-2\sum_{i=1}^{N}X_{ik}\left(Y_{i}-\sum_{j=1}^{p}X_{ij}\beta_{j}\right)=0$$
Now we can re-write the sum inside the bracket as $\sum_{j=1}^{p}X_{ij}\beta_{j}=\bf{x}_{i}^{T}\boldsymbol{\beta}$  So you get:
$$\sum_{i=1}^{N}X_{ik}Y_{i}-\sum_{i=1}^{N}X_{ik}\bf{x}_{i}^{T}\boldsymbol{\beta}=0$$
Now we have $p$ of these equations, and we will "stack them" in a column vector.  Notice how $X_{ik}$ is the only term which depends on $k$, so we can stack this into the vector $\bf{x}_{i}$ and we get:
$$\sum_{i=1}^{N}\bf{x}_{i}\rm{Y}_{i}=\sum_{i=1}^{N}\bf{x}_{i}\bf{x}_{i}^{T}\boldsymbol{\beta}$$
Now we can take the beta outside the sum (but must stay on RHS of sum), and then take the invervse:
$$\left(\sum_{i=1}^{N}\bf{x}_{i}\bf{x}_{i}^{T}\right)^{-1}\sum_{i=1}^{N}\bf{x}_{i}\rm{Y}_{i}=\boldsymbol{\beta}$$


Are the SID, admin tokens, etc. created on the first boot after SysPrep, before going through setup?

Question: I have a virtual machine running in hyper-v that I would like to clone a dozen times. I ran sysprep, shutdown and then duplicated the VHD, assigning it to the other servers already setup in Hyper-V.
Before I copied/assigned the VHD to all the servers, I did "test" it by assigning it to one and then booting into windows. I did NOT go through setup, just made sure Windows booted to the choose your language screen.
All VMs are Gen 1 Hyper-V Machines running Windows Server 2012.
Now I'm getting Domain Trust errors and all machines have the same SID.
Did I mess up by booting once after running sysprep, or is the machine's SID assigned after you do the first boot setup?
Follow-up: Just ran sysprep again on each vm as an easy fix.
Answer: Yeah, you borked it. The specialize configuration pass runs as soon as the image is booted and this configuration pass is responsible fro generating the machine SID.
Lesson learned: Once your sysprep image shuts down do not start it and then use it as your template.
As Jesus mentioned in his answer, duplicate machine SIDS aren't a problem in and of themselves, but if you're wondering why they exist it's because you booted the image and then used it as your template. If this concerns you then create a new template image.
Comment: I see. Well, at least I learned something. Thanks!!
Comment: Glad to help...
Answer: It's not the SID that causes you issues - having duplicate machines SIDS actually won't cause a problem. However SysPrep does reset a lot of other machine specific settings - some which you may have triggered if you boot into the machine before you deployed the duplicates.
Before starting from scratch I would simply remove the machines to a work group and rejoin them to the domain.
Comment: A system should be removed from the domain, or never joined to the domain in the first place, before running Sysprep.


Get all cells of dataframe as tuple of (index, column) for passing to df.apply()

Question: I want to get index and column name of every cell in the Pandas data frame.
For example, in data frame generated from the code below
df = pd.DataFrame({1 : np.arange(1, 6),
               2 : np.arange(6, 11),
               3 : np.arange(11, 16),
               4 : np.arange(16, 21),
               5 : np.arange(21, 26)},
              index=[1, 2, 3, 4, 5])

I want to access an index/column name combination of value's row index and value's column name such as [1,1] for 1, [2,1] for 2, [3,1] for 3 etc...
Ultimate goal is to update every value in the data frame based on its position within the data frame with df.apply(). Index and column names (equivalent and ordered identifiers in n x n data frame) are needed to pull values from another data frame.
Thanks!
Comment: **You totally don't need to do this, so don't do it** (*"Ultimate goal is to update every value in the data frame based on its position within the data frame with df.apply()"*) If you need to *"pull values from another data frame"*, then use the `df.join()` command, that's what it's there for. Learn dataframe idiom, don't just try to brute-force code into something superficially resembling it (which wouldn't be scalable or performant, anyway). The whole point of dataframes is that we almost never pass around wholesale lists of coords, certainly not for the entire df, let alone a large slice.
Answer: I would suggest using a own function for doing that. You can access each column of the dataframe by using the dict-like notation. In addition to get the desired element by accessing the needed index/row I would use .ix as shown below
import pandas as pd

df = pd.DataFrame({1 : np.arange(1, 6),
               2 : np.arange(6, 11),
               3 : np.arange(11, 16),
               4 : np.arange(16, 21),
               5 : np.arange(21, 26)},
              index=[1, 2, 3, 4, 5])

def get_from_coords(df, x, y):
    return df[x].ix[y]

So for example:
In [2]: get_from_coords(df, 2, 1)
Out[2]: 6

The docs provide detailed information about indexing pandas dataframes.
Update since I missunderstood the question as clarified in the comments:
def look_for_value(df, value):
    l = []
    for row in df.itertuples():
        print(row)
        if value in row[1:]:
            # appending a tuple of the format `(index name, column name)`
            l.append((row[0], df.columns[row.index(value)-1]))
    return l

def look_using_generator(df, value):
    return [(row[0], df.columns[row.index(value)-1]) for row in df.itertuples() if value in row[1:]]

I am iterating through all the rows of the dataframe using .itertuples() which is faster than .iterrows() and looking for the desired entry/value. If the value is found in the row a tuple containing the index and column name is stored to a list which is returned at the end. I provided a kind of step-by-step solution in the first function and a one-liner using a generator in list comprehension.
Edit since OP pointed out he needs to have the column and index names to change the corresponding value:
Let's say we want to find all values 6 and replace them with 66:
for item in look_using_generator(df, 6):
    df[item[0]].ix[item[1]] = 66

Comment: I am trying to do the opposite! For value 6 I want to get (2,1). I would like to avoid nested for loops.
Comment: Sorry, I missunderstood.
Comment: Not a problem, I added some clarification to the question.
Comment: What about duplicate entries? Do you want to get the first or the last occurrence only or rather a list containing the coordinates of all of them?
Comment: I would prefer a pair of index and column name for every value. Duplicate entries as well. I am populating a matrix of pearson coefficients of similarities where index and columns are ids of users.
Comment: Thanks for the update!
I was looking for something like this to iterate over every single value. I think this is a standard Pandas approach in modifying data frames.

indx = df.index.values
for i in indx:
    for j in indx:
        df.set_value(j, i, updated_value = 10)
Comment: Why do you want to iterate over every value? This could be rather slow when working with bigger data since pandas' operations are row-optimized.
Comment: is there a better to update every value in the data frame based on it's position within the data frame (index and column)?
Comment: The dataframe initialized in your question would return `1` for  `df[1].ix[1]` So let's say, you want to change this value to `42`. This can be done as: `df[1].ix[1] = 42`. You should not edit values while iterating as stated in the [docs](http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.DataFrame.iterrows.html) (see Note #2).
Comment: Further edit to my post. If this does not solve your problem and since these comments here seem to get kind of chatty messy, you should think about clarifying what _exactly_ you want to achieve and _why_.
Answer: Use df.columns[column position] to get column label.  Inversely: use df.columns.get_loc("column label") to get column position.
Similarly for row, df.index[row position] to get the row index. Inversely: use df.index.get_loc('index label') to get row position.
As for your question, it is straightforward to loop over the dataframe by row and column position, then access by .iloc
E.g.:
def lookup(df, value):
    l = []
    for i in range(df.shape[0]):
        for j in range(df.shape[1]):
            if df.iloc[i, j] == value:
                l.append((df.index[i], df.columns[j]))
    return l


Advice for Large Backup on Plesk Mail Server

Question: We have a Plesk 9.5.4 server with 2423 domains and according to SELECT * FROM psa about 8811 e-mail accounts.
When we do a backup it takes about three days and the backup file is about 65 GB big.
We need a long term strategy for disaster recovery to solve this problem of long restores. If I restore a 65 GB file that takes three days my clients would leave me in droves. What can I do?
Answer: If you have only a single server running your service(s), it may not be feasible to avoid high down times in case of disaster.
The best you can do is to have another server for higher availability. Then, the appropriate replication mechanisms should be applied to keep both server in sync. When a disaster occurs, you will be safe by failing over to the second server.
For the backup process, you need to make full backup. Then, differential/incremental backups should be enough. I am not sure if there is such option in your control panel.
Comment: Thank you about the advice for another server for HA. As for Plesk e-mail backup, it does not support incremental backups.


A criterion to obtain the feet of two altitudes of an acute triangle

Question: $\triangle{ABC}$ is an acute triangle, and $R$ is the foot of the altitude from $C$. If $H$ and $K$ are the reflections of $R$ across $\overline{\mathit{BC}}$ and $\overline{\mathit{AC}}$, respectively, and if $P$ and $Q$ are the intersections of $\overline{\mathit{HK}}$ with $\overline{\mathit{BC}}$ and $\overline{\mathit{AC}}$, respectively, $P$ is the foot of the altitude from $A$, and $Q$ is the foot of the altitude from $B$.
I would appreciate an explanation to this statement. I know that $\overline{\mathit{CH}} \cong \overline{\mathit{CR}} \cong \overline{\mathit{CK}}$, and $\mathrm{m}\angle\mathit{HCK} = 2 \cdot \mathrm{m}\angle{C}$. (I doubt this is relevant - $\mathrm{m}\angle{A} = \mathrm{m}\angle\mathit{CAK}$ and $\mathrm{m}\angle{B} = \mathrm{m}\angle\mathit{CBH}$.)
Answer: Since $$\angle CRQ = \angle QKC = \angle QHC$$ we have cyclic quadrilateral $QRHC$. But $B$ is also on this circle since $HCRB$ is also cyclic (because $\angle CHB + \angle CRB = 90+90 = 180$). So $BQ\bot AC$.
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/84604/discussion-on-answer-by-greedoid-a-criterion-to-obtain-the-feet-of-two-altitudes).


Ocultar coluna na hora da edição

Question: Tenho uma tela de cadastro com alguns atributos, no qual na hora da edição eu não quero carregar a senha, pois somente quero alterar o nome e o email, ou seja, eu quero ocultar a coluna senha apenas na edição.
OBS: Todo CRUD está funcionando.
CadastroCliente.xhtml:

    <h:panelGrid columns="2">

    </h:panelGrid>

ConsultaCliente.xhtml:

        <h:outputText value="#{cliente.nome}" />

        <h:outputText value="#{cliente.email}" />

            <f:param name="id" value="#{cliente.id}" />

            <f:setPropertyActionListener value="#{cliente}" target="#{consultaClientesBean.clienteSelecionado}" />

OBS: Todo CRUD está funcionando.
Answer: Normalmente na edição o id não será nulo, já que foi criado no banco e tudo o mais.
Se isso for mesmo verdade, você pode usar algo assim:
<h:panelGroup rendered="#{cadastroClientesBean.cliente.id eq null}">

</h:panelGroup>

Se você tem alguma regra mais específica pode fazer alguma coisa no controlador, ou até mesmo no XHTML, se estiver no contexto.

Ocultar coluna na hora da edição


How to update input value using ref in ReactJS

Question: I have one input which i want to update using ref only(not using setState).
constructor(props) {
super(props);
    this.Increase = this.Increase.bind(this)
    this.textInput = React.createRef();
    this.state = {
        inputVal: -1
    };
}

<input type="number" ref={textInput} placeholder="count"/>
Increase

and i have one function where i want to use to increase input value
Increase(event){
//access ref here and update input value using ref
}

Any help will be appreciated.
Thanks
Answer: It's just like changing a mere normal field value.
You could just do :
Increase(event) {
    event.preventDefault();
    this.textInput.current.value += 1;
}

Comment: i want to use ref
Comment: You're actually using ref (the one you created). Or are you talking about the hook??
Answer: The first step to fix this problem is to pass the ref properly:

    <input type="number" ref={this.textInput} placeholder="count"/>
    Increase

then you should add this Increase function:
Increase(event) {
    event.preventDefault();
    this.textInput.current.value = "2" //everything you want;
}

Comment: also you can visit this https://jsfiddle.net/9bgxr372/12/


Determine best clustering algorithm for geospatial data

Question: I have a dataset of longitudes and latitudes for stores in New York City. The data consists of only three columns - longitude, latitude, and store ID.
I want to use python to cluster these stores by using longitude and latitude. Of course ID is not clusterable so I will remove it from the data before clustering.
The type of clusters I want…
I want clusters that are density based. If there's a large number of stores in an area, the algorithm should recognize that. I also want the algorithm to be able to recognize outliers. Third, I don't want the clusters to be drastically different in size. They don't need to be the exact same of course, because density will play a part in that. But at the same time, I don't want a cluster with 100 stores when the average number of stores in a cluster should be around 20. Lastly. I need the algorithm to recognize that this is geospatial data. I need it to cluster accordingly.
What is the best clustering algorithm to use for this geospatial data?
I know the obvious answer seems like DBSCAN. But I promise I have tried it out and have done a lot of research on tuning those parameters. However it doesn't meet the criteria I'm looking for.
What other algorithms will be worth a try?
Comment: I would consider Gaussian mixtures [among others](https://scikit-learn.org/stable/modules/clustering.html). Since there are multiple density-based methods, you may wish to further qualify what you mean by "best clustering algorithm".
Comment: In many data sets, your density requirement is in conflict with clusters being somewhat balanced in size. Did you plot your data and are you sure that reasonable density-based clusters will not differ much in size?
Comment: What do you mean by "I want clusters that are density based". Density, as opposed to what?
Comment: pdfcluster https://cran.r-project.org/web/packages/pdfCluster/index.html is another method for density-based clustering. Not sure whether it exists in python though.
Comment: Could you post a visualization (something like a map) of your data, or of simulated data similar to the real data you're working with?
Comment: I would just do a hierarchical approach with some constraints on a store joining a cluster based not only on distance but metrics of the cluster itself such as store count.
Answer: I am not very familiar with the peculiarities of geospatial data. As a result, I'm not sure what you mean when you say "I need the algorithm to recognize that this is geospatial data".
This sounds like a perfect use case of K-means clustering to me. You essentially have an XY plane, and you need to group the points together based on their literal distances to each other.
I would try K-means, and adjusting the parameters (especially the "number of clusters/means") until you're either visually satisfied, or you can take advantage of some objective measure of clustering quality like the silhouette coefficient.


How to analyze incoming SMS on Android?

Question: How can I code in Android such that my app can analyze an incoming SMS and perhaps block it or do something(maybe move to a different SMS folder) BEFORE the SMS actually raises a notification telling the user of a new SMS? I would target Android 2.1 and above.
I would want to analyse incoming SMS for user specified spam words, and if found would want to delete/mark as read/move the message to a different folder.
Answer: I use this code, as a BroadcastReceiver:
<code>public void onReceive(Context context, Intent intent)
{
    //this stops notifications to others
    this.abortBroadcast();

    //---get the SMS message passed in---
    Bundle bundle = intent.getExtras();
    SmsMessage[] msgs = null;
    String str = "";
    if (bundle != null)
    {
        //---retrieve the SMS message received---
        Object[] pdus = (Object[]) bundle.get("pdus");
        msgs = new SmsMessage[pdus.length];
        for (int i=0; i<msgs.length; i++){
            msgs[i] = SmsMessage.createFromPdu((byte[])pdus[i]);
            str += "SMS from " + msgs[i].getOriginatingAddress();
            from = msgs[i].getOriginatingAddress();
            str += " :";
            str += msgs[i].getMessageBody().toString();
            msg = msgs[i].getMessageBody().toString();
            str += "\n";
        }
        if(checksomething){
            //make your actions
            //and no alert notification and sms not in inbox
        }
        else{
            //continue the normal process of sms and will get alert and reaches inbox
            this.clearAbortBroadcast();
        }
  }
</code>
remember to add it in manifest and add a higgest priority (100) for broadcast or sms will go first to inbox and get the alert notification.
<code>    <receiver android:name=".SmsReceiver">
        <intent-filter android:priority="100">
            <action android:name="android.provider.Telephony.SMS_RECEIVED"></action>
        </intent-filter>
    </receiver>
</code>
Hope it helps you.
Answer: Well u can trap the incoming SMS, but i think u will not be able to block the notification .....
If u want to delete SMS here is a thread that can help....
How to delete an SMS from the inbox in Android programmatically?
Comment: Now abortBroadcast(); method can be used for restricting the incoming message to go to inbox.
Comment: well I need Android to NOT show the default 'new message' notification if my application has already handled the incoming SMS. So that is an important requirement for the best user experience :(
Comment: well there are many threads on StackOverflow that will tell u it's not possible without using your own extended classes of those that are hidden....... but yes you wan wait for other's reply also........
Answer: This code works on my 2.3.3 device. HTC MyTouch 4g Slide.
the abortBroadcast suppresses the notificationsound + notification on status bar + does not allow the SMS to go to the inbox.
Some users had mentioned that it does not work on real devices and works only on emulator, that is not always the case. If the priority is 100 then on this particular device the code works as expected.

How to analyze incoming SMS on Android?


How can I verify that my solution is correct when finding the PDF and CDF of a random process?

Question: If I have $X(t) = \sin(\omega t + \theta)$ where $\omega$ is constant and $\theta$ ~ $U[0,2\pi]$, and I need to find the PDF and CDF of $X(t)$, then how can I confirm my solutions for the PDF and CDF are valid?
Answer: Apart from the usual mathematical techniques of checking your derivation, checking for consistency (e.g., differentiating the CDF to see whether it equals the PDF), and considering whether your answer makes sense, whenever there is a question of determining the distribution of a random variable (or collection of variables) you almost always have available the option of simulation.
In many cases this is a mechanical, nearly mindless exercise -- and it is carried out using such different reasoning processes (it's a programming exercise rather than a mathematical one) that it serves as an excellent check.
For this purpose it's a good idea not to be clever or efficient about the simulation: make the code reflect the information you have as closely as possible.  In the example below, the process $X$ is computed as
X <- outer(theta, times, function(theta, t) sin(omega*t + theta))

That's simple to code and easy to compare to your formula.  The rest of the code is just pre- and post-processing.
You can find thousands of worked examples here on Cross Validated: search for names of common uniform random number generators, such as runif and rnorm, made popular on the R platform.
As an example, below is R code to simulate your process for eight randomly-chosen times $t$ and display histograms of the simulated values.  Over those histograms are plotted the graphs of the PDF I think is the solution.  You can see these graphs agree closely with the histograms, thereby bolstering the likelihood I am right.  (You can perform a distributional goodness of fit test of each graph for a more rigorous check, but that's a matter for a separate thread.)  If you wish to check the CDF, plot the empirical cumulative distribution functions (available through the ecdf function in R) and overplot those with your CDF to compare.  See, inter alia, https://stats.stackexchange.com/a/471775/919, https://stats.stackexchange.com/a/472768/919, or https://stats.stackexchange.com/a/464101/919 for examples.

The code is below.  Because you are not asking for the answer, I have omitted the body of the PDF function f.  Nevertheless, this figure already reveals a lot: the distribution of $X(t)$ does not vary with $t.$
#
# Specify the process by initializing its parameters.
#
omega <- 4/3
#
# Select a set of "times" at which to sample it.
#
times <- sort(runif(8, 0, 2*pi/omega))
#
# Sample the process `n` times, storing each independent sample in a row of `X`.
#
n <- 1e5
theta <- runif(n, 0, 2*pi)
X <- outer(theta, times, function(theta, t) sin(omega*t + theta))
#
# Plot histograms of each X(t).
#
h <- function(x, t) {
  hist(x, xlim=c(-1,1), freq=FALSE, breaks=50, col="#f8f8f8", border="#505050",
       xlab=expression(X[t]), cex.lab=1.25,
       main=paste0("t=",signif(t,3)), cex.main=1)
  f <- function(x) # (Body omitted.)
  curve(f(x), add=TRUE, n=201, lwd=2, col="Red")
}
par(mfrow=c(2,4))
invisible(sapply(seq_along(times), function(i) h(X[,i], times[i])))
par(mfrow=c(1,1))

Comment: That was a very thoughtful response. Thank you for spending the time to introduce this capability to me


Beginner Programming SO with System Call

Question: I have to sum two number while first number is not equal to -1, and the numbers have just one digit.
I have to use read() and write().
#include <unistd.h>
#include <errno.h>

int main()
{   int Somma;
    int One;
    int Two;

    do
    {   write(1, "\nFirst Number: ", 15);
        if(read(0, &One, sizeof(int)) == -1)
            perror("Error First Read");

        if(One != -1)
        {   write(1, "Second Number: ", 15);
            if(read(0, &Two, sizeof(int)) == -1)
                perror("Error Second Read");

            Somma = One + Two;
            Somma -= 48;

            write(1, "Sum: ", 5);
            if(write(1, &Somma, sizeof(int)) == -1)
                perror("Error Write");
        }

    }while(One != -1);

    return 0;
}

Now, I have some problems. First of all, when One is equal to -1, the program continues into if statement...
The second one is that the last write(), print the number and a strange characters (a square with 0014 code into...).
What's wrong?
Thank you in advance
Comment: Oh ok. Besides from the answer below, I'd like to say that since you're working on linux, try a little not to capitalize the first letter in your variable names. Another thing is, use `STDIN_FILENO` instead of 0 and `STDOUT_FILENO` instead of 1 in `read(2)` and `write(2)` calls. It'll be easier to read.
Comment: Is there any reason you're not using glibc functions like printf(3) and scanf(3)? You're using `read(2)` wrong, reading 4 bytes, which is equal to 4 characters. `read(2)` reads binary data, not actual characters.
Comment: The professor says that we have to use read() and write() in unistd.h library...
Comment: Are you supposed to read the numbers via stdin?
Comment: In this exercize, I have to read via stdin...in the next one I will have to read from an input file...
Answer: I guess you are on Linux or some POSIX system.
The read(2) & write(2) syscalls are reading bytes. On modern Linux systems the terminal is usually using UTF-8 encoded strings (which you might process using libunistring if you have to). For efficiency reasons, you should (manually) buffer the I/O, i.e. try to read or write in blocks of several kilobytes (typically 16Kbytes).
You should obviously handle the byte count (returned by these syscalls). Don't forget to handle error conditions (e.g. by displaying them with perror(3) then exit(EXIT_FAILURE);)
Your code is wrong, because sizeof(int) is often 4, and your call to read might read only 1,2, or 3 bytes (but very often, it would read 4 bytes). You have to handle buffering, and that is part of the exercise.
You could use sscanf(3) and snprintf(3) to make the string to int conversions and backwards. Don't forget the error cases.


Heroku geolocation dns and Amazon Route 53

Question: I have an application running on Heroku in 2 regions, eu and us. Let's say: myapp-eu.herokuapp.com and myapp-us.herokuapp.com.
I would like to setup a geolocation dns that points users to the nearest region when they visit our site at www.myapp.com.
What I've done until now is using Amazon Route 53 to setup 2 CNAME record:

www.myapp.com CNAME myapp-eu.herokuapp.com (when the geo is eu)
www.myapp.com CNAME myapp-us.herokuapp.com (when the geo is us)

But Heroku does not accept the same CNAME to be used in 2 different apps.
Anyone has successfully setup a geo dns that works with Heroku please?
Thanks!
Comment: One solution I've found is to use 2 additional servers outside heroku router, in which I would install haproxy. So, I'd have haproxy-eu and haproxy-us that point to heroku apps correspondingly. In each of these proxies, I would modify the host header to by-pass heroku host checking.

But it's seem over complicated :). The question now is that whether there is a dns server that support geolocation and host header modification?
Answer: I was going to post the solution with HAProxy, which I think I have given as an answer here before (though a search didn't turn it up).
That (or something similar, you could also use varnish or nginx) will be the only viable approach, given the global app namespace limitation of Heroku, since no DNS service can do what you are wanting -- host header rewrites cannot be accomplished with DNS configuration alone.  The request will have to pass through a system of some kind that can rewrite the header in flight.
However, you would only need one proxy, not two. Here's why:
If the hostname is myapp.example.com, then configure the US Heroku deployment to simply expect that hostname.
Then don't configure the EU deployment to expect a custom hostname; you'll be using the non-vanity hostname myapp-eu.herokuapp.com.
Route 53 would be configured to return the Heroku endpoint as the answer for US queries, and the proxy endpoint for EU requests.  The proxy would rewrite the host header to myapp-eu.herokuapp.com and send the request to the Heroku EU endpoint, but US requests would go directly to the Heroku US endpoint, which would expect the hostname the client is already using.
You could also avoid the proxy and use CloudFront as the proxy, instead.  Note that this solution only works with 2 destinations -- US and EU in this case -- due to the global configuration restrictions of CloudFront (only 1 CloudFront distribution can be configured for a given incoming hostname)... but for this solution, 1 is all we need.  One destination (US) uses a direct connect and requires no rewrite, while the other destination (EU) proxies through CloudFront and gets a rewrite.
Create a CloudFront distribution.  Configure it to expect requests for myapp.example.com.  Configure it to use the non-vanity Heroku EU endpoint hostname, myapp-eu.herokuapp.com as its custom origin server, and do not configure it to whitelist the Host header from the original request.   Whitelist any other headers you do require.  Optionally, disable caching.  CloudFront will then rewrite the Host: header from myapp.example.com to the configured Origin server hostname, which would be the EU endpoint.
Then, as before, configure Route 53 to return the US Heroku endpoint CNAME for requests from locations that should go to the US endpoint, but to return the CloudFront dxxxxxxxx.cloudfront.net CNAME for requests from locations that should go to the EU endpoint.
Either way, you'll be paying for transport of the traffic you send to the endpoint that goes through the proxy or CloudFront, so you might want to use that for the endpoint that you expect to see the lower amount of traffic... so you may want to transpose EU and US in the above examples, which arbitrarily assume US would see more traffic, so it routes directly.
Neither of these solutions should add significant round-trip time to requests.  If you go the proxy route, you are likely to find that a very small machine, like the t2.micro or t2.nano can handle more traffic that you might expect, since processing a minimal.
Comment: Thank you for the detailed response! I think I'd go for the proxy solution.


History and future of algebraic curves and the like?

Question: Now that Fermat's last theorem has been proven, and also elliptic curves see widespread use in simple everyday applications, I would love to learn how the related theories came into beeing, how they evolved, and what the unsolved problems are in this corner of mathematics. I want to see the landscape, so to say.
The history has already been discussed to some extent here: History of elliptic curves
So... what are the big open questions? What are the lesser ones?
Comment: I think you need to be more specific about what you're looking for. Even then the answers are probably going to turn into Wikipedia articles. Anyway, BSD is still open. Get to work!
Answer: Your question covers a vast amount of material and without specifying further it is hard to say exactly what you're looking for. Also, you did not state the level of material you were looking for. Almost any text on Algebraic Geometry (or more specifically Arithmetic Algebraic Geometry), Elliptic Curves, or Number Theory related to these areas would contain such problems and their history.
To name a few texts at various level:
Elliptic Curves: Milne : A classic introductory text to these areas.
Elliptic Tales : A nice general historical overview with some discussion of open problems.
Rational Points on Elliptic Curves : An undergraduate text relating to these areas. It does talk about some open problems.
Arithmetic of Elliptic Curves : A very dense graduate text introducing elliptic curves.
Elliptic Curves, Modular Forms, and Their L-functions  : A more readable, accessible to undergraduates, text spanning elliptic curves and open problems.
Elliptic Curves : Another graduate introductory text to elliptic curves. I found this more accessible than Silverman's book for a first approach.
Elliptic Curves: Number Theory and Cryptography : Another great undergraduate book for elliptic curves. This is probably less difficult than the book of Silverman.
Arithmetic Geometry : A dense introduction to the results/research of arithmetic geometry.
Arithmetic Algebraic Geometry : Yet another dense introduction to the results/research of arithmetic geometry.
Journey through Genius : A well written text discussing several big math problems. Though this will not focus on your areas of interest for much of the book, it is a cheap and good read.
Fermat's Last Theorem : A general introductory text to these areas through Fermat's Last Theorem.
Modular Forms and Fermat's Last Theorem : Another general introductory text to these areas through Fermat's Last Theorem.
Fermat's Last Enigma : A more readable book on the history of FLT.
As for easy access papers:
Elliptic Curves: Milne : Milne's book referenced above.
Elliptic Curves : Lecture notes on elliptic curves.
Introduction to Fermat's Last Theorem : A nice history by Cox on the history of FLT.
Fermat's Last Theorem : A readable introduction to the theories going into FLT.
The ABC Conjecture : A discussion on another big (possibly still open, a proof is being verified) open problem.
From the Taniyama-Shimura Conjecture to Fermat's Last
Theorem
Wiles' Proof of Fermat's Last Theorem : A discussion of the techniques that went into FLT.
36 Unsolved Problems in Number Theory : A list of unsolved number theory problems.
More Unsolved Problems : More unsolved problems.
Introduction to Arithmetic Geometry : A nice introductory text to arithmetic geometry.
Roadmap to Learning Arithmetic Algebraic Geometry : Same as above
I do believe there are several OCW courses on the MIT website whose notes you could look through that would also give a bit more material for you to look through. Hopefully, this is what you were hoping for. Possible areas for you to look into would be the ABC conjecture (you could look here for more on this but these papers are accessible to VERY few people), BSD (Birch and Swinnerton-Dyer Conjecture), you might look into Perfectoid Spaces, ranks of elliptic curves, the Langland's Program, etc etc etc. As I said, the field is very big and vibrant. It depends on if you are looking more towards the elliptic curve end, algebraic number theory side, or algebraic geometry side of things.
Comment: Thanks for a splendid answer. I was looking for an introductory treatment and the links seem very interesting, My level of understanding is advanced undergraduate - fresh graduate.

It is hard to know what one does not know, hence the diffuse question.
Comment: @R.s Not a problem. This is my general area that I am working in for my PhD so I always have links ready. Note that many of the texts can be found for free online with searching. I did not link them here as I am unaware which are available legally and which are not. Since you seem new to the area, perhaps you should try some of the general interests texts first before trying texts which specifically try to introduce you to the actual mathematics of the field - which often require years of study because of the vast amount of mathematics the materials touches upon: algebra, analysis, geo., cetc
Comment: The book by Lawrence Washington, Elliptic Curves: Number Theory and Cryptography, is a really nice introduction (with proofs), although I'm only half way through it. Not familiar with the other texts, but will look at them. The third one about rational points, or one book by Silvermann (don't recall the name; starts with an introduction to algebraic curves), might be my next reads.
Comment: @R.s Many of them are good but the problem very quickly becomes that introductory texts take 'short cuts' by presenting material which can be easily presented to undergraduates. When trying to find a slightly more advanced book, one often finds the difference between the previous book and the next very steep. This is because the usual techniques of the field require commutative alg, alg. geometry, and various pieces of analytic and alg number theory. If you can follow it, Husemullers book would be a good next step or "Primes of the form $x^2+ny^2$". It all depends on your level of interest.
Comment: Found it: "Silverman: The arithmetic of elliptic curves" is what I was thinking about initially, but now I will try to locate the Husemullers text.
Comment: I would personally say Husemullers book is better to start with. Without doubt, Silvermans covers more ground but it moves very quickly, the exercises are more difficult, and it requires far more knowledge going in. I moved on to many more texts before coming back to Silvermans. Though if you're looking to just read the material and not learn it, Elliptic Curves, Modular Forms, and their L-functions would be the best route. Good luck with the reading!


Can regions be printed within a node template?

Question: Can regions be printed within a node template, or are regions strictly confined to page templates?
Answer: You can print regions inside any template, but they aren't available out of the box in the node.tpl.php template. To make them available, you'll create a new variable for use in your node.tpl.php template that'll contain all the region content.
Creating new template variables is done by using a preprocess function. In your theme's template.php file, create a function that looks like this:
function mytheme_preprocess_node(&$variables) {
}

Replacing mytheme with your theme's short name. Now, in order for Drupal to recognize this new preprocess function, you need to rebuild your site's theme registry. This is as straightforward as going to Configuration → Development → Performance and pressing the "Clear all caches" button at the top.
Now, the way preprocess functions work is that $variables contains an array that corresponds to your template's available variables. For example, in node.tpl.php, $submitted contains the author byline. In the preprocess function above, it's found in $variables['submitted'].
To mimick what you have in page.tpl.php, where you have an array called $page that contains all the regions, you'd want to populate $variables['page'].
The problem is that $page in node.tpl.php is already populated with a true/false value that lets you know whether you're looking at the node by itself or in a listing.
So to avoid that name collision, populate $region instead:
function mytheme_preprocess_node(&$variables) {

  // Get a list of all the regions for this theme
  foreach (system_region_list($GLOBALS['theme']) as $region_key => $region_name) {

    // Get the content for each region and add it to the $region variable
    if ($blocks = block_get_blocks_by_region($region_key)) {
      $variables['region'][$region_key] = $blocks;
    }
    else {
      $variables['region'][$region_key] = array();
    }
  }
}

Then, in your theme's node.tpl.php template, you can render any region by doing the following:
<?php print render($region['sidebar_first']); ?>

Where sidebar_first is the name of the region you want to render.
Comment: There's also a quick 'n' dirty solution that worked for me ``. From here: http://www.webomelette.com/add-region-node-template
Comment: Finally a working solution for Drupal 7 :)
Comment: This solution is nifty. Could you give examples as to when is it most applicable to print regions within node.tpl.php instead of having it in page.tpl.php?
Comment: It depends on your design. Sometimes the layout might put the sidebar in a place where it is much easier to theme if it were inside the node div instead of outside of it.
Comment: Don't forget about appropriate region wrappers! See [system_page_alter()](https://api.drupal.org/api/drupal/modules%21system%21system.module/function/system_page_alter/7) for reference. Remember that `block_get_blocks_by_region()` returns only array of blocks and if you want to render it as a region, you need to add region wrapper.


google sites on a private cloud

Question: Can google sites/apps be installed on your own infrastructure say the way Drupal or Wordpress could?
I can't seem to find resources pointing me towards the way to do this. Appreciate any pointers.
Answer: Simply put: No .
Google sites/apps have tiers of 'security'/'privacy' (Government, education, business); but nothing you can put on your own hosts.
Comment: Thanks - that is what I thought - I was led to believe that Google might have an onsite installable option - but I came accross nothing online to support that assumption. This has been validating - appreciate it!
Answer: No, in short. A lot of the marketing towards it is geared around the fact that it will be a decentralised, outsourced, managed system. Locally hosting would pretty much defeat the point.
Answer: There are a number of options out there to host your own services similar to what Google offers, either on a private cloud, or more traditionally.
Zimbra is one example, and there are plenty of others.

google sites on a private cloud


Override child component's internal quasar components CSS in VueJS

Question: I'm using quasar components in VueJS to make an app. I have a child component being imported in parent component. For the child, I have overrided and applied custom css by accessing the quasar classes directly in its style tag. The problem is after importing child in parent class, the parents css is applied on child. Even if I use scope,module or deep it has no effect as I want to change the quasar components internal root css.
I have also tried setting the child's style on runtime in created() but it does not have any effect and doesn't accept !important keyword as well.
Is there any way to apply custom child's css on quasar components root/internal css and not let the parent css get applied on child? Please help.
Answer: You can use v-deep to override Quasar's classes in the style section.

Please note that your first class (.q-item in my example) must be the given class of the Quasar's component that you use. If you use q-item component, and inside it Quasar put q-item__label, and inside it q-item__ something...  use q-item as first argument above v-deep as in my example.
Comment: in vue 3 v-deep must be a function, so write v-deep()
Answer: you must remove scope field from your style tag, then you can override quasar classes.
because when you have scope, the framework append hash code to all your class and id name in your style and your class name has been changed.
Answer: you can something like this . this is an example for override quasar toggle component
<template>
 <q-toggle
      ref="qToggleRef"
      style=""
      outlined
      :disable="props.disable"
      :label="void 0"
      bg-color="grey-1"
    >

    </q-toggle>
</template>


Angular 6 Array of http requests fails

Question: I need some help with this loop of http requests, Here i am trying to loop some requests and push response into an array but the code exists before i get any response. Thank you for your help and suggestions
 publicgetMultipleTeamsData(teamIds:Array<string>,dateRange:any):
    Observable<any[]>{
          let response = [];
          teamIds.forEach(teamId => {
            this.getTeamDetails(teamId, dateRange)
            .map( tdata => {
              response.push(tdata);
            });
          });
          return Observable.forkJoin(Observable.of(response));
        }

Answer: Seems that you planned to put some data to response in map and then forkJoin all responses but it doesn't work so :).
As for me, you should do in such way:
public getMultipleTeamsData(teamIds:Array<string>,dateRange:any): Observable<any[]>{
    let requests = [];
    teamIds.forEach(teamId => {
      requests.push(this.getTeamDetails(teamId, dateRange))
    });
    return Observable.forkJoin(...requests);
}

Answer: You can try this
public getMultipleTeamsData(teamIds:Array<string>,dateRange:any) {
 let arrObservables = teamIds.map(teamId => this.getTeamDetails(teamId, dateRange))

 return Observable.forkJoin(...arrObservables);
}

Answer: securityActions() {
  let response = [];
  teamIds.forEach(teamId => {
    this.getTeamDetails(teamId, dateRange)
      .map(tdata => {
        response.push(tdata);
      });
  });
  return Observable.forkJoin(Observable.of(response));
}

Let's talk it out :

create an array of responses
  for each team ID, make an HTTP and push the response in it once the call is done
  Don't wait for the calls to end, and make HTTP calls out of the responses.
  Don't subscribe to the calls, which doesn't trigger them

If you want to make it work, try something like this :
return forkJoin(teamIds.map(ID => this.getTeamDetails(ID, dateRange)));

Angular 6 Array of http requests fails


how to build complex array with questions and answers? - javascript

Question: i have:
var answers = [];

                var frage1 = document.getElementById("frage1").innerHTML;
                frage1antwort1 = $('#frage1antwort1:checked').length; f.Example Value = 0
                frage1antwort2 = $('#frage1antwort2:checked').length; f.Example Value = 0
                frage1antwort3 = $('#frage1antwort3:checked').length; f.Example Value = 1

                var frage2 = document.getElementById("frage2").innerHTML;
                frage2antwort1 = $('#frage2antwort1:checked').length; f.Example Value = 0
                frage2antwort2 = $('#frage2antwort2:checked').length; f.Example Value = 0
                frage2antwort3 = $('#frage2antwort3:checked').length; f.Example Value = 1

                var frage3 = document.getElementById("frage3").innerHTML;
                frage3antwort1 = $('#frage3antwort1:checked').length; f.Example Value = 1
                frage3antwort2 = $('#frage3antwort2:checked').length; f.Example Value = 0
                frage3antwort3 = $('#frage3antwort3:checked').length; f.Example Value = 0

                var frage4 = document.getElementById("frage4").innerHTML;
                frage4antwort1 = $('#frage4antwort1:checked').length; f.Example Value = 0
                frage4antwort2 = $('#frage4antwort2:checked').length; f.Example Value = 1
                frage4antwort3 = $('#frage4antwort3:checked').length; f.Example Value = 0

                var frage5 = document.getElementById("frage5").innerHTML;
                frage5antwort1 = $('#frage5antwort1:checked').length; f.Example Value = 0
                frage5antwort2 = $('#frage5antwort2:checked').length; f.Example Value = 1
                frage5antwort3 = $('#frage5antwort3:checked').length; f.Example Value = 0

i am trying to build an array with questions and selected answer for it. nothing is right by me.
as result [Question=1, Answer= 3], something like that. please help!
Comment: I don't see where you actually push values onto the `answers` array.
Comment: all my ideas are wrong, and its not working. i hope for some clean solutions from you...
Answer: why are you trying arrays ? you can do the same easily with objects.
please check http://www.w3schools.com/js/js_objects.asp
for Example
function question(que,ans1,ans2,ans3)
{
this.question=que;
this.answer1=ans1;
this.answer2=ans2;
this.answer3=ans3;
}

and then
var frage1=new question(document.getElementById("frage1").innerHTML, $('#frage1antwort1:checked').length,$('#frage1antwort2:checked').length,$('#frage1antwort3:checked').length);

you just need to define question object then create new objects or assign the values and grab whenever required.
This will give you clear idea Working with objects
Comment: you just need to define question object then create new objects or assign the values and grab whenever required.
Comment: thank you it works well! can i save all 5 question in one object which i can build as a result for responce??
Comment: it will be easy if you create an array of 5 question object :)
Comment: yes i did it like this:  function surveyResultFull(frage1, frage2, frage3, frage4, frage5) {
            this.frage1 = frage1;
            this.frage2 = frage2;
            this.frage3 = frage3;
            this.frage4 = frage4;
            this.frage5 = frage5;
        }
Comment: and filling it here:  var frage5 = new surveyResult(document.getElementById("frage5").innerHTML, $('#frage5antwort1:checked').length, $('#frage5antwort2:checked').length, $('#frage5antwort3:checked').length);

                var result = surveyResultFull(frage1, frage2, frage3, frage4, frage5);


Input form table: Dynamically created delete button at end of row does not work

Question: When clicking on "Insert Row" I am not able to make the "Delete" button for this dynamically created row work. Does anybody know the reason?
I guess it is because the deleteRow() function needs a row index, which is not yet defined in the moment of row creation. But I have no clue how to circumvent this problem.
I'd like to avoid using jQuery. Any help is appreciated.

function deleteRow(r) {
  var i = r.parentNode.parentNode.rowIndex;
  document.getElementById("myTable").deleteRow(i);
  console.log(i);
}

function insertRow(r) {
  var table = document.getElementById("myTable")
  var row = table.insertRow(-1);

  var type = ["text",           "text",         "number",     "number"];
  var name = ["crypto_ticker",  "crypto_name",  "crypto_qty", "crypto_invested_sum"];
  var placeholder = ["e.g. BTC","e.g. Bitcoin", "e.g. 5",     "e.g. 2000"];
  var required = ["true",       "true",         "true",        "true"];

  for (n=0;n<4;n++) {
    var cell=row.insertCell(n);
    var t=document.createElement("input");
      t.type = type[n];
      t.name = name[n];
      t.placeholder = placeholder[n];
      t.required = required[n];
    cell.appendChild(t);
  }

  var cell1=row.insertCell(4);
  var button=document.createElement("button");
    button.id = "myButton";
    button.textContent = "Click me";
    button.addEventListener("click", deleteRow(this), true);
  cell1.appendChild(button);

}

Übersicht der Coins

        Ticker
        Name
        Anzahl
        Investierte Summe
        Löschen

          <input type="text" name="crypto_ticker" value="BTC" placeholder="e.g. BTC" required="true"/>

          <input type="text" name="crypto_name" value="Bitcoin" placeholder="e.g. Bitcoin" required="true"/>

          <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

          <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

          <button onclick="deleteRow(this)">Delete</button>

          <input type="text" name="crypto_ticker" value="ETH" placeholder="e.g. BTC" required="true"/>

          <input type="text" name="crypto_name" value="Ethereum" placeholder="e.g. Bitcoin" required="true"/>

          <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

          <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

          <button onclick="deleteRow(this)">Delete</button>

<button onclick="insertRow()">Insert Row</button>
Answer: In your deleteRow() function you can use this which will refer to your button (the one you clicked on). Then follow the tree up to the TR you want to remove
function deleteRow() {
  var row = this.parentNode.parentNode;
  console.log(row);
  row.parentNode.removeChild(row);
}

function deleteRow() {
  console.log(this)
  var row = this.parentNode.parentNode;
  console.log(row);
  row.parentNode.removeChild(row);
}

function insertRow(r) {
  var table = document.getElementById("myTable")
  var row = table.insertRow(-1);

  var type = ["text", "text", "number", "number"];
  var name = ["crypto_ticker", "crypto_name", "crypto_qty", "crypto_invested_sum"];
  var placeholder = ["e.g. BTC", "e.g. Bitcoin", "e.g. 5", "e.g. 2000"];
  var required = ["true", "true", "true", "true"];

  for (n = 0; n < 4; n++) {
    var cell = row.insertCell(n);
    var t = document.createElement("input");
    t.type = type[n];
    t.name = name[n];
    t.placeholder = placeholder[n];
    t.required = required[n];
    cell.appendChild(t);
  }

  var cell1 = row.insertCell(4);
  var button = document.createElement("button");
  button.class = "myButton";
  button.textContent = "Click me";
  button.addEventListener("click", deleteRow, true);
  cell1.appendChild(button);

}

Übersicht der Coins

      Ticker
      Name
      Anzahl
      Investierte Summe
      Löschen

        <input type="text" name="crypto_ticker" value="BTC" placeholder="e.g. BTC" required="true" />

        <input type="text" name="crypto_name" value="Bitcoin" placeholder="e.g. Bitcoin" required="true" />

        <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true" />

        <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true" />

        <button onclick="deleteRow()">Delete</button>

        <input type="text" name="crypto_ticker" value="ETH" placeholder="e.g. BTC" required="true" />

        <input type="text" name="crypto_name" value="Ethereum" placeholder="e.g. Bitcoin" required="true" />

        <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true" />

        <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true" />

        <button onclick="deleteRow()">Delete</button>

<button onclick="insertRow()">Insert Row</button>
Comment: I was a bit too quick in answering.I can delete the newly created row. But if I try to delete one of the two rows, which are already in I get the same error message again in your solution.
Answer: Adding the deleteRow function right to the newly created button works, even though inline functions are supposed to be avoided. If somebody knows a more elegant way please tell me.

    function deleteRow(r) {
      var i = r.parentNode.parentNode.rowIndex;
      document.getElementById("myTable").deleteRow(i);
    }

    function insertRow(r) {
      var table = document.getElementById("myTable")
      var row = table.insertRow(-1);

      var type = ["text",           "text",         "number",     "number"];
      var name = ["crypto_ticker",  "crypto_name",  "crypto_qty", "crypto_invested_sum"];
      var placeholder = ["e.g. BTC","e.g. Bitcoin", "e.g. 5",     "e.g. 2000"];
      var required = ["true",       "true",         "true",        "true"];

      for (n=0;n<4;n++) {
        var cell=row.insertCell(n);
        var t=document.createElement("input");
          t.type = type[n];
          t.name = name[n];
          t.placeholder = placeholder[n];
          t.required = required[n];
        cell.appendChild(t);
      }

      var cell=row.insertCell(4);
      var b = document.createElement("input");
        b.type = "button";
        b.value = "Delete";
        b.onclick = function() {
          var i = b.parentNode.parentNode.rowIndex;
          document.getElementById("myTable").deleteRow(i);
        };
      cell.appendChild(b);
    }

Übersicht der Coins

        Ticker
        Name
        Anzahl
        Investierte Summe
        Löschen

          <input type="text" name="crypto_ticker" value="BTC" placeholder="e.g. BTC" required="true"/>

          <input type="text" name="crypto_name" value="Bitcoin" placeholder="e.g. Bitcoin" required="true"/>

          <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

          <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

          <button onclick="deleteRow(this)">Delete</button>

          <input type="text" name="crypto_ticker" value="ETH" placeholder="e.g. BTC" required="true"/>

          <input type="text" name="crypto_name" value="Ethereum" placeholder="e.g. Bitcoin" required="true"/>

          <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

          <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

          <button onclick="deleteRow(this)">Delete</button>

<button onclick="insertRow()">Insert Row</button>
Answer: Your code was pretty good.  Just made 2 mistakes.
Just needed to tweak your JavaScript & it works fine.

    function insertRow(r){
      var table = document.getElementById("myTable")
      var row = table.insertRow(-1);

      var type = ["text",           "text",         "number",     "number"];
      var name = ["crypto_ticker",  "crypto_name",  "crypto_qty", "crypto_invested_sum"];
      var placeholder = ["e.g. BTC","e.g. Bitcoin", "e.g. 5",     "e.g. 2000"];
      var required = ["true",       "true",         "true",        "true"];

      for (n=0;n<4;n++) {
        var cell=row.insertCell(n);
        var t=document.createElement("input");
          t.type = type[n];
          t.name = name[n];
          t.placeholder = placeholder[n];
          t.required = required[n];
        cell.appendChild(t);
      }

      var cell1=row.insertCell(4);
      var button=document.createElement("button");
        button.textContent = "Delete";
        button.onclick = function(){deleteRow(this);};
      cell1.appendChild(button);

    }
    function deleteRow(r){
        r.parentNode.parentNode.parentNode.removeChild(r.parentNode.parentNode);
    }

Übersicht der Coins

            Ticker
            Name
            Anzahl
            Investierte Summe
            Löschen

              <input type="text" name="crypto_ticker" value="BTC" placeholder="e.g. BTC" required="true"/>

              <input type="text" name="crypto_name" value="Bitcoin" placeholder="e.g. Bitcoin" required="true"/>

              <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

              <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

              <button onclick="deleteRow(this)">Delete</button>

              <input type="text" name="crypto_ticker" value="ETH" placeholder="e.g. BTC" required="true"/>

              <input type="text" name="crypto_name" value="Ethereum" placeholder="e.g. Bitcoin" required="true"/>

              <input type="number" name="crypto_qty" value="5" placeholder="e.g. 5" required="true"/>

              <input type="number" name="crypto_invested_sum" value="3000" placeholder="e.g. 2000" required="true"/>

              <button onclick="deleteRow(this)">Delete</button>

    <button onclick="insertRow()">Insert Row</button>


twitter api not working with foreach loop in php

Question: I cannot get the foreach loop to display any data from the twitter api
Please see code below. However, it does display data on print_r array.
thank you for your time in advance
     ini_set('display_errors', 1);
     require_once('TwitterAPIExchange.php');

 /** Set access tokens here - see: https://dev.twitter.com/apps/ **/
  $settings = array(
      'oauth_access_token' => "*******",
       'oauth_access_token_secret' => "*******",
       'consumer_key' => "********",
       'consumer_secret' => "*******"
      );

       /** Perform a GET request and echo the response **/
       /** Note: Set the GET field BEFORE calling buildOauth(); **/
       $url = 'https://api.twitter.com/1.1/search/tweets.json';
        $getfield = '?q=love+you&count=20';
        $requestMethod = 'GET';
        $twitter = new TwitterAPIExchange($settings);
        $string = json_decode($twitter->setGetfield($getfield)
         ->buildOauth($url, $requestMethod)
         ->performRequest(),$assoc = TRUE);
        if($string["errors"][0]["message"] != "") {echo "
Sorry, there
          was a problem.

Twitter returned the following error
       message:

".$string[errors][0]["message"]."

";exit();}

    foreach($string as $tweets) {

      echo $tweets['name'] . '
';

    }

    cannot get the foreach loop to display any data from the twitter api

Please see code below. However, it does display data on print_r array.
thank you for your time in advance
 /**here is the a sample of the array data but when the foreach loop is
 added the screen is blank no error is dispayed no data is displayed**/

Array
(
    [statuses] => Array
    (
        [0] => Array
            (
                [metadata] => Array
                    (
                        [iso_language_code] => en
                        [result_type] => recent
                    )

                [created_at] => Thu Mar 05 23:16:36 +0000 2015
                [id] => 573623174589472768
                [id_str] => 573623174589472768
                [text] => @TheVampsband #TheVampsVIP Please choose me and
    @Charlotte_94x to winShe hit tweet limitWe love you so much! We
     beg89
                [source] => Twitter for iPhone

Comment: If this code display something in `print_r()`, then attach some sample. Btw, in your `json_decode()` function, change second paramater to just `TRUE` instead of `$assoc = TRUE`. It's OK, but not necessary.
Comment: ok attached a little sample but the array displays all the data from a user
Comment: There is missing start of the `print_r()` which is more important to solve your problem.
Answer: So first of all you loop over wrong array. Array you looking for is under statuses index, so change your loop into:
foreach($string['statuses'] as $tweets) {
    echo $tweets['name'] . '
';
}

Look at your print_r() and check carefully how sub array are build. E.g. if you want to print name of each tweet author, you should modify yor echo in foreach to:
echo $tweets['user']['name'] . '
';

because output array has structure
array(..., 'user' => array('name' => 'Example name', ...));

Comment: Analyze your output array carefully. Each subarray need to be accessed by index to get its value.


Is it mandatory to select Devices as Universal for iPhone apps?

Question: I am working on an iOS app that needs to run on iphones only. Therefore, on deployment settings, I have selected Devices as iPhone instead of Universal (screenshot below)

When I went through Apple's App Store Review Guidelines, I came across the statement: "iPhone apps should run on iPad whenever possible" (screenshot below)

Then, I ran my sample app on iPad simulators (10.5 inch and 12.9 inch). Following are the screenshots of the sample app running on those simulators.

On clicking the resize button at the bottom-right, the screenshots look as below.

Is there any possibility of my app being rejected during the review process considering the settings I have set? Or do I need to set the app as Universal?
Comment: No, it's not necessary to deploy your app as universal. There are lot of apps that are for iphone only
Comment: Those screenshots look OK to me; If your app is iPhone only then that is how it is expected to look on an iPad
Answer: As per Apple Guideline Your iPhone app should run successfully on an iPad as well but it will be presented with a scaled screen (letter box view).
Only if you are using any device specific capabilities UIRequiredDeviceCapabilities  then you have to mention that one in info.plist file. So that your app will be visible only for iPhone and NOT iPad on App Store like WhatsApp app.


How to add Parent and Child to a Views 3 table format using Groupping Field?

Question: Working with views and a taxonomy list is a popular subject at DA. Many of the examples I've found refer to making a jump list of taxonomy terms and parent terms similar to what you may see with a nested vocabulary on your site's core taxonomy page [1] (following the list terms link on any given vocabulary). My Views view goal is just as simple and I'm nearly there except for two important requirements.
My Views view is a catalog of node content titles and truncated descriptions categorized by a two level vocabulary. (Think of finding something at a hardware store). The goal is a single page without a sorting requirement (think of it as a very small hardware store, so I skipped the answers suggesting wild card sorting in the URL). I'm nearly there using the table format where I've used the setting Groupping Field Nr. 1: Content: My_Taxo to organize the node content. This creates a separate table for each group and a caption element for each category.
The first missing requirement is the parent term.
The instructions for a simple vocab list say to add the Relationship, Term parent. This adds several new items to Fields: add list page. Unfortunately, none in this list are distinguished as parent term, which means I can't easily add a parent taxonomy term field, add a child term field, go back into Table: Settings set Groupping Field Nr.1 to be the parent and Nr.2 to child and be done with it. (9_9)

Content: All taxonomy terms
Taxonomy term: Name
Taxonomy term: Rendered Taxonomy term
Taxonomy term: Term description
Taxonomy term: Term edit link
Taxonomy term: Term ID
Taxonomy term: Weight
Taxonomy vocabulary: Description
Taxonomy vocabulary: Machine name
Taxonomy vocabulary: Name
Taxonomy vocabulary: Vocabulary ID

The second missing requirement, which may only be correctly answered by the first problem, is the captioned tables do not sort in the same order as the weights set on the taxonomy page list. To try to set this requirement I added, Sort criteria: (menu_category) Taxonomy term: Weight (asc)
Answer: Starting from scratch with a dummy content type that contains just a title and a taxonomy term.
1) Create the simple taxonomy: Category_hierarchy {i.e. 1, 1.1, 1.2, 2, 3, 3.1};
2) Create the simple content type. {title, taxonomy [Category_hierarchy];
3) Create the view:

Format: Table
   + Settings
  - Grouping field Nr.1: Category_hierarchy
Fields
  + Content: Title
  + Content: Category_hierarchy
      -- Exclude from display.
Sort criteria
  + Content: Title (asc)
Relationships
  + Content: Taxonomy terms on node [1]
      -- Vocabulary: Category_hierarchy
      -- [custom] Identifier: Category_hierarchy_child

[1] This is the only taxonomy-type Relationship available.
Results show the node titles grouped by the child relationship as expected.
4) Revisiting the View now that we see what we can simply construct, we try to get the parent term added to the display. Now, under the Relationships section is a whole new list of options titled "Taxonomy term". So we add one more to get the missing relationship, parent.

Relationships
  + Taxonomy term: Parent term
      -- Relationship: Category_hierarchy_child
      -- [custom] Identifier: Category_hierarchy_parent

If we peek into Table's settings, we would hope this might make the new relationship an available option under Grouping field Nr.1. But the grouping field options are still only Category_hierarchy and Title.
Just to be sure I added Grouping filed Nr.2 with Category_hierarchy to see if Views can do something magical and sort out the relationships automatically.
Nope. But it was worth a try. Its clear there must be some other way to populate that menu. Maybe Views has made some kind of magical field we can add from under Fields?
And that's just what has happened. We now have a new type of available field, 'Taxonomy term':
Taxonomy term: Name
 - The taxonomy term name.
Taxonomy term: Rendered Taxonomy term
 - The Taxonomy term of the current relationship rendered using a view mode.
Taxonomy term: Term description
 - The description associated with a taxonomy term.
Taxonomy term: Term edit link
 - Provide a simple link to edit the term.
Taxonomy term: Term ID
 - The tid of a taxonomy term.
Taxonomy term: Weight
 - The term weight field
Taxonomy vocabulary: Description
 -The taxonomy vocabulary description
Taxonomy vocabulary: Machine name
 - Machine-Name of the vocabulary a term is a member of. This will be the vocabulary that whichever term the "Taxonomy: Term" field is; and can similarly cause duplicates.
Taxonomy vocabulary: Name
 - Name of the vocabulary a term is a member of. This will be the vocabulary that whichever term the "Taxonomy: Term" field is; and can similarly cause duplicates.
Taxonomy vocabulary: Vocabulary ID
 - The taxonomy vocabulary ID
I've crossed out the unlikely candidates. And added each, in turn, to the Field section and looked for the field in Table's settings. Good gravy. They both worked!
I certainly don't expect tutorial answers. But I was already writing up a bug report for another problem I encountered so, my bug is your gain.


How to use KNN to classify data in MATLAB?

Question: I'm having problems in understanding how K-NN classification works in MATLAB.´
Here's the problem, I have a large dataset (65 features for over 1500 subjects) and its respective classes' label (0 or 1).
According to what's been explained to me, I have to divide the data into training, test and validation subsets to perform supervised training on the data, and classify it via K-NN.
First of all, what's the best ratio to divide the 3 subgroups (1/3 of the size of the dataset each?).
I've looked into ClassificationKNN/fitcknn functions, as well as the crossval function (idealy to divide data), but I'm really not sure how to use them.
To sum up, I wanted to
- divide data into 3 groups
- "train" the KNN (I know it's not a method that requires training, but the equivalent to training) with the training subset
- classify the test subset and get it's classification error/performance
- what's the point of having a validation test?
I hope you can help me, thank you in advance
EDIT: I think I was able to do it, but, if that's not asking too much, could you see if I missed something? This is my code, for a random case:
<code>nfeats=60;ninds=1000;
trainRatio=0.8;valRatio=.1;testRatio=.1;
kmax=100; %for instance...
data=randi(100,nfeats,ninds);
class=randi(2,1,ninds);
[trainInd,valInd,testInd] = dividerand(1000,trainRatio,valRatio,testRatio);
train=data(:,trainInd);
test=data(:,testInd);
val=data(:,valInd);
train_class=class(:,trainInd);
test_class=class(:,testInd);
val_class=class(:,valInd);
precisionmax=0;
koptimal=0;
for know=1:kmax
      %is it the same thing use knnclassify or fitcknn+predict??
      predicted_class = knnclassify(val', train', train_class',know);
      mdl = fitcknn(train',train_class','NumNeighbors',know) ;
      label = predict(mdl,val');
      consistency=sum(label==val_class')/length(val_class);
      if consistency>precisionmax
          precisionmax=consistency;
          koptimal=know;
      end
  end
mdl_final = fitcknn(train',train_class','NumNeighbors',know) ;
label_final = predict(mdl,test');
consistency_final=sum(label==test_class')/length(test_class);
</code>
Thank you very much for all your help
Comment: I don't know too much about K-NN in terms of MATLAB, but I think the first answer in this link explains why you have three subsets: http://stats.stackexchange.com/questions/9357/why-only-three-partitions-training-validation-test.
To summarize (in case you don't want to read it), testing is to initially fit your model, validation is to adjust and then pick the best fit, and testing is to get a gauge of real-world performance.
Comment: I think I was able to do it, but if someone could check my code, that would be amazing. I don't know if there's a better way of comparing the true test_label and the predicted label.
Comment: Your edit looks good, but I'm not using your matlab version, and mine is very different, so I can't reliably acknowledge. Sorry. You should use some real-world data with an obvious features-class-relation to test your program, not random data and class.
Answer: For your 1st question "what's the best ratio to divide the 3 subgroups" there are only rules of thumb:

The amount of training data is most important. The more the better.
Thus, make it as big as possible and definitely bigger than the test or validation data.
Test and validation data have a similar function, so it is convenient to assign them the same amount
of data. But it is important to have enough data to be able to recognize over-adaptation. So, they
should be picked from the data basis fully randomly.

Consequently, a 50/25/25 or 60/20/20 partitioning is quite common. But if your total amount of data is small in relation to the total number of weights of your chosen topology (e.g. 10 weights in your net and only 200 cases in the data), then 70/15/15 or even 80/10/10 might be better choices.
Concerning your 2nd question "what's the point of having a validation test?":
Typically, you train the chosen model on your training data and then estimate the "success" by applying the trained model to unseen data - the validation set.
If you now would completely stop your efforts to improve accuracy, you indeed don't need three partitions of your data. But typically, you feel that you can improve the success of your model by e.g. changing the number of weights or hidden layers or ... and now a big loops starts to run with many iterations:
1) change weights and topology, 2) train, 3) validate, not satisfied, goto 1)
The long-term effect of this loop is, that you increasingly adapt your model to the validation data, so the results get better not because you so intelligently improve your topology but because you unconsciously learn the properties of the validation set and how to cope with them.
Now, the final and only valid accuracy of your neural net is estimated on really unseen data: the test set. This is done only once and is also useful to reveal over-adaption. You are not allowed to start a second even bigger loop now to prohibit any adaption to the test set!
Comment: I like your answer, but I had a question: why do you weigh training more if your amount of data is huge in relation to the total number of weights? I thought if your number of weights is higher, then you'd want to use more training data so as to get a better fit. I just don't get why the reverse is also the case.
Comment: Thanks, Zhouster. You're absolutely right, I corrected the passage.
Comment: I get it, the purpose and how to partition and use the subgroups in theory, but how to do it with KNN?
I didn't get how to use KNN functions, even less how to manipulate its weights/topology based on validation results. Can you help me with that also?
Either way, thank you for your answer
Comment: That, my friend, is more an art than science! You need a lot of experience with training artificial neural nets to improve their accuracy. And I mean "a lot"! It is not days, it's really months. The best thing is to have somebody near to you being experienced with ANN and let you learn while he is doing this job. I fear neither stackoverflow nor any tutorial is sufficient to teach the feeling you need to do the right thing at the right time. But just start, and try, and learn! My advise: chose only one hidden layer and only few weights. Reduce your 65 features first, e.g. with PCA.
Comment: I think I was able to do it, but if someone could check my code, that would be amazing. I don't know if there's a better way of comparing the true test_label and the predicted label.
Comment: "The amount of training data is most important. The more the better.". Not true at all, that's a common mistake. This will likely lead to overfitting, that is the classifier tends to learn the structure of the training set and will not be able to generalize to previously unseen patterns. The training set should be not so big, but complete: in other words it must cover all the aspects of the problem you're studying.
Comment: @Alessiox and how do you achieve that without knowing when your training data "covers all aspects"?! The answer is: by using as much training data as possible! You ignore the very important fact, that the aspects need to naturally distributed. Small data covering all aspects overweights the rare cases, so that the final net is good at exceptions but makes huge errors on the mainstream input.


How to displaying translate tab in the node

Question: I have Multilingual and internalization Module installed on a Drupal installation. I need to have the translate link option on every node as the following screenshot:

Also I need if the content (node) in some language has no translated version in other language, when clicking to the language choose bar, it go to the default home page in the other language or Page Not found.
Answer: This tab does show when:

You have more than one language installed. Locale module needs to be enabled.
You have Content Translation or Entity Translation modules enabled (not i18n)
You have set up the content to be translatable, in the the content type settings page (if it's a node). When using Entity Translation, prior to that, you should define the entity type (e.g. node) to be translatable.
Your user has permissions to translate content.
Comment: See also http://drupal.stackexchange.com/a/50958/10086


Estou Com dificuldade para Abrir uma nova activity depois de tirar uma foto com o android studio

Question: Código
public class primeiraTela extends AppCompatActivity {

    ImageView imageView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_primeira_tela);

        Button botaocan = findViewById(R.id.botaoCanid);

        botaocan.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
                startActivityForResult(intent, 0);
            }
        });

    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        super.onActivityResult(requestCode, resultCode, data);

        Bitmap bitmap = (Bitmap) data.getExtras().get("data");
        imageView.setImageBitmap(bitmap);

        if (requestCode == 0) {
            if (resultCode == Activity.RESULT_OK) {
                Intent fotoinfo = new Intent(this, fotocomInfo.class);
                startActivity(fotoinfo);
            } else {
            }

        }
    }
}

Erro:

E/AndroidRuntime: FATAL EXCEPTION: main
                    Process: believe.com.br.believeundb, PID: 4189
                    java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=0, result=-1, data=Intent {
  act=inline-data (has extras) }} to activity
  {believe.com.br.believeundb/believe.com.br.believeundb.primeiraTela}:
  java.lang.NullPointerException: Attempt to invoke virtual method 'void
  android.widget.ImageView.setImageBitmap(android.graphics.Bitmap)' on a
  null object reference
                        at android.app.ActivityThread.deliverResults(ActivityThread.java:4094)
                        at android.app.ActivityThread.handleSendResult(ActivityThread.java:4137)
                        at android.app.ActivityThread.-wrap20(ActivityThread.java)
                        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1529)
                        at android.os.Handler.dispatchMessage(Handler.java:102)
                        at android.os.Looper.loop(Looper.java:154)
                        at android.app.ActivityThread.main(ActivityThread.java:6123)
                        at java.lang.reflect.Method.invoke(Native Method)
                        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:867)
                        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:757)
                     Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'void
  android.widget.ImageView.setImageBitmap(android.graphics.Bitmap)' on a
  null object reference
                        at believe.com.br.believeundb.primeiraTela.onActivityResult(primeiraTela.java:41)
                        at android.app.Activity.dispatchActivityResult(Activity.java:6931)
                        at android.app.ActivityThread.deliverResults(ActivityThread.java:4090)
Answer: Tente dessa forma dentro do seu onActivityResult:
        if (resultCode == RESULT_OK && requestCode == TAKE_PICTURE_RC) {
        Uri photoUri;
        if (data != null && data.getData() != null) {
            photoUri = data.getData();
        }

        String path = photoUri.getPath();

        try {
            InputStream imageStream = getContentResolver().openInputStream(photoUri);
            Bitmap bitmap = BitmapFactory.decodeStream(imageStream);
            //seu código aqui
            imageStream.close();

        } catch (IOException e) {
            e.printStackTrace();
        }


Updating modules fails with XAMPP on Windows 10

Question: I'm pretty sure I installed Drush 8 correctly on Windows 10. I ran the command window as administrator. I changed to my Drupal directory, but I get "Command pm-update needs a higher bootstrap level to run - you will need to invoke drush from a more
functional Drupal environment to run this command. The drush command 'pm-update' could not be executed.
Drush was not able to start (bootstrap) the Drupal database.
Following are all the messages & commands I entered.
ColorConsole [Version 2.4.2000 ] Microsoft Windows [Version 10.0.14393] (c)        2016 Microsoft Corporation. All rights reserved.
C:\Users\samst\Downloads>cd C:\xampp\htdocs\D8
C:\xampp\htdocs\D8>drush status Drupal version : 8.2.4
Site URI : http://default
Database driver : mysql
Database hostname : localhost
Database port :
Database username : root
Database name : d8
PHP configuration : C:\xampp\php\php.ini
PHP OS : WINNT
Drush script : C:/Users/samst/AppData/Roaming/Composer/vendor/drush/drush/drush.php
Drush version : 8.1.5
Drush temp directory : C:\Users\samst\AppData\Local\Temp
Drush configuration :
Drush alias files :
Drupal root : C:\xampp\htdocs\D8
Drupal Settings File : sites/default/settings.php
Site path : sites/default
Sync config path : sites/default/files/config_zKz1ApBZ7jZluMMX-WwnTZyMM0eVUEH6DkWwb6HxM_V0Y0izLCsKyaGVKs7EDZiAE FFWlaSJ4g/sync
C:\xampp\htdocs\D8>drush drush pm-update The drush command 'drush pm-update' could not be found. Run drush cache-clear drush to clear the [error] commandfile cache if you have installed new extensions. Drush was not able to start (bootstrap) the Drupal database. [error] Hint: This may occur when Drush is trying to: * bootstrap a site that has not been installed or does not have a configured database. In this case you can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line. See drush topic docs-aliases for details. * connect the database through a socket. The socket file may be wrong or the php-cli may have no access to it in a jailed shell. See http://drupal.org/node/1428638 for details.
Drush was attempting to connect to: Drupal version : 8.2.4
Site URI : http://default
Database driver : mysql
Database hostname : localhost
Database port :
Database username : root
Database name : d8
PHP configuration : C:\xampp\php\php.ini
PHP OS : WINNT
Drush script : C:/Users/samst/AppData/Roaming/Composer/vendor/drush/drush/drush.php
Drush version : 8.1.5
Drush temp directory : C:\Users\samst\AppData\Local\Temp
Drush configuration :
Drush alias files :
Drupal root : C:\xampp\htdocs\D8
Drupal Settings File : sites/default/settings.php
Site path : sites/default
Sync config path : sites/default/files/config_zKz1ApBZ7jZluMMX-WwnTZyMM0eVUEH6DkWwb6HxM_V0Y0izLCsKy aGVKs7EDZiAEFFWlaSJ4g/sync
C:\xampp\htdocs\D8>drush cache-clear drush 'drush' cache was cleared. [success]
C:\xampp\htdocs\D8>drush pm-update Command pm-update needs a higher bootstrap level to run - you will need to invoke drush from a more [error] functional Drupal environment to run this command. The drush command 'pm-update' could not be executed. [error] Drush was not able to start (bootstrap) the Drupal database. [error] Hint: This may occur when Drush is trying to: * bootstrap a site that has not been installed or does not have a configured database. In this case you can select another site with a working database setup by specifying the URI to use with the --uri parameter on the command line. See drush topic docs-aliases for details. * connect the database through a socket. The socket file may be wrong or the php-cli may have no access to it in a jailed shell. See http://drupal.org/node/1428638 for details.
Drush was attempting to connect to: Drupal version : 8.2.4
Site URI : http://default
Database driver : mysql
Database hostname : localhost
Database port :
Database username : root
Database name : d8
PHP configuration : C:\xampp\php\php.ini
PHP OS : WINNT
Drush script : C:/Users/samst/AppData/Roaming/Composer/vendor/drush/drush/drush.php
Drush version : 8.1.5
Drush temp directory : C:\Users\samst\AppData\Local\Temp
Drush configuration :
Drush alias files :
Drupal root : C:\xampp\htdocs\D8
Drupal Settings File : sites/default/settings.php
Site path : sites/default
Sync config path : sites/default/files/config_zKz1ApBZ7jZluMMX-WwnTZyMM0eVUEH6DkWwb6HxM_V0Y0izLCsKy aGVKs7EDZiAEFFWlaSJ4g/sync
C:\xampp\htdocs\D8>

I am running D8 with XAMPP on Windows 10. I've spent a significant amount of time searching the web for a solution. I tried changing the 'host' line in settings.php. That did not help.
Comment: Sorry, that was not the way the comment was meant. I was responding to your question whether all commands of Drush are working, if not it's not due to Acquia's Dev Desktop but to limited (development) functionality available in Windows (such as missing tools like rsync). I'm using Windows myself, so, I know from first hand how frustrating things can be and that installing these kind of applications takes a lot of effort. For me Acquia's Dev Desktop has made a lot of things (on Windows!) a lot more easier. So, I hope that this can help you too. No offence at all to Windows user, I'm one myself.
Comment: Not sure what the solution could be to your problem, but did you already try using Acquia's Dev Desktop 2 (https://dev.acquia.com/downloads). It provides a "DAMP" stack, having Drush natively in it...
Comment: I used ADD for several months until I ran into a Drush problem with it because my site is not hosted at Acquia.
Comment: I'm not using Acquia's hosting solution either, just using Dev Desktop 2 as my local development environment.
Comment: Is Drush fully functional (all commands working) on your setup? I've not tried ADD for several months. Maybe they fixed the problem with Drush ...
Comment: Every command I use regularly is working fine, if something is not working it's due to being on Windows. Running pm-update gives me: "Update information last refreshed: ma, 19/12/2016 - 14:29 No code updates available. [ok]"
Comment: I appreciate you trying to help, but a response of "if something is not working it's due to being on Windows" is not helpful. There are many of us newbies who run on Windows.
Comment: The "needs a higher bootstrap level to run" problem has been reported many times with many different solutions. Have you tried searching here for some of the solutions? Anyway, you should run the update command with the debug option so you/we get more information.
Comment: I have searched for other solutions. As of now I can again update & install modules through the UI. There was an issue with XAMPP not having a current certificate. Drush still does not work, but I will have to put it aside for now because I have higher priority stuff to work on. Too bad that Drush is so problematic on Windows 10.
Comment: same issue no ansewrs
Answer: Check if mysql binary is on your path.
Just run mysql from the same terminal window. If it works, then it will be a problem with your db config in drupal.
If it doesn't work, add the location of mysql binary C:\xampp\mysql\bin for me, to the PATH variable. It will fix it.


Multidimensional Hensel lifting

Question: I have a question about a practical application of (some) generalised form of Hensel's Lemma. I cannot find it stated in an appropriate form in Bourbaki or anywhere else, so here goes ...
Let $p$ be an odd prime: we work over the $p$-adic numbers $Q_p$ with ring of integers $Z_p$ and residue field $F_p$. I have a bunch of quadratic and quartic polynomial equations in N variables, with coefficients in $F_p$, and I have a (non-unique) solution set in $F_p^{N}$.
To simplify things assume that the number of variables N exceeds the total number of equations.
Is it possible to conclude from the non-vanishing of some sort of generalised Jacobian determinant, that my solution set lifts to $Z_p$?
Comment: Please do use LaTeX to write down your mathematics, as otherwise it appears very cumbersome. In the FAQ section you can find directions on this...
Answer: The same proof method works in higher dimension.
Let $F: \mathbb{Z}_p^m \to \mathbb{Z}_p^n$ be our system of polynomial functions.
If $x \in \mathbb{Z}_p^m$ satisfies
$$ F(x) \equiv 0 \pmod {p^e} $$
then
$$ F(x) \equiv p^e y \pmod {p^{e+1}} $$
for some vector $y$. Furthermore, the Taylor expansion about $x$ tells us
$$ F(x + p^e z) \equiv p^e y + p^e dF(x) \cdot z \pmod{p^{e+1}} $$
so as long as
$$ dF(x) \cdot z \equiv -y \pmod p$$
always has a solution for the vector $z$ (e.g. more variables than equations and $dF(x)$ has full rank), then lifts exist.
Comment: (to anybody seeing this comment:) Is it possible to get a proper reference for the statement in this answer?
Comment: @Daniel Take a look at proposition 6.4 in Silverman's "Advanced topics in the arithmetic of elliptic curves." In particular, to get the statement in the question, take $R = \Bbb{Z}_p$ and $\mathscr{X} = \operatorname{Spec}(\Bbb{Z}_p[x_1,\dots, x_m]/(F_1,\dots, F_n))$. (It is only proved in the case when $n = 1,$ however.) See also exercise 4.27 in the same book.

Multidimensional Hensel lifting