Difference between scikit-learn implementations of PCA and TruncatedSVD

Question: I understand the relation between Principal Component Analysis and Singular Value Decomposition at an algebraic/exact level. My question is about the scikit-learn implementation.
The documentation says: "[TruncatedSVD] is very similar to PCA, but operates on sample vectors directly, instead of on a covariance matrix.", which would reflect the algebraic difference between both approaches. However, it later says: "This estimator [TruncatedSVD] supports two algorithm: a fast randomized SVD solver, and a "naive" algorithm that uses ARPACK as an eigensolver on (X * X.T) or (X.T * X), whichever is more efficient.". Regarding PCA, it says: "Linear dimensionality reduction using Singular Value Decomposition of the data to project it ...". And PCA implementation supports the same two algorithms (randomized and ARPACK) solvers plus another one, LAPACK. Looking into the code I can see that both ARPACK and LAPACK in both PCA and TruncatedSVD do svd on sample data X, ARPACK being able to deal with sparse matrices (using svds).
So, aside from different attributes and methods and that PCA can additionally do exact full singular value decomposition using LAPACK, PCA and TruncatedSVD scikit-learn implementations seem to be exactly the same algorithm. First question: Is this correct?
Second question: even though LAPACK and ARPACK use scipy.linalg.svd(X) and scipy.linalg.svds(X), being X the sample matrix, they compute the singular value decomposition or eigen-decomposition of $X^T*X$ or $X*X^T$ internally. While the "randomized" solver doesn't need to compute the product. (This is relevant in connection with numerical stability, see Why PCA of data by means of SVD of the data?). Is this correct?
Relevant code: PCA line 415. TruncatedSVD line 137.
Comment: could you add a link to the code
Comment: drake - I think I agree with you on first Q. don't understand the second. what do you mean 'they compute the singular value decomposition or eigen-decomposition of XT∗XXT∗X or X∗XTX∗XT internally' - you have just  shown the code where it is all done using SVD on X? - numerical issues refer to first computing covariance matrix (call it C)  then finding eigenvectors of C
Comment: I would guess the ARPACK approach has to do with something like [Arnoldi iteration](https://en.wikipedia.org/wiki/Arnoldi_iteration), so it only has to do matrix-vector products. (In principle these sort of iterative methods do not even an explicit $X$, just a pair of routines `Xtimes()` and `Xt_times()`. This is common for large sparse matrices in PDE solvers, for example.)
Comment: @seanv507 Regarding 2nd question -- I guess that scipy.linalg.svd(X) computes the svd by doing the eigen-decomposition of $X^T * X$ or/and  $X * X^T$. Same thing for linalg.svds(X). Quoting: "a fast randomized SVD solver, and a "naive" algorithm that uses ARPACK as an eigensolver on (X * X.T) or (X.T * X)". See also last line in http://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.svds.html . The only way I can understand the first quote is that the randomized algorithm is the only one that doesn't compute the covariance/gram matrix
Comment: @GeoMatt22 Can you elaborate on your comment? Do you mean that ARPACK or LAPACK approaches don't suffer from numerical instabilities because they don't need to compute the covariance matrix?
Comment: @drake I do not know the details of these packages. I am just vaguely familiar with these iterative methods that only require matrix-vector products. In cases like [PDE](https://en.wikipedia.org/wiki/Laplacian_matrix#Example_of_the_Operator_on_a_Grid)s or [PageRank](https://en.wikipedia.org/wiki/Google_matrix) the goal is usually saving computation time, as the matrix is huge but very sparse.
Comment: Drake,  your scipy quote shows that svds(sparse) is a 'naive'  algorithm calculating XX^t.  But scipy. linalg. svd uses dgesdd which doesn't ( https://groups.google.com/forum/m/#!topic/julia-dev/mmgO65i6-fA )
Answer:
PCA and TruncatedSVD scikit-learn implementations seem to be exactly the same algorithm.

No: PCA is (truncated) SVD on centered data (by per-feature mean substraction). If the data is already centered, those two classes will do the same.
In practice TruncatedSVD is useful on large sparse datasets which cannot be centered without making the memory usage explode.

numpy.linalg.svd and scipy.linalg.svd both rely on LAPACK _GESDD described here: http://www.netlib.org/lapack/lug/node32.html (divide and conquer driver)
scipy.sparse.linalg.svds relies on ARPACK to do a eigen value decomposition of XT . X or X . X.T (depending on the shape of the data) via the Arnoldi iteration method. The HTML user guide of ARPACK  has a broken formatting which hides the computational details but the Arnoldi iteration is well described on wikipedia: https://en.wikipedia.org/wiki/Arnoldi_iteration

Here is the code for the ARPACK-based SVD in scipy:
https://github.com/scipy/scipy/blob/master/scipy/sparse/linalg/eigen/arpack/arpack.py#L1642 (search for the string for "def svds" in case of line change in the source code).
Comment: @drake I disagree that "the procedures are different (PCA uses covariance matrix and SVD uses data matrix)". PCA is a name for the type of analysis. One can use different algorithms and implementations to carry it out. EIG of cov matrix is one method, SVD of centered data matrix is another method, and then EIG and SVD can be performed by various methods as well. Does not matter - all of that is PCA.
Comment: One can support sparse data efficiently (TruncatedSVD), the other cannot (PCA). This is why we have 2 classes.
Comment: But people want PCA and they might not know that PCA is just SVD on centered data.
Comment: If that's the reason, then I'd call them SVD and SparseSVD (or similar) to avoid confusion.
Comment: @amoeba Thanks for the clarification/correction on the terminology. What you say makes more sense to me, given that SVD and EIG are algebraic theorems/methods with a wider scope than PCA
Comment: Thanks! I forgot to mention that PCA centered the data matrix, I saw that in the code, and that my question was about  a column-wise centered X. Then what I don't understand is why there are two classes with different names (PCA and TruncatedSVD) when the only difference is that PCA centers X. Wouldn't it make more sense that scaling/centering was just an option/hyperparameter within a single class? Those 2 classes don't reflect the algebraic difference between PCA and SVD -- the former does an eigenvalue decomposition of the covariance matrix and the later a sing. value decom. of X.
Comment: In my opinion it is confusing for someone who is learning PCA and SVD to have these two classes.
Comment: I see... The point is that even though the *exact* result is the same if the data matrix has been centered, the procedures are different (PCA uses covariance matrix and SVD uses data matrix), which give rise to numerical differences (PCA numerically unstable, see http://stats.stackexchange.com/a/87536/53653). This isn't reflected in scikit-learn implementations, as far as I see. I understand your point, though. Thanks.
Comment: Note that the PCA class (in scikit-learn 0.18 and later) has all the solvers of  the TruncatedSVD class in addition to the traditional LAPACK solver. By default, on large datasets a small value for n_components, the fast but approximate randomized SVD will be used as it is the only tractable solver on medium sized datasets (e.g. 10000 samples x 1000 features with n_components=30).
Comment: I saw that, as I said in the question. And in my opinion, that's a reason for unifying them in the same class. Anyway, that's just a suggestion. My doubts are now clarified.
Comment: But on sparse data, we cannot physically center the data (the resulting dense array would be too large too fit in memory or the SVD would be too slow to compute on the dense array). Therefore cannot compute the true PCA model on sparse data. Naming truncated svd on uncentered data PCA would be incorrect. This is why we decided to introduce the TruncatedSVD class: to compute a truncated SVD when computing a true PCA is not possible.
Comment: @ogrisel ( wouldn't it make more sense to refactor to use common code). An intermediate position might be to threshold the means? eg subtract mean based on eg mean/standard deviation and desired sparseness
Comment: It's already using a lot of common code. Removing the column-wise means of a sparse matrix will remove the zeros of the matrix with high probability. This is not possible.
Answer: There is also a difference in how attribute explained_variance_ is calculated.
Let the data matrix $\mathbf X$ be of $n \times p$ size, where $n$ is the number of samples and $p$ is the number of variables. And $\mathbf{X}_c$ is the centered data matrix, i.e. column means have been subtracted and are now equal to zero in this matrix. Assume that we are reducing the dimensionality of the data from $p$ to $k \lt p$.
Then for sklearn.decomposition.PCA we have the following expressions:
$$\mathbf{X}_c \approx \mathbf{U}_k \mathbf{S}_k \mathbf{V}_k^T \qquad (\text{truncated SVD of } \mathbf{X}_c);$$
$$\mathbf{L}_k = \frac{1}{n-1} \mathbf{S}^2_k \quad \Longleftrightarrow \quad \lambda_j = \frac{s_j^2}{n-1}, \quad \forall j =1,\ldots,k; \qquad(*)$$
where $\mathbf{L}_k = \mathrm{diag}(\lambda_1, \ldots, \lambda_k)$ is the matrix of $k$ largest eigenvalues of the covariance matrix $\mathbf{C} = \frac{1}{n-1} \mathbf{X}_c^T\mathbf{X}_c$, and $\mathbf{S}_k = \mathrm{diag}(s_1, \ldots, s_k)$ is the matrix of $k$ largest singular values of $\mathbf{X}_c$. You can simply prove $(*)$ if you substitute truncated SVD of $\mathbf{X}_c$ in the expression for the covariance matrix $\mathbf{C}$ and compare the result with the truncated eigendecomposition $\mathbf{C} \approx \mathbf{V}_k \mathbf{L} \mathbf{V}_k^T$ (here it was done for full decompositions). Matrix $\mathbf{L}_k$ is called explained_variance_ attribute in sklearn.decomposition.PCA.
But for sklearn.decomposition.TruncatedSVD only the following holds:
$$\mathbf{X} \approx \tilde{\mathbf{U}}_k \tilde{\mathbf{S}}_k \tilde{\mathbf{V}}_k^T \qquad (\text{truncated SVD of } \mathbf{X}).$$
In this case we can't get simple equiation like $(*)$ that will link $\mathbf{L}_k$ and $\tilde{\mathbf{S}}_k$, because substitution of truncated SVD of $\mathbf{X}$ in the expression $\mathbf{C} = \frac{1}{n-1} \mathbf{X}_c^T\mathbf{X}_c = \frac{1}{n-1}\mathbf{X}^T\mathbf{X} - \frac{n}{n-1}\bar{x}\bar{x}^T$ will not be very useful. So explained_variance_ in sklearn.decomposition.TruncatedSVD is calculated instead by np.var(X_transformed, axis=0), where X_transformed = $\mathbf{X} \tilde{\mathbf{V}}_k$ – matrix of scores (new features).


nested ifelse in R

Question: I have 2 variables promo_freq  and disc_freq . Based on the values of both these variables I want to create a 3rd variable promo_segmnt . However, the nested ifelse in R does not seen to work and the only value I am getting for my 3rd variable is "NA". As per my data, this must not be the output.
Please find the code mentioned below.
Thanks in advance.
Promotion1$promo_segmnt <-ifelse(Promotion1$disc_freq == "1.Low_disc" & Promotion1$promo_freq == "1.Low_promo","1.Low",
ifelse(Promotion1$disc_freq == "2.Med_disc" & Promotion1$promo_freq == "3.High_promo","3.High",
ifelse(Promotion1$disc_freq == "3.High_disc" & Promotion1$promo_freq == "3.High_promo","3.High ","NA")))

Comment: can you provide sample input and expected output?
Answer: If you are looking for an alternative to using if_else then you could do this is using case_when from the dplyr package:

suppressPackageStartupMessages(library(tidyverse))

disc_freq <- c("1.Low_disc", "2.Med_disc", "3.High_disc", "4.na")
promo_freq <- c("1.Low_promo", "3.High_promo", "3.High_promo", "4.na")
Promotion1 <- data.frame(disc_freq, promo_freq)

mutate(Promotion1, promo_segmnt = case_when(
  disc_freq == "1.Low_disc" & promo_freq == "1.Low_promo" ~"1.Low",
  disc_freq == "2.Med_disc" & promo_freq == "3.High_promo" ~"3.High",
  disc_freq == "3.High_disc" & promo_freq == "3.High_promo" ~"3.High",
  TRUE ~"NA"))

#>     disc_freq   promo_freq promo_segmnt
#> 1  1.Low_disc  1.Low_promo        1.Low
#> 2  2.Med_disc 3.High_promo       3.High
#> 3 3.High_disc 3.High_promo       3.High
#> 4        4.na         4.na           NA

Answer: Normal scenario works for me.
disc_freq <- c("1.Low_disc", "2.Med_disc", "3.High_disc", "4.na")
promo_freq <- c("1.Low_promo", "3.High_promo", "3.High_promo", "4.na")
Promotion1 <- data.frame(disc_freq, promo_freq)

Promotion1
disc_freq    promo_freq
<fctr>       <fctr>
1.Low_disc  1.Low_promo
2.Med_disc  3.High_promo
3.High_disc 3.High_promo
4.na    4.na

Promotion1$promo_segmnt <-
  ifelse(
  (Promotion1$disc_freq == "1.Low_disc") &
  (Promotion1$promo_freq == "1.Low_promo"),
  "1.Low",
  ifelse(
  (Promotion1$disc_freq == "2.Med_disc") &
  (Promotion1$promo_freq == "3.High_promo"),
  "3.High",
  ifelse(
  (Promotion1$disc_freq == "3.High_disc") &
  (Promotion1$promo_freq == "3.High_promo"),
  "3.High ",
  "NA"
  )
  )
  )

[1] "1.Low"   "3.High"  "3.High " "NA"

It is possible there are some extra spaces in your columns. you can remove it using stringr package.
library(stringr)

Promotion1 $disc_freq<- str_trim(Promotion1 $disc_freq, side = "both")
Promotion1 $promo_freq<- str_trim(Promotion1 $promo_freq, side = "both")

then try ifelse again.
Comment: Thanks a ton man! It worked. There was a problem with extra spaces. Removed it using the logic you gave. Now my program works fine :)

nested ifelse in R


Intent : Direct binding Activity to BroadcastReceiver without service?

Question: First of all I wanted to give codes here but the codes are way to long, it can be found here
http://www.c-sharpcorner.com/article/creating-and-scheduling-alarms-in-android/
The codes in the article doesn't create service.
As per my understanding, BroadcastReceiver is used to serve an activity based on its corresponding Service ( via an intent ).
Normally we would declare such :
Intent serviceIntent = new Intent(MainActivity.this, CustomeService.class);
   startService(serviceIntent);
   registerReceiver(mReceiver, mIntentFilter);

where mReceiver will be something like :
private BroadcastReceiver mReceiver = new BroadcastReceiver() {

      @Override
      public void onReceive(Context context, Intent intent) {

       }
}

But there's something confusing me in AlarmManager class in the example that I shared by the link :
intent = new Intent(this, MyBroadcastReceiver.class);
            pendingIntent = PendingIntent.getBroadcast(
                    this.getApplicationContext(), 280192, intent, PendingIntent.FLAG_CANCEL_CURRENT);
            alarmManager = (AlarmManager) getSystemService(ALARM_SERVICE);
            alarmManager.setRepeating(AlarmManager.RTC_WAKEUP, Calendar.getInstance().getTimeInMillis() + (i * 1000), 10000
                    , pendingIntent);

The line that is confusing me is this :
intent = new Intent(this, MyBroadcastReceiver.class);
the type of this MyBroadcastReceiver.class is BroadcastReceiver itself, and not Service.
So where's the Service that this MyBroadcastReceiver.class received broadcast from then?
Does BroadcastReceiver somehow create its own service?
Comment: hmm, what actually do you want to achieve? (i know i repeat that question)
Comment: I mean where's the service that this MyBroadcastReceiver receive broadcast from? I'm sorry if not being clear enough, this is a learning android in 24 hours for me where taking a rest is for weak. My head is tired & full of confusion right now, if it's still not clear please just tell me.
Answer:
the type of this MyBroadcastReceiver.class is BroadcastReceiver itself, and not Service

Correct. It is being used with PendingIntent.getBroadcast(), notgetService()`.

So where's the Service that this MyBroadcastReceiver.class received broadcast from then?

Few broadcasts are sent by some Service, though that is certainly possible. In this case, the broadcast is being sent from a system process, as part of sending the PendingIntent, when the alarm time comes around.

Does BroadcastReceiver somehow create its own service?

I am not sure what you mean by "create" here. A BroadcastReceiver may delegate its work to a Service. That is a common pattern with AlarmManager: have the alarms trigger a WakefulBroadcastReceiver, which in turn delegates work to an IntentService. In that case, the reason for the indirection stems from the way that AlarmManager works with WakeLocks.
Comment: Thank you very much for the explanation I hope I could upvote many times.


Hide source code in PHP GTK

Question: i have a peculiar question.
Iam developing a php GTK2 desktop application, and my purpose is to hide the source code from the client, to whom i will handle the desktop application (deploy in his system), once it is completed.
Our motto is to enable the client to run the application, without the need for a browser.
I have heard about tools like ioncube encoder that can encrypt the php source code.
I want the same functionality to be achieved with the GTK application.
Others must not be able to see the source code. How can this be achieved ?
Comment: Was your client aware of that? (receiving de-open-sourced software, which costs extra)
Comment: We are developing this project as our own, and as such, there is no client now. Once the project is completed, we go to the clients' location and describe him the features. If he is interested, then we will sell him. The cost would be very less though at the initial stage. But the encoding software costs are not kwown to the client.
Answer: If you want to go the paid for route, ioncube seems good. I've found a nice article that explains this and gives you some options - it seems like Zend has the best solution if you go by price.
http://www.zubrag.com/articles/php-source-code-encoders-and-protectors.php
I hope this helps you.
Answer: I have never heard of a good free solution for this.
There is a couple of vendors providing tools to do this. By googling I found this page that presents different solutions:
http://www.seocompany.ca/software/free-encryption-software.html
I think the Alladin usb key encryption system is one of the most robust one, but not free.
And you can also make a copy of your code and remove all comments, use only non-explicit variable names etc, so that even if your customer breaks the encryption, it would still be hard to understand the code...
Hope this helps!
P
Comment: just a note: what Piero talks about in the end is usually called "obfuscation", I believe there are tools for it.
Comment: Is it possible to develop the project as a regular php web application, and then open the site's link in the GTK window, without revealing the site's link? That might fool the client. What do you say guys ?
Comment: If it is possible, i am willing to go ahead with this approach, as it is a regular development and takes less time. Please help.

Hide source code in PHP GTK


Good machine learning algo for partial derivatives?

Question: Does anyone know of good robust algos to estimate partial derivatives of a regression model? I am talking about a general regression model like this:
$\mathbb{E}(y|x_1, x_2, ... x_n) = f(x_1, x_2, ... x_n)$, where I want estimates of $\frac{\partial f}{\partial x_k}$.
These are of great importance in medicine, economics, social sciences, etc. Now a linear model would give an approximation of these partial derivatives (its parameters) around the mean point.
This approximation can be made valid further around the mean by the inclusion of polynomial terms, but the exponential increase in numbers of parameters as the polynomial degree rises makes this infeasible with a linear model (except perhaps with Lasso?).
For better local estimates over the whole input space, natural candidates would be kernel regression and feedforward neural networks, which have smooth forms (unlike tree-based methods) and from which partial derivatives are easy to recover. But I don't know about the robustness of these estimates, as the main aim of these models is to reduce prediction error, not find the right derivatives...
I know gradient boosting (and really all regression models) give a "partial dependance", but they are not partial derivatives: they are a function $f_k(x_k)$ that gives the mean value of the predicted regression function $\widehat{f}$ for a given value of $x_k$, averaging over the realisations of the other variables $(x_1, x_2 ... x_n)$ in the training dataset.
Any help would be greatly appreciated!
Comment: Have you checked out Gaussian process regression? You can learn partial derivatives directly (e.g. see http://www.gaussianprocess.org/gpml/chapters/RW9.pdf).
Comment: Thanks, no I don't know anything about GP, I'm going to dig in! Do you know anything about the robustness of estimated derivatives though?
Comment: So if I get this right, you would get random predictions of partial derivatives, which can then be used to compute local mean and variance? This is great!
Comment: I came across this question and must say I am also very interested in this particular question. I was wondering whether you have already come across relevant research or are doing some yourself? I would be very interested in applying it to my own Bioinformatics research, especially when it has some theoretical justification making it suitable for publication (an issue with neural nets I suppose)
Comment: Hi, I haven't found any solid literature yet (but I haven't looked very hard either). So far potential candidates would seem to be: regularized linear regression (LASSO or ridge, with large basis expansions), kernel regression, neural networks (with some flavors potentially better than other, eg. I suspect extreme learning machines, dropout nets and deep learning should get better derivatives than standard backprop) and Gaussian processes.
Answer: I am not sure if this is what you are searching for, but I'll give it a try.
I think you can use Gaussian process regression and also get a nice robustness measure.
Let's say you have $m$ observations $x_1, ..., x_m\in \mathbb R^n$ and a target variable $y_i \in \mathbb R$ for each $x_i$. For a Gaussian process with covariance function $k$, the joint density of a function value $f$ at an new point $x^*$ (the idea is that $y=f+\varepsilon$ where $\varepsilon$ is Gaussian noise) is given by
$$\left(\begin{array}{c}
\mathbf{f}\\
f^{*}
\end{array}\right)=\mathcal{N}\left(\mathbf{0},\left(\begin{array}{cc}
K+\sigma^{2}I & \mathbf{k}^{*}\\
\mathbf{k}^{*} & k\left(x^{*},x^{*}\right)
\end{array}\right)\right)$$
where $\mathcal N$ denotes the Gaussian distribution, $\mathbf K$ is the matrix of pairwise covariance function values between the $x_i$ and $\mathbf k^*$ is the vector of covariance function values between $x^*$ and all $x_i$ from the training set.
From that it is easy to compute the posterior mean (i.e. the prediction at $x^*$) and covariance for $f^*$ (i.e. the measure of certainty which I think you mean with robustness). For all the formulae see the book by Rasmussen and Williams.
The interesting fact is that if $k(x_i,x_j)=\mbox{cov}(f_i,f_j)$, then $$\frac{\partial}{\partial x_{i\ell}}k(x_i,x_j)=\mbox{cov}\left(\frac{\partial}{\partial x_{i\ell}} f_i,f_j\right)$$ and $$\frac{\partial^2}{\partial x_{i\ell}\partial x_{jt}}k(x_i,x_j)=\mbox{cov}\left(\frac{\partial}{\partial x_{i\ell}} f_i,\frac{\partial}{\partial x_{jt}}f_j\right)$$ (again, check chapter 9.4 of Rasmussen and Williams). This means, you can just treat the partial derivatives as "unobserved function value $f^*$" and adapt the covariance functions in the above equation to the respective partial derivatives of the original covariance function. Then you simply predict the derivative like you would predict $f^*$. As a measure of certainty, you can look at the posterior covariance which is easily computes as well.
There is a nice  Gaussian process toolbox for matlab written by Hannes Nickisch. It can probably help you play around with GPs.
Comment: You're welcome. I hope you can use it.


Finding possible coefficients to a system of linear equations

Question: I have this system of linear equations
$$\begin{cases}
x_1+x_2+3x_3=2 \\
x_1 + 2x_2 + 4x_3 = 3 \\
x_1 + 3x_2 + \alpha x_3 = \beta \\
\end{cases}$$
And I have to find the values of $\alpha$ and $\beta$
that makes the system:

Have a single solution
Have multiple solutions
Have no solutions

I'm trying to use the matrix of the coefficients to calcule the determinant and I'm ending with $\alpha - 5$, meaning that $\alpha$ should be not 5 to  make the system have a single solution, but messing around with a calculator online I found that actually, $\alpha$ needs to be different from 6 to make the system have a single solution, regardless the $\beta$ value.
To make the system have multiple solutions, I also found that $\alpha = 6$ and $\beta = 5$, and to make it have no solutions, $\alpha = 6$ and $\beta \neq 5$.
My question is how to find those values? Because apparently the method I am trying does not work.
Answer: Hint: Your system equation can be writen by
$\begin{cases}
x_1+ x_2 + 3x_3& =2\\
\quad \quad x_2 + x_3& =1\\
\quad  (\alpha - 5) x_3 &= (\beta -4)
\end{cases}$.
Answer: A single solution is guaranteed when the LHS are linearly independent.
By subtracting the first equation from twice the second, we get
$$x_1+3x_3+5x_3=4,$$ the LHS of which coincides with the third when $\alpha=5$.
Then, if $\beta=4$, the RHS also coincide and the system is compatible.
Comment: So, the values that I got from the online calculator are wrong and the way that I was originally going was right?
Comment: Just confirming that the values from the online calculator were wrong, I used another website to test and I got exactly those values.


Verificar Números Pares

Question: Preciso verificar os números pares entre x e y, porém quando executo o código ele fica carregando a página eternamente e mostra no console somente o número x. Estou rodando o código no Google Chrome.
function pares(x, y) {
    while (x <= y) {
      if (x % 2 == 0) {
        console.log(x);
        x++;
      }
    }
  }
  pares(32, 321);

Comment: Vc só incrementa x quando ele é par, aí ele passa a ser ímpar e nunca mais é incrementado (e por isso nunca sai do while). Faça um [teste de mesa](https://pt.stackoverflow.com/a/220526/112052) no código e entenderá o problema :-)
Comment: Genial cara, muito obrigado pela ajuda :D
Answer: Como a outra resposta já disse, bastava incrementar o x fora do if. Do jeito que está, seu código só incrementa x se ele for par. Mas aí ele passa a ser ímpar e nunca mais é incrementado (pois não entra mais no if), e por isso o while nunca termina.

Mas se você quer imprimir somente os números pares, não precisa incrementar x de um em um.
Basta você verificar se o primeiro número é par (se não for, incrementa), e a partir daí, basta ir somando de 2 em 2. Assim você já pula todos os ímpares, evitando if's desnecessários dentro do while:

function pares(x, y) {
    // verifica se x é par (se não for, soma 1)
    if (x % 2 != 0) {
        x++; // x passa a ser par
    }
    // nesse ponto eu garanti que x é par
    while (x <= y) {
        console.log(x);
        x += 2; // soma 2, assim já vai para o próximo número par
    }
}

pares(32, 321);

Obs: o snippet acima não está mostrando todas as linhas (no meu browser, está mostrando apenas do 222 ao 320). Mas você pode abrir o console do seu browser para ver toda a saída (pressione F12 e escolha a aba "Console").
Answer: Faça o incremento da variável "x" fora do seu IF. Altere o código da seguinte forma:

function pares(x, y) {
    while (x <= y) {
      if (x % 2 == 0) {
        console.log(x);

      }
   x++;
    }
  }
  pares(32, 321);

Verificar Números Pares


How to check the stability of a neural network?

Question: Is there any way or theory to prove that a neural network is stable? For instance, when I use an NN to learn from a dataset, how could I prove that the NN's learning will converge and give the adequate result? Thanks in advance for your help!
Answer: When used in control theory, we use Lyapunov stability to show that the neural network's weights are bounded. We can do this by using robust weight updates. So yes, in a control system you can in fact show that neural networks are stable in the sense of Lyapunov. Usually you do this by assuming there are "ideal" weights the neural network can take on, that are bounded, although these are not necessarily known. You also assume that when these ideal weights are being used, there is a bounded difference between the actual function, and the approximation by the neural network.
This field has been around for years. I suggest you read any neural network paper by Frank Lewis. Another resource would be the book "Neural Network Control of Robot Manipulators and Nonlinear Systems" where Lewis is one of the authors. It details how to use neural networks to approximate nonlinear functions that appear in control systems, and everything is based off Lyapunov stability techniques.
Comment: Incorrect answer; OP is asking for convergence of the training process. Lyapunov stability characterizes the dynamic behavior of the trained network.
Answer: There's no way of knowing that. Most likely the cost function will have many different local minima, to ensuring convergence is kind of difficult, unless of course you are close to one of them, in which case is just a matter of using a small learning rate.
The typical argument authors use to show this is taking the neurons in the hidden layers and permuting them. This will not change the cost function, but it will change the location in parameter space. This implies that there are several local minima in your optimization problem.
Comment: I see, so that's why i found out the NN sometimes gives wrong results. Furthermore, is there any theory about deciding the exact number of layers and neurons for the network?
Comment: @Student That comes from understanding the problem you are trying to solve. The most obvious thing to take into account: the more neurons you add, the more training examples you'll need
Comment: Thank you very much for your advice!
Comment: Yes, you can show a neural network is stable in control applications. This has been well studied. See my answer.


SARSA update rule

Question: The update rule for the SARSA algorithm, which is mentioned here is the following.
$Q(s_t,a_t) \leftarrow Q(s_t,a_t) + \alpha [r_{t+1} + \gamma Q(s_{t+1}, a_{t+1})-Q(s_t,a_t)]$
My question is, why can't the value functions be used in the RHS of the update rather than choosing a sample action? In other words, why can't the update be as follows?
$Q(s_t,a_t) \leftarrow Q(s_t,a_t) + \alpha [r_{t+1} + \gamma V(s_{t+1})-Q(s_t,a_t)]$
Will this update rule work? If it does, what advantage does the SARSA update have over this update rule?
Answer: I agree with what Sean said. I'll add this little bit to answer this question you asked more concretely:

Will this update rule work? If it does, what advantage does the SARSA update have over this update rule?

The SARSA update rule can converge to different values than the Q-learning rule (which is, like Sean said, essentially what you suggested). This is due to the difference between on-policy and off-policy that he also described. An on-policy algorithm (like the SARSA update rule) converges to the optimal values for the policy that your agent is also using to gather experience. Off-policy algorithms converge to values for a policy that is different from the policy being followed by the agent to gather experience.
The behaviour policy (the one that the agent uses to gather experience) is typically going to be something like $\epsilon$-greedy, where with some nonzero probability you select suboptimal (random) actions. An on-policy algorithm like SARSA takes this into account, it converges to values that are still correct given the knowledge that your agent is sometimes going to be "stupid" and do something random. Q-learning (off-policy learning with a pure greedy policy as "target policy", the policy you're computing values for) is going to converge to values that are actually only going to be correct later on, when your agent switches over to a completely greedy policy.
This distinction can, for example, be important in situations where you care about learning "safe" behaviour during the learning process, where you don't just care about learning optimal behaviour to run after the learning process. Suppose, for example, that you have a robot who starts near a cliff, and needs to walk to another point along the same cliff.

A Q-learning algorithm is going to converge to values that tell the agent that it's optimal to walk right along the cliff, because that's the shortest path.
SARSA is going to converge to different values, which will tell the agent that it's optimal to first walk a safe distance away from the cliff, and then move towards the goal, only getting close to the cliff again when also being close to the destination. It will converge to these values because it knows of itself that it will sometimes be stupid and take a random action. If it happens to be close to the cliff when it takes a random action, it will fall down and you have to buy a new robot which can be very expensive. So, it will learn (because it's learning values for the $\epsilon$-greedy policy) that the optimal behaviour is to first walk away from the cliff, so that it won't fall even if it sometimes takes a random action.
Answer: It one sense, it can: Your second equation is the update rule for Q-learning. (See the definitions in Ch. 6 of Sutton and Barto's draft edition of Reinforcement Learning, along with the relation $V(S) = \max_a Q(S,a)$. I don't have a written source for this at hand, but it's covered in early sections of this course.)
The major difference is that SARSA is on-policy: It learns the $Q$ values of the policy that it's following. Off-policy learners, Q-learning included, improve a policy different from that used to generate the data.
Here's a summary the differing advantages between on- and off-policy, again from Sutton and Barto:

On-policy methods are generally simpler and are considered first. Off-policy methods require additional concepts and notation, and because the data is due to a different policy, off-policy methods are often of greater variance and are slower to converge. On the other hand, off-policy methods are more powerful and general. They include on-policy methods as the special case in which the target and behavior policies are the same. Off-policy methods also have a variety of additional uses in applications. For example, they can often be applied to learn from data generated by a conventional non-learning controller, or from a human expert.

Ch. 6 of the draft, very much worth a read, also includes a comparison of SARSA and Q-learning from several perspectives, to include performance on an example and conditions for convergence to the optimal policy.
Comment: $v_{\pi}(s)$ and $v_{*}(s)$ are both options for value function as OP has written it. The former, $v_{\pi}(s)$  is to me a more natural reading of OP's suggested formula, since it would be odd to have $q_{\pi}(s, a)$ evaluation of current policy from SARSA being estimated alongside a maximising state value function. In that case, the suggested algorithm could be *Expected SARSA*, which is also off-policy, but has some interesting pros and cons compared to SARSA or Q-Learning.

SARSA update rule


About single-valued function

Question: I was reading a paper about compression algorithm:

In order to optimality fit the line segments to the curve, Bellman's
  algorithm assumes that the input data is a valid (i.e., single-valued)
  function; thus, the trajectory cannot contain no loops.

What does it means with valid function? I suppose he intends to have a single-valued function, but: what's a single-valued function? According to wikipedia definition:

A single-valued function is an emphatic term for a mathematical
  function in the usual sense. That is, each element of the function's
  domain maps to a single, well-defined element of its range.

But this sounds to me like an injective function. Can you confirm that both (single-valued function and injective function) mean the same thing?
Answer: Of course not! A map $f \colon X \to Y$ is injective when $f(x_1)=f(x_2)$ implies $x_1=x_2$. But, strictly speaking, you have to know what a function is, i.e. what a single-valued function is.
Comment: Probably "funzione univoca" or "funzione ad un solo valore". But I believe that multi-valued functions need a name, since functions already have one :-)
Comment: thanks. So what's the best way to translate single-valued in italian language?
Answer: They are not the same thing. In an injective function each element of the co-domain is mapped to by at most 1 element of the domain (so two distinct elements of the domain cannot be mapped to the same element of the co-domain). In a single-valued mapping (or a function) it is the other way round: each element of the domain is mapped to at most one element of the co-domain.
For example, the square root mapping (on positive real numbers) is not single-valued as the square root of 4 (for example) is not uniquely determined. The square root mapping maps 4 to both 2 and -2 and so is not single-valued. However, if we restrict the square root mapping and demand that the output be positive we get a function (an injective function as it turns out)

About single-valued function


subset of all real valued functions of a real variable is subring but not an ideal

Question: $STATEMENT:$
Let $R$ be the ring of all real-valued functions of a real variable.
The subset $S$ of all differentiable functions is a subring of R but not an ideal of R.
$Question:$
I think subset $S$ is clearly subring of $R$. ($f$,$g$ $\in$ $S$ $\Rightarrow$ $f$ - $g$ $\in$ $S$, $fg$ $\in$ $S$)
But I cannot imagine $S$ is not an ideal of $R$.
Can you give some example?
Thanks for all answer.
Comment: Do you think $f$ differentiable implies $fg$ differentiable for all $g$? For example, what if $f\equiv 1$ and $g$ isn't differentiable?
Comment: Is it right $f,g$ differentiable implies $fg$ differentiable? If just $f$ differentiable, I think you are right.
Comment: Yes, from the basic definition of the derivative as a limit of a difference quotient (show this!).
Comment: If $\lim_{h\to 0} \frac{f(x+h)-f(x)}{h} = f'(x)$ and $\lim_{h\to 0} \frac{g(x+h)-g(x)}{h} = g'(x)$, then
$\lim_{h\to 0} \frac{f(x+h)g(x+h)-f(x)g(x)}{h} = \lim_{h\to 0} \frac{f(x+h)g(x+h)-f(x)g(x)+g(x+h)f(x)-g(x+h)f(x)}{h} = \lim_{h\to 0} \frac{g(x+h)[f(x+h)-f(x)]}{h} + \lim_{h\to 0} \frac{f(x+h)[g(x+h)-g(x)]}{h} = g(x)f'(x) + g'(x)f(x)$
Answer: It's not an ideal because it's not true that $rs \in S$ for every $r \in R$ and $s \in S$. In other words, if $s$ is a differentiable function, and $r$ is any general real-valued function, then $rs$ is not always differentiable. In general, it may not even be continuous!
For example, observe that $s = e^{x}$ is differentiable. Hence $s \in S$. However, let
$$
r =
\begin{cases}
1 & \text{if } x \in \mathbb{Q}\\
-1 & \text{if } x \not\in \mathbb{Q}.
\end{cases}
$$
Clearly $r \in R$ because it is a real-valued function. But note that
$$
rs =
\begin{cases}
e^x & \text{if } x \in \mathbb{Q}\\
-e^x & \text{if } x \not\in \mathbb{Q}.
\end{cases}
$$
is not differentiable. In fact it's not even continuous. So $rs \not\in S$. Hence it is not true that $rs \in S$ for every $r \in R$, $s \in S$, so we see that $S$ is not an ideal of $R$.
However, it's clearly a subring since it's nonempty and the product and difference of differentiable functions is differentiable.
Comment: thanks for answer.


Canvas image rotation performance

Question: Image to help visualize code

Question
Is there some way to make the code below even more performant? I feel like I'm missing something when it comes to only rendering the part of the image that's visible rather than the entire image.
Code

// ------------
// Setup canvas
// ------------

let rotation = 0;
const ratio = window.devicePixelRatio || 1;
const canvas = document.createElement('canvas');
document.body.appendChild(canvas);
canvas.width = canvas.clientWidth * ratio;
canvas.height = canvas.clientHeight * ratio;
const context = canvas.getContext('2d');
context.scale(ratio, ratio);

// ----------
// Load image
// ----------

let imageReady = false;
const image = new Image();
      image.onload = () => {
        imageReady = true;
      }
      image.src = 'https://i.imgur.com/ltOYsck.png';

// --------------
// Animation Loop
// --------------

window.requestAnimationFrame(loop)

function loop() {
  context.clearRect(0,0,canvas.width, canvas.height);

  if (imageReady === true) {
    context.save();
    context.translate(canvas.clientWidth/2, canvas.clientHeight/2 + 1000)
    context.rotate(rotation * Math.PI / 180);
    context.drawImage(
      image,
      0,
      0,
      image.width,
      image.height,
      -image.width/4,
      -image.height/4,
      image.width * 0.5,
      image.height * 0.5
    );
    context.restore();
  }

  rotation += 0.1;
  if(rotation >= 360) rotation = 0;

  window.requestAnimationFrame(loop)
}
canvas {
  width: 1200px;
  height: 200px;
  background: lightgreen;
}
Comment: What is it bout this 5 stacked canvases? I can't read anything about it in the question. This is a very bad idea indeed. Just do the stacking on a single canvas.
Comment: You could use one canvas instead of multiple canvas on top of each other. You should call requestAnimationFrame at the start of the loop so that you're slightly less likely to miss a frame. You could probably use CSS animations for this. They are generally faster than JavaScript.
Comment: One minor - consider postponing your render loop until the image data has loaded, to avoid the `if (imageReady === true)` check every frame
Comment: Changing the size of a canvas is slightly faster than clearRect so you could increment and decrement the height (or assign the height to itself but I'm not sure if that will work).
Comment: No problems !
Comment: You could avoid calling save and restore by calling resetTransform instead. Everything I've mentioned so far are really just micro optimisations. I think the only way to really make this faster would be to use WebGL. If you've never used WebGL before then it might seem like overkill if all you want are some spinning images.
Comment: I've used OpenGL (not WebGL) before so I might be able to help you out if you decide to go down that path.
Comment: @Kerndog73 It looks like resetTransform isn't supported in Edge so I'll have to avoid it for the PC folks out there. I will spend some time tonight looking into WebGL to see if that will make things a little more performant. Thank you for taking the time to help.
Comment: @Kerndog73 what are you saying? Wherever you call rAF it will get scheduled for the next frame, the position in the callback you call has absolutely no incidence whatsoever on whatever: no other external callbacks can be added in between since the code in cb gets executed synchronously. And now even worse, **changing the size of canvas is the slowest way to clear its context**. It will reset the whole context to defaults, reassign a new bitmap in memory and other slow stuff like that.
Comment: What performance issues are you actually having?
Comment: Is this your real image? Do you really need to draw it as an image? Drawing it as a path might improve perfs. Otherwise yes, a 4400×4400 image is slow to render, most graphic cards won't have enough memory to store it in their own cache.
Comment: @Kaiido I'm not quite sure what I was thinking when I made my comment about RAF. Maybe I thought it was asynchronous or something. I saw a benchmark for various methods for clearing the canvas and changing the size was fastest. Although, that was a couple of years ago so maybe things have changed. Now that you mention all of the things that happen when you resize, clearRect should be faster. That benchmark made no sense!
Comment: @Kaiido Found the old benchmark. https://jsperf.com/canvas-clear-speed/44 Assigning the width to itself is much faster than clearRect on my system.
Comment: @Kerndog73 didn't ran it yet but that benchmark proves nothing... There is nothing drawn on your context nor any properties of the context that has been modified. So yes resetting a clear canvas might be no-op, but, that's never what you do IRL.
Comment: @Kerndog73 Thank you for the tips. I tried keeping it straight CSS at first, but due to the size of the images and the style recalculations every frame it caused the rest of the UI to take a hit on performance (would see very slight lag every once in a while). I will look into using height to clear the canvas :)
Comment: @DacreDenny ah right! thanks for pointing that out :)
Comment: @MattWaywhen stacking 5 canvases on top of each other with rotating images (4400 x 4400) spikes the Chrome CPU usage from 9% (4 canvases) to 70% (5 canvases). I assume this is due to the large images and lack of memory (I will attempt to cut them down). But other than that I was wondering if there is anything in the code above that could be done to optimize the way the image is being drawn/rotated with performance in mind.
Comment: @Kaiido It's a simplified version of the real image - the real one has a lot more details and is not just a path. I will spend some time tonight and see just how far I could cut the image down while still maintaining good quality. Are multiple canvases the way to go when trying to do the above 5x? And do you see anything wrong with the drawing/rotating code that could be improved? I noticed here they mention not to scale the image within the drawImage method - https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Optimizing_canvas#Don%E2%80%99t_scale_images_in_drawImage
Answer: This "Answer" doesn't answer your question regarding your code, as I think it is the wrong approach in the first place ...

I'm trying to achieve an effect with 4-5 canvases over each other all spinning at different speeds
  However after 5 layers the CPU usage in Chrome goes through the roof (from 9% to 70%).

... but it includes an example that rotates 50 of your layers without problem. ~10%CPU and ~15%GPU; on a Laptop

The reason for the large images is because I want it to look crisp on Retina devices, so everything is twice as big as it needs to be (4400 x 4400)

Especially when you want a crisp image you avoid pixel graphics. The Image you use is perfect to convert it into a Vector.
4400x4400 Pixels means 4400x4400x4 Byte (~73MB) of raw image data that JS has to juggle around, every time you render (a part of) that image.

/*
  // set random rotation speeds for each layer; and a bit more.
  // code is commented out, because at the moment this is defined in the CSS
  let i=1;
  for(let elm of document.querySelectorAll('#example>g')){
    // give this particular elm an initial rotation and add a little scale to it
    const sx = (Math.random() * .2 + .8).toFixed(4);
    const sy = (Math.random() * .2 + .8).toFixed(4);
    const rotate = Math.floor(Math.random() * 360);

    const transform = `scale(${sx},${sy})rotate(${rotate}deg)`;
    const duration = Math.floor(Math.random() * 25000 + 5000) + "ms";

    elm.style.setProperty("transform", transform);
    elm.style.setProperty("animation-duration", duration);

    console.log(`#example>g:nth-child(${i++}){animation-duration:${duration};transform:${transform}}`);
  }
*/
@keyframes rotate {
  from {
    transform: rotate(0deg);
  }
  to {
    transform: rotate(360deg);
  }
}

#example>g>use {
  animation-name: rotate;
  animation-duration: inherit;
  animation-timing-function: linear;
  animation-iteration-count: infinite;
  fill: rgba(64, 64, 64, .25);
}

/* the following CSS has been generated by the JS */
#example>g:nth-child(1){animation-duration:6235ms;transform:scale(0.9831,0.9672)rotate(317deg)}
#example>g:nth-child(2){animation-duration:27794ms;transform:scale(0.9849,0.8447)rotate(290deg)}
#example>g:nth-child(3){animation-duration:9000ms;transform:scale(0.8721,0.8287)rotate(149deg)}
#example>g:nth-child(4){animation-duration:13234ms;transform:scale(0.8428,0.8676)rotate(198deg)}
#example>g:nth-child(5){animation-duration:26854ms;transform:scale(0.9927,0.9902)rotate(136deg)}
#example>g:nth-child(6){animation-duration:20110ms;transform:scale(0.9047,0.8027)rotate(158deg)}
#example>g:nth-child(7){animation-duration:23603ms;transform:scale(0.9949,0.8478)rotate(32deg)}
#example>g:nth-child(8){animation-duration:27045ms;transform:scale(0.8898,0.8352)rotate(166deg)}
#example>g:nth-child(9){animation-duration:24821ms;transform:scale(0.8344,0.9232)rotate(127deg)}
#example>g:nth-child(10){animation-duration:11986ms;transform:scale(0.8612,0.8307)rotate(232deg)}
#example>g:nth-child(11){animation-duration:7464ms;transform:scale(0.8422,0.9567)rotate(175deg)}
#example>g:nth-child(12){animation-duration:6435ms;transform:scale(0.9263,0.8725)rotate(330deg)}
#example>g:nth-child(13){animation-duration:29691ms;transform:scale(0.9359,0.9489)rotate(260deg)}
#example>g:nth-child(14){animation-duration:12935ms;transform:scale(0.8948,0.9912)rotate(107deg)}
#example>g:nth-child(15){animation-duration:21544ms;transform:scale(0.8496,0.9085)rotate(120deg)}
#example>g:nth-child(16){animation-duration:16241ms;transform:scale(0.9520,0.8554)rotate(111deg)}
#example>g:nth-child(17){animation-duration:23578ms;transform:scale(0.8677,0.9682)rotate(234deg)}
#example>g:nth-child(18){animation-duration:15650ms;transform:scale(0.8582,0.8700)rotate(54deg)}
#example>g:nth-child(19){animation-duration:11847ms;transform:scale(0.9447,0.9884)rotate(139deg)}
#example>g:nth-child(20){animation-duration:9324ms;transform:scale(0.9153,0.9116)rotate(316deg)}
#example>g:nth-child(21){animation-duration:17702ms;transform:scale(0.9779,0.8993)rotate(37deg)}
#example>g:nth-child(22){animation-duration:13465ms;transform:scale(0.9961,0.8083)rotate(194deg)}
#example>g:nth-child(23){animation-duration:15226ms;transform:scale(0.9492,0.8862)rotate(183deg)}
#example>g:nth-child(24){animation-duration:12913ms;transform:scale(0.8593,0.9157)rotate(28deg)}
#example>g:nth-child(25){animation-duration:23357ms;transform:scale(0.9777,0.8756)rotate(118deg)}
#example>g:nth-child(26){animation-duration:6081ms;transform:scale(0.8035,0.8200)rotate(342deg)}
#example>g:nth-child(27){animation-duration:15580ms;transform:scale(0.8807,0.8166)rotate(242deg)}
#example>g:nth-child(28){animation-duration:18051ms;transform:scale(0.9160,0.8838)rotate(102deg)}
#example>g:nth-child(29){animation-duration:6058ms;transform:scale(0.8111,0.9906)rotate(3deg)}
#example>g:nth-child(30){animation-duration:8070ms;transform:scale(0.9133,0.8374)rotate(60deg)}
#example>g:nth-child(31){animation-duration:16621ms;transform:scale(0.8352,0.9580)rotate(339deg)}
#example>g:nth-child(32){animation-duration:27493ms;transform:scale(0.9611,0.8951)rotate(146deg)}
#example>g:nth-child(33){animation-duration:18659ms;transform:scale(0.8012,0.9420)rotate(234deg)}
#example>g:nth-child(34){animation-duration:26061ms;transform:scale(0.9279,0.8988)rotate(16deg)}
#example>g:nth-child(35){animation-duration:29827ms;transform:scale(0.9939,0.8191)rotate(289deg)}
#example>g:nth-child(36){animation-duration:20111ms;transform:scale(0.9248,0.8385)rotate(107deg)}
#example>g:nth-child(37){animation-duration:17507ms;transform:scale(0.8673,0.9479)rotate(330deg)}
#example>g:nth-child(38){animation-duration:12485ms;transform:scale(0.9069,0.9609)rotate(163deg)}
#example>g:nth-child(39){animation-duration:13963ms;transform:scale(0.8223,0.8398)rotate(195deg)}
#example>g:nth-child(40){animation-duration:9466ms;transform:scale(0.8756,0.9944)rotate(81deg)}
#example>g:nth-child(41){animation-duration:5128ms;transform:scale(0.8159,0.9781)rotate(118deg)}
#example>g:nth-child(42){animation-duration:19802ms;transform:scale(0.8892,0.9996)rotate(55deg)}
#example>g:nth-child(43){animation-duration:15241ms;transform:scale(0.8090,0.9250)rotate(169deg)}
#example>g:nth-child(44){animation-duration:17820ms;transform:scale(0.8080,0.8366)rotate(301deg)}
#example>g:nth-child(45){animation-duration:12900ms;transform:scale(0.8853,0.9383)rotate(298deg)}
#example>g:nth-child(46){animation-duration:27931ms;transform:scale(0.8182,0.9923)rotate(155deg)}
#example>g:nth-child(47){animation-duration:5307ms;transform:scale(0.9641,0.9780)rotate(299deg)}
#example>g:nth-child(48){animation-duration:12744ms;transform:scale(0.8974,0.9224)rotate(121deg)}
#example>g:nth-child(49){animation-duration:28452ms;transform:scale(0.8455,0.8270)rotate(125deg)}
#example>g:nth-child(50){animation-duration:7462ms;transform:scale(0.9797,0.9203)rotate(351deg)}
<svg id="example" version="1" xmlns="http://www.w3.org/2000/svg" viewBox="-2200 -2200 4400 1000">
  <defs><path id="asteroid" d="M-101 2154c-47-5-101-21-161-48-37-16-80-38-115-59l-19-11-19-5c-34-9-119-34-161-46-299-90-509-170-611-235-32-21-49-36-65-59-29-43-69-89-138-158-65-65-125-121-264-245-115-104-126-114-135-128-17-26-36-76-56-143-16-56-28-106-58-234-37-159-51-214-69-266-19-58-35-86-52-94-8-4-15-10-22-21-43-64-86-256-98-436-7-99-3-183 11-239 8-33 15-48 35-74 79-104 183-309 255-502 23-60 36-100 54-166 6-23 14-41 27-67 91-184 347-474 570-642 112-85 208-135 282-146 132-21 195-36 408-96 192-54 296-81 432-110 90-20 116-24 172-29 28-2.5 116-2.5 148 0 116 9 228 32 352 72 118 38 254 96 353 151l16 9 24 23c204 191 424 408 580 571 144 150 252 273 311 351 40 52 77 109 108 164 15 26 46 89 58 115 47 106 79 215 89 308 2.3 20 2 24-3.5 53-33 172-57 334-63 433-3 44-2.5 64 1.4 129 9 147 9 223 2 304-13 141-52 252-117 333-18 22-22 27-41 39-47 31-88 66-133 111-40 41-69 75-107 127-47 64-87 130-148 243-43 78-52 93-73 120-47 58-129 139-211 209-132 111-255 188-347 217-31 10-50 13-76 13-17 0-21-.3-30-2.3-35-8-52-10-94-10-37 0-54 1.1-89 6-89 12-189 40-292 81-47 18-147 49-219 67-53 13-92 21-134 27-20 3-56 3.5-73 1.5z"/></defs>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
  <g><use xlink:href="#asteroid" /></g>
</svg>
Comment: You said the image couldn't be rendered as a path? If it can, then canvas will offer you about the same perfs as svg (https://jsfiddle.net/ecybd0mw/) Even if you do write SVG as vectors, it gets rasterized just the same when painted to screen.
Comment: @Kaiido nice, didn't know about Path2D. And last time I've worked with the canvas it didn't use the GPU *(there was something about that)*. Seems I have to get an update; thx.
Comment: Thank you for taking the time to answer my question Thomas. And wow it looks like you're right - looks like vector is the way to go with this one.

Canvas image rotation performance


Proving $\binom{p-1}k = (-1)^k \pmod p$ using fermat's little theorem.

Question: Proving $\binom{p-1}k = (-1)^k \pmod p$ using Fermat's little theorem.
Im not sure how to approach this. Though I think we might use an arbitrary number $x$ and create a polynomial with an exponent $p-1$, but I am not sure how.
Comment: Is $p$ a prime?
Answer: We know that: $\binom{p}k = \binom{p-1}{k-1} + \binom{p-1}{k}$. Now, if $p$ is a prime then $p | \binom{p}k$. Reduce the first equation modulo $p$ gives:
$$\begin{align}
\binom{p-1}{k} &= -1 \binom{p-1}{k-1}\pmod p &\implies \\
\binom{p-1}{k} &= (-1)^k \binom{p-1}{0}\pmod p &\implies \\
\binom{p-1}{k} &= (-1)^k \pmod p
\end{align}$$
Comment: using fermats...
Comment: I just offered an alternative approach (which I also think it's quite nice). No need to accept it.
Comment: How did you write $\binom{p-1}{k-1}$ as $\binom{p-1}{0}$?
Comment: @CoolMath200 Actually, I substituted it with $(-1)^{k-1} {{p-1} \choose{0}}$. If it helps, set $f(k) =  {{p-1}\choose{k}} \mod p$. Then the first equation is $f(k) = -f(k-1)$.


How to solve $\frac{99.5−0.8n}{\sqrt{0.16n}}=-1.645$?

Question: $\frac{99.5−0.8n}{\sqrt{0.16n}}=-1.645$
Simplifying and I get
$99.5−0.8n+1.645\sqrt{0.16} n^{1/2}=0$
$0.8n-1.645\sqrt{0.16} n^{1/2}-99.5=0$
Can't use quadratic formula.
Comment: `Can't use quadratic formula` Why? It *is* a quadratic in $\sqrt{n}$.
Comment: Let $n=a^2$ for some real or complex $a$ And then use quadratic formula
Answer: The idea you had was on the right track, you just need to continue by squaring both sides.
$$\frac{99.5-0.8n}{\sqrt {0.16n}}=-1.645$$
$$99.5-0.8n=-1.645 \sqrt {0.16n}$$
$$(99.5-0.8n)^2 = (-1.645 \sqrt {0.16n})^2$$
$$9900.25-159.2n+0.64n^2=.432964n$$
$$9900.25-159.632964n+0.64n^2=0$$
From this, you can use any methods you know, such as the quadratic formula.
Comment: Squaring both sides is not necessary at all, and might introduce fictitious solutions.
Answer: Instead of the absurdly specific
$\dfrac{99.5−0.8n}{\sqrt{0.16n}}=-1.645
$,
consider
$\frac{a−bn}{\sqrt{cn}}=-d
$
with
$a, b, c, d > 0$.
Then
$a-bn
=-d\sqrt{n}
$.
Squaring,
and remembering that
we have to check the roots
because of this,
$a^2-2abn+b^2n^2
=d^2n
$
or
$b^2n^2-(2ab+d^2)n+a^2 = 0$.
Applying the
standard quadratic formula,
$\begin{array}\\
n
&=\dfrac{(2ab+d^2)\pm \sqrt{(2ab+d^2)^2-4a^2b^2}}{2b^2}\\
&=\dfrac{(2ab+d^2)\pm \sqrt{4a^2b^2+4abd^2+d^4-4a^2b^2}}{2b^2}\\
&=\dfrac{(2ab+d^2)\pm \sqrt{4abd^2+d^4}}{2b^2}\\
&=\dfrac{(2ab+d^2)\pm d\sqrt{4ab+d^2}}{2b^2}\\
\end{array}
$
Since
$4ab+d^2 > 0$,
the roots are real;
since
$(2ab+d^2)^2
\gt 4abd^2+d^4
$,
both roots are positive.
Now substitute your parameters.
Comment: Your $c$ disappeared.


CentOS 6 | bash_profile giving errors?

Question: I have a .bash_profile, and its giving me an error "Unexpected end of file" I have no clue whats wrong, here it is,
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

# User specific environment and startup programs
export PS1="\u@\h\\$"
PATH=$PATH:$HOME/bin

export PATH

It shows syntax error at line 11. But there is no line 11?...
Also, its not setting my ps1, but I'm supposing thats because the unexpected end of file.
Comment: Whilst bash is generally considered topical here, using dreamweaver on windows and uploading to unix isn't something a sysadmin would do. You'll probably find the answer to your dilema [here](http://serverfault.com/search?q=dos2unix).
Comment: Google: Windows vs. Linux newline.
Comment: Do you write it on the Windows and upload to the Linux?
Comment: @quanta Yes. Using dreamweaver
Comment: I just tried seeing if there was an extra line using the vi command, but it showed only 10 lines. Very weird.
Comment: @quanta Thanks, but I don't understand, even in vi editor it shows only 10 lines?
Comment: Google: How to show special characters in `vi`.
Answer: So, I believe there are special characters in your file.
You can check the same by using vim, instead of vi
Or if you open the file in vi, you can use this command to see the special characters.
:set list

After fixing it, you can use this command to get back to normal viewing mode
:set nolist

Once you remove the special characters, it should work fine.
Also, if you have emacs installed, that can also help you in the same, though I love vim and would always like to stick with that :)

CentOS 6 | bash_profile giving errors?


How to display a form in controller method

Question: I have a form defined in mymodule/src/Form/myForm.php
namespace Drupal\mymodule\Form;

use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;

class myForm extends FormBase {

/**
 * Specify the form ID.
 */
public function getFormId() {
  return 'mymodule_myform';
}

  /**
   * Build the actual form.
   */
  public function buildForm(array $form, FormStateInterface $form_state) {

    // Form API code here...

Originally, this routing worked fine and the form displayed at the correct path:
mymodule.single_tab:
  path: '/node/{node}/edit/performance'
  defaults:
    _form: '\Drupal\mymodule\Form\myForm'
    _title: 'Add performance'

Then I needed to add a custom access check, restricting to a specific content type:
mymodule.single_tab:
  path: '/node/{node}/edit/performance'
  defaults:
    _title: 'Add performance'
    _controller: '\Drupal\mymodule\Controller\mymoduleController::content'
  requirements:
    _custom_access: '\Drupal\mymodule\Controller\mymoduleController::checkAccess'

The access check works fine. Now I think I have to display my form in the controller:
namespace Drupal\mymodule\Controller;

use Drupal\Core\Controller\ControllerBase;
use Drupal\Core\Form\FormBuilderInterface;
use Drupal\Core\Access\AccessResult;
use Drupal\node\Entity\Node;

/**
 * Controller for Add Performance single tab.
 */
class mymoduleController extends ControllerBase {

  /**
   * Returns the add single performance form.
   */
  public function content() {
    $form = $this->formBuilder->getForm('Drupal\mymodule\Form\myForm');
        return [
          '#type' => 'markup',
          '#markup' => $form
        ];
  }

This fails to display the form, and I get:
Call to a member function getForm() on null

I tried defining my form builder class inside the controller instead, but same result. What am I doing wrong?
Comment: @pushka, these are three different issues. 1) you can just `return $form;` 2) the error message is because you didn't inject the form builder service or didn't use the static `\Drupal::formBuilder()` 3) `_form` invokes a controller too, a special form controller.
Comment: Can't you just return the form? I don't think you need to have a controller.
Comment: I think I need a controller for the [custom access check](https://drupal.stackexchange.com/questions/201320/add-a-local-task-tab-for-a-content-type-only)?
Answer: Thanks to Kevin's comment, I realised that I can just return the form AND still use the custom access check reference to a controller method:
mymodule.single_tab:
  path: '/node/{node}/edit/performance'
  defaults:
    _form: '\Drupal\add_performance\Form\myForm'
    _title: 'Add performance'
  requirements:
    _custom_access: '\Drupal\add_performance\Controller\mymoduleController::checkAccess'
  options:

Comment: Can you add the access check method to your Form class?


Hide running service such as Nginx from a port scan

Question: When I run a Port Scan to my server through NMap (like this)...
$ nmap -A -p443 example.com

...then I get an output like...
Starting Nmap 7.70 ( https://nmap.org ) at 2018-03-29 00:14 BST
Nmap scan report for example.com (22.214.171.124)
Host is up (0.019s latency).
rDNS record for 18.104.22.168: 126.96.36.199.example.com

PORT    STATE SERVICE  VERSION
443/tcp open  ssl/http nginx
...

However when I run a curl call to check the Server header I can see that my settings overwritten the by-default value:
$ curl -I https://example.com

...
server: my awesome web server

So my concern is how to hide the nginx result from NMap to enhance the server security.
I have been browsing this other post too (/questions/620384/hiding-services-from-port-scan) however I don't see a way to re-use the answer for my case.
Any potential help is more than welcomed.
Cheers,
Comment: nmap doesn't require the `Server` header to determine the running web server. There are many other ways to find out what it is. Spending time trying to hide the server software is useless.
Answer: Nmap uses other information not easily hidden to fingerprint software. https://nmap.org/book/vscan.html
Trying to hide this adds only a minuscule amount of security anyway because attackers just run scripts trying various exploits and seeing or not seeing that you run nginx will not help or hinder an attacker in any way.


Properties of compact operators

Question: I'm working on the following problem, regarding compact operators.

Let $T$ and $S$ be bounded linear operators on an infinite-dimensional Hilbert space. Prove or disprove the following statements:

If $T^n=I$ for some $n$, then $T$ is not compact.
If $T^2=0$ then $T$ is compact.
If $TS$ is compact then either $T$ or $S$ is compact.

The definition of compactness that I'm familiar with is: a linear operator $T$ is compact if and only if whenever $(x_n)$ is a bounded sequence, $(Tx_n)$ contains a convergent subsequence.
What I've done: For (1), I have seen the proof that an idempotent operator is compact if and only if it has finite rank. For (2), I am not sure how to start. For (3), I have only been able to prove the converse (if either $T$ or $S$ is compact than $TS$ is compact).
Any help on this problem? Thanks!
Answer: Here are some hints to help you along the way.
Use the converse of (3) to show that the negation of (1) is false (i.e., (1) is true).
For (2), consider weighted shift operators, with sequence of weights $(1,0,1,0,1,0,\ldots)$.  Show this operator is non-compact but squares to zero.
For (3), the answer to (2) shows that this is false.
Answer:
If $T^n=I$ and $T$ is compact, then $T^n$ would be compact and so would be the identity, which is not possible in an infinite dimensional Hilbert space.

Let $H$ be the space of square summable sequences and $T\colon (x_k)_{k\geq 1}\mapsto (\lambda_k x_{k+1})_{k\geqslant 1}$, where $\lambda_k=1$ if $k$ is even and $\lambda_k=0$ if $k$ is odd.

Properties of compact operators


converting a recursive formula into a non-recursive formula.

Question: We found a recursive formula for the following problem:
For any positive integer $n$, let $b(n)$ be the number of ways that you can write $n$ as a sum using only the numbers 1, 2, and 3 where the order of the sum doesn't matter.
our recursive formula is $$b(n) = b(n-3)+b(n-2)-b(n-5)+1$$
from the values $$b(0)=1, b(1)=1, b(2)=2, b(3)=3, b(4)=4, b(5)=5, b(6)=7, b(7)=8, b(8)=10, b(9)=12, b(10)=14$$
we are trying to find a non-recursive formula for this sequence, so that we can find the values for $b$ at 2012 through 2017
please also explain HOW to get the formula, as this is for marks.
Comment: There's a lot of theory to be found [at Wikipedia](http://en.wikipedia.org/wiki/Recurrence_relation). Especially see the sections _Solving non-homogeneous recurrence relations_ and _Linear homogeneous recurrence relations with constant coefficients_. Note that the general method will have you solving a 6th-degree equation, so it may not be practical here.
Comment: This is [A001399](http://oeis.org/A001399) -- in fact the first hit found by searching the On-Line Encyclopedia of Integer Sequences for your known values. It gives the simple formula $b(n)=\operatorname{round}(\frac{(n+3)^2}{12})$.
Answer: The conventional way to get a closed form for that recurrence would be first to get rid of the constant term by subtracting the equation for $n-1$ from the one for $n$:
<code>   b(n)          = b(n-2) + b(n-3)          - b(n-5)          + 1
-         b(n-1) =          b(n-3) + b(n-4)          - b(n-6) + 1
   -------------------------------------------------------------
   b(n) = b(n-1) + b(n-2)          - b(n-4) - b(n-5) + b(n-6)
</code>
You then need to find the roots of the characteristic polynomial of this equation,
$$t^6 - t^5 - t^4 + t^2 + t - 1 $$
I cheated and used Wolfram Alpha for that, but it can also be done by hand in this case because there are many $\pm1$ roots. It factors as
$$ (t^3-1)(t^2-1)(t-1) = (t+1)(t-1)^2(t^3-1) = (t+1)(t-1)^3(t-\zeta)(t-\zeta^2) $$
where $\zeta$ is a complex third root of unity. (That $(t^3-1)(t^2-1)(t-1)$ factorization sure looks like it has something to do with the 1,2,3 of your original problem statement, doesn't it?)
So the general solution of the recurrence is
$$ b(n) = c_1 + c_2n + c_3n^2 + c_4(-1)^n + c_5\zeta^n + c_6\zeta^{2n} $$
Since the sequence is obviously real, we must have $c_5=c_6$, so this becomes
$$ b(n) = c_1 + c_2n + c_3n^2 + c_4(-1)^n + c_5 \cdot\begin{cases} 2 & \text{when }3\mid n \\ -1 & \text{otherwise}\end{cases}$$
You will then need to find the $c_i$s by solving the system of linear equations you get by plugging in 5 known values of $b(n)$. Finally you can plug $n=2012$ and so forth into the formula.
Edit: Oops -- the appropriate condition for the result to be real is not $c_5=c_6$, but $c_5=\overline{c_6}$, and there can be an imaginary part in these constants. It is probably easiest then to subsume $c_1$, $c_5$ and $c_6$ into new (real) constants $k_0$, $k_1$, $k_2$, and write
$$ \tag{*} b(n) = c_2n + c_3n^2 + c_4(-1)^n + k_{n\,\bmod\,3}$$
Then six known values of $b(n)$ are necessary to find all of the constants -- and a good thing that, because five values would not have been enough to know from the sample values what the constant term we started out by eliminating from the recurrence was.
Even more edit: Note that the form of $(*)$ makes good geometric sense, since the original problem

let b(n) be the number of ways that you can write n as a sum using only the numbers 1, 2, and 3 where the order of the sum doesn't matter.

simply asks for the number of nonnegative integer solutions to $2x+3y\le n$, or in other words the number of points with integral coordinates in a triangle bounded by the horizontal and vertical axes and a line with slope $-\frac32$. To a first approximation we'd expect this to be $\frac 12 (n/3)(n/2)$, so we ought to have $c_3=\frac{1}{12}$. The other terms are there just to correct for fencepost errors. It is to be expected that there are corrections with period 2 and 3 as the axis intercepts of the hypotenuse sweeps past lattice points. It may be surprising that there is no similar periodic variation in the coefficient of the linear term $c_2n$, but that is probably because $2$ and $3$ are coprime, so the density of lattice points on the hypotenuse is the same for all $n$. It would be a different matter if it had been "using only the numbers 1, 2 and 4", for example. End edits.

If simply finding the values of $b(2012)$ upto $b(2017)$ is all you want, it is probably less work just to program a computer to fill in a table of $b(n)$ upto that number, using your recursive equation to fill in each value based on the ones you have already computed.
Answer: Write the recurrence as:
$$
b(n + 5) = b(n + 3) + b(n + 2) - b(n) + 1
$$
Define the generating function:
$$
B(z) = \sum_{n \ge 0} b(n) z^n
$$
Multiply the recurrence by $z^n$, sum over $n \ge 0$ and recognize sums like:
\begin{align}
\sum_{n \ge 0} b(n + r) z^n
  &= \frac{B(z) - b(0) - b(1) z - \ldots - b(r - 1) z^{r - 1}}{z^r} \\
\sum_{n \ge 0} z^n
  &= \frac{1}{1 - z}
\end{align}
to get:
$$
\frac{B(z) - b(0) - b(1) z - b(2) z^2 - b(3) z^3 - b(4) z^4}{z^5}
  = \frac{B(z) - b(0) - b(1) z - b(2) z^2}{z^3}
       + \frac{B(z) - b(0) - b(1) z}{z^2}
       - B(z)
       + \frac{1}{1 - z}
$$
Solving and writing as partial fractions:
\begin{align}
B(z)
  &= \frac{1}{1 - z - z^2 + z^4 + z^5 - z^6} \\
  &= \frac{17}{72} \cdot \frac{1}{1 - z}
       + \frac{1}{4} \cdot \frac{1}{(1 - z)^2}
       + \frac{1}{6} \cdot \frac{1}{(1 - z)^3}
       - \frac{1}{8} \cdot \frac{1}{1 + z}
       - \frac{2 + z}{9 (1 + z + z^2)}
\end{align}
The first terms are easy to handle by the generalized binomial theorem,
for $m$ a positive integer:
$$
(1 + u)^{-m}
  = \sum_{k \ge 0} \binom{-m}{k} u^k
  = \sum_{k \ge 0} (-1)^k \binom{k + m - 1}{m - 1} u^k
$$
Note that $\binom{k + m - 1}{m - 1}$ is a polynomial of degree $m - 1$ in $k$.
We are left with (here $\omega = - (1 + \mathrm{i} \sqrt{3}) / 2$ is a cubic root of 1):
\begin{align}
B_2(z)
  &= \frac{2 + z}{1 + z + z^2} \\
  &= \frac{2 + z}{(1 - \omega z) (1 - \omega^2 z)} \\
  &= \frac{1}{1 - \omega z} - \frac{1}{1 - \omega^2 z}
\end{align}
This is definitely getting ugly...
We could also write:
\begin{align}
B_2(z)
  &= \frac{2 + z}{1 + z + z^2} \\
  &= \frac{(2 + z) (1 - z)}{1 - z^3} \\
  &= \frac{2 - z - z^2}{1 - z^3}
\end{align}
What $B_2$ contributes to $b(n)$ is the coefficient of $z^n$:
\begin{align}
[z^n] B_2(z)
  &= [z^n] \frac{2 - z - z^2}{1 - z^3} \\
  &= 2 [z^n] (1 - z^3)^{-1}
      - [z^{n - 1}] (1 - z^3)^{-1}
      - [z^{n - 2}] (1 - z^3)^{-1} \\
  &= \begin{cases}
         2 & \text{if } 3 \mid n \\
        -1 & \text{otherwise}
     \end{cases}
\end{align}
Splicing all the above together gives a formula for $b(n)$, which can be used to compute the values asked for.


How do PGMs factor in to modern ML?

Question: I just finished the three-part series of Probabilistic Graphical Models courses from Stanford over on Coursera. I got in to them because I realized there is a certain class of problem for which the standard supervised learning approaches don't apply, for which graph search algorithms don't work, problems that don't look like RL control problems, that don't even exactly look like the kind of clustering I came to call "unsupervised learning".
In my AI courses in the Institute, we talked briefly about Bayes Nets, but it was almost as if professors considered that preamble to hotter topics like Neural Nets. Meanwhile I heard about "Expectation Maximization" and "Inference" and "Maximum Likelihood Estimation" all the time, like I was supposed to know what they were talking about. It frustrated me not to be able to remember statistics well enough to feel these things, so I decided to fill the hole by delving deeper in to PGMs.
Throughout, Koller gives examples of how to apply PGMs to things like image segmentation and speech recognition, examples that seem completely dated now because we have CNNs and LSTMs, even deep nets that encode notions of uncertainty about their beliefs.
I gather PGMs are good when:

You know the structure of the problem and can encode domain knowledge that way.
You need a generative model.
You want to learn more than just one X -> Y mapping, when you instead need a more general-purpose model that can be queried from several sides to answer different kinds of questions.
You want to feed the model inputs that look more like probability distributions than like samples.

What else are they good for? Have they not been outstripped by more advanced methods for lots of problems now? In which domains or for which specific kinds of problem are they still king? How are they complementary to modern advanced methods? Should I dedicate time to reading any of Koller & Friedman's massive tome on this subject? How dated is this set of MOOCs?
Answer: This is quite late as an answer, but I hope others can benefit from it:

What else are they good for?

Any time you need to model causality and/or feedback loops, you need PGMs. Especially chain graphs are particularly expressive.

Have they not been outstripped by more advanced methods for lots of
problems now?

I would not say so no. Most of the hype about deep learning etc, comes from the results obtained by the models. The underlying graph has a much stricter condition on the flow of the computation. Whereas in the case of PGMs the flow computations is determined by the set of factors in practice thus it can be largely dissociated from the independence assumptions encoded by the graph.

In which domains or for which specific kinds of problem are they still
king?

To my knowledge, they are extensively used in computational biology, especially in genetics. Not sure if they are still the king though.

How are they complementary to modern advanced methods?

If modern advanced methods imply deep learning, machine learning, etc. I see some PGM structures appear in machine learning libraries. But it is best to think of them as another way of attacking the same problem from a different directions.

Should I dedicate time to reading any of Koller & Friedman's massive
tome on this subject?

If you are interested in PGMs then yes at least first few chapters on inference and learning, else no.

How dated is this set of MOOCs?

Not sure.
Comment: +1 for mentioning causality. Much of the current discourse around robustness / shift stability / fairness etc. centers around causal notions, which are inevitably expressed as PGMs. Nevertheless, the actual computations people do are very different from the things Koller & Friedman discuss. In addition, there is still active research around message passing schemes for various purposes. It is more of a niche topic, though. For image/audio things like the OP describes, I think it's safe to say that nobody would use a method that doesn't have at least some DL component.

How do PGMs factor in to modern ML?


Angular server side rendering keeps going down to 502 errors

Question: So I have been running my front end on the server for a few weeks now(angular Server Side rendering). I keep running into this issue where the front end goes down to a 502 error. I have to restart the server ever few hours to ensure that it is back up. The traffic is not crazy and everything seems to be fine(in my console logs- no errors etc) until the point it suddenly goes down. the moment I restart the ssr server, it works fine again. I use the universal library that is the standard for server side rendering in Angular.  What could be the issue? what do I need to monitor? RAM? CPU? Something else?
Comment: Is your server talking to another server? Is there a load balancer in between?
Comment: yes- it talks to another server, my front end and back end are hosted separately. I had not put a load balancer. Currently there are hardly any interaction between the 2 servers (1000 requests a day probably). Is it required?
Comment: a load balancer may or may not be required, but that wasn't my point. The point is, HTTP 502 usually happens when one server (the one eventually replying with a 502) has to talk to another server and fails, for some reason. That's why I asked.
Comment: Check the logs for whatever bit of your stack is returning the 502 error.
Answer: HTTP 502 typically means that one server (the one originating the HTTP 502 response) tried to talk to another server and failed.
You mention that rebooting the "first" server (the one eventually handing out the 502) fixes the issue, which probably means there's some kind of non-persistent problem on that server.
Possible reasons:

memory exhaustion: if your frontend server has to spawn a new process or thread to talk to the backend, it may not be able to do this.

Check RAM utilization (free -m, top) and RAM limits, both global (/etc/security/limits.conf) and per process (cat /proc/PID/limits, where PID is the PID of your process).

number of open connections: maybe your frontend has a lot of open connections to the backend server, which means at some point it can't open a new one, and restarting closes those connections.

Run ss -tlpnao | grep <backend server IP> (or any other port) and compare the number of connections with the values of sysctl net.ipv4.ip_local_port_range and sysctl net.ipv4.tcp_fin_timeout .
I would also run a tcpdump -nni any host <backend ip> -v to check what's going on from a packet perspective. Do you get a reply? If so, what kind of reply? Or does the frontend simply never get a reply from the backend? This may help you find the root cause.


call external form in ajax callback

Question: I have two forms (A & B) in custom module. Form A containing three fields(each field is dependent drop down) and Form B containing only one field (markup field to show data).
The problem is that when I change either field from Form A then its dependent fields within form A an also markup field from Form B should be updated via ajax.
Here is the callback function used in FORM A. After calling, it returns fields from Form A. But I want it also return field of FORM B.
(One Callback to render two forms at a time)
<?php
 function mymodule_ajax_callback($form, $form_state) {
     return array(
        '#type' => 'ajax',
        '#commands' => array(
            ajax_command_replace("#key_wrapper", render($form['keys'])),
            ajax_command_replace("#display_wrapper", render($form['display'])),
        ),
     );
 }
?>

Answer: You can use..
function custom_conditional_form($form, &$form_state) {
  $form['form1']['drupal_love'] = array(
    '#title' => t('Do you love Drupal?'),
    '#type' => 'radios',
    '#options' => array('yes' => t('Yes'), 'no' => t('No')),
  );
  $form['form1']['how_much_love'] = array(
    '#title' => t('Great! Tell us more:'),
    '#type' => 'textfield',
    '#states' => array(
      'visible' => array(
        ':input[name="drupal_love"]' => array('value' => 'yes'),
      ),
      'required' => array(
        ':input[name="drupal_love"]' => array('value' => 'yes'),
      ),
    ),
  );
  $form['form1']['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Save'),
  );

  return $form;
}

Comment: My two forms place in two blocks, then how this code will be helpful ???
Comment: @WaQaRAli you should have (and should now edit your question to) included that information plus anything else you haven't already mentioned. We only know what you provide, and if you don't provide enough information, we are left guessing which will cause an answer that may be correct, but not useful in your situation
Comment: @Geoff now check you can see some information.
Answer: you could look at implementing something like Block Refresh
Block Refresh allows an administrator to configure a block to refresh it's content via 3 different methods: automatically via a timer (eg every 30 seconds) , manually via a 'refresh' link, and once on page load.

Block Refresh uses jQuery/AJAX so the refresh happens "behind-the-scenes".

Block refresh can be handy to serve fresh content in certain blocks on otherwise cached pages. You can opt to bypass drupal's cache for the refresh callbacks for each block.

I have used this with a bit of jQuery like this:

set it up to use a link
a link will appear on the block which is ugly, use some css to hide it (display:none)
use a simple jQuery onChange() to force the link to activate
jQuery('#selector').onChange(function(){ jQuery('#refresh').click(); });

The specifics would depend on block names, etc.
Comment: But i want to load contents in FORM B based on selected value in either field of FORM A. ie if user selects value from first field in FORM A then all the fields under this first field within this form and also field under FORM B should be updated based on selected field.
Comment: So the question really isnt about ajax, but how to relate two forms on two different blocks, and base the options in one on the value of the other?
Comment: The 1st field in FORM A consists Taxonomy parent term.
The 2nd field in FORM A consists Taxonomy child term.

When user selects value from 1st field then in 2nd field all the child terms of selected parent are shown in 2nd field(Note: this is working fine),

But Form B should also show all the contents having child terms of selected parent(Needs to be done).
Comment: Thats the sort of information that should be included in a question. This would be simplest using a Views block, Views block refresh module and a contextual filter... Assuming i now have ALL the information.
Comment: plz can you show some example???
Answer: I have solved the problem with help of little js snippet. I attached a js file to form and got my charming work.
(function($, Drupal)
 {
  function selectAListener()
   {
   // $("edit-term") is our select element in form A
    $("#edit-term").once("select-a-listener", function()
     {
     // $(this) refers to $("edit-term")
      $(this).change(function()
       {
        // $("edit-termh") is our select element in form B
        // $(this) refers to $("edit-term")
        // We need to set the value of the select element in
        // form B, then call $.change() which will trigger the #ajax
        $("#edit-termh").val($(this).val()).change();
       });
     });
   }

   function selectBListener()
   {
    $("#edit-chld").once("select-b-listener", function()
     {
      // $(this) refers to $("edit-select-a")
      $(this).change(function()
       {
        // $("edit-chldh") is our select element in form B
        // $(this) refers to $("edit-chld")
        // We need to set the value of the select element in
        // form B, then call $.change() which will trigger the #ajax
        $("#edit-chldh").val($(this).val()).change();
       });
     });
    }
Drupal.behaviors.formExample = {
      attach:function()
      {
       selectAListener();
       selectBListener();
       }

}}(jQuery, Drupal));

call external form in ajax callback


Are given functions regulated?

Question: Definition 1. A function $t: [a, b] \rightarrow \mathbb{R}$ is called a step function when a $k \in \mathbb{N}$ and numbers $z_0,...,z_k$ with $a = z_0 \leq z_1 \leq ... \leq z_k = b$ exist, such that for all $i \in \{1,2,...k\}$ the restriction $t |_{(z_{i-1},z_{i})}$ is constant.
Set $T([a,b])=\{\text{step functions on}\; [a,b]\}$.
(1) Let $f\colon [0,1] \rightarrow \mathbb{R}$ be defined by $$f(x) = \left\{ \begin{array}{rcl} 1, & \mbox{if} & x \in \mathbb{Q} \\ 0, & \mbox{if} & x \notin \mathbb{Q} \end{array} \right.$$
(2) Let $h\colon [0,1] \rightarrow \mathbb{R}$ be defined by $$h(x) = \left\{ \begin{array}{rcl} x, & \mbox{if} & x \in \{\frac{1}{n}:n\in\mathbb{N}\} \\ -x, & \mbox{if} & x \notin \{\frac{1}{n}:n\in\mathbb{N}\} \end{array} \right.$$
(a) Is $f$ regulated (https://en.wikipedia.org/wiki/Regulated_function ($\star$)), i.e.: Does $f$ satisfy: $\forall \epsilon>0$ exists $\tau\in T([0,1])$ satisfying $\|f-\tau\|_{\infty}=\sup\limits_{x\in[0,1]}|f(x)-\tau(x)|<\epsilon$?
(b) Is $h$ regulated?
Regarding (a): I think that $f$ does not have the desired property, essentially because $\mathbb{Q}$ and $\mathbb{R}\setminus \mathbb{Q}$ are dense in $\mathbb{R}$. Take $\epsilon =1/2$. If $f$ has the property $\star$, then there exists $\tau\in T([0,1])$ such that $\|f-\tau\|_{\infty}=\sup\limits_{x\in[0,1]}|f(x)-\tau(x)|<1/2$, i.e. $1/2>\sup\limits_{x\in[0,1]}|f(x)-\tau(x)|=|f(x')-c|$, since $\tau$ is piecewise constant (probably here I have to be careful that $x'\neq z_i$ ). The distance  $ |f(x')-c|$ is either $|c|$ or $|1-c|$, depending on if either $x'\in\mathbb{R}\setminus\mathbb{Q}$ or $x'\in\mathbb{Q}$. How to proceed? There must be a contradiction somewhere. Or how else to do it?
Regarding (b): My guess is that $h$ is regulated, but I'm not sure. However, I'm not sure how to define a suitable step function $\tau$ depending on $\epsilon$. Or is $h$ not regulated as well?
Answer: Hints:
For (a) - consider looking at a sufficiently small ball around $x'$ (since $\tau$ is a step function you can find a $\delta$ such that its constant on $(x'-\delta, x'+\delta)$).
For (b) - choose a small positive number $z$ and consider separately the functions $h|_{[0, z]}$ and $h|_{[z, 1]}$. For the former, $|h|$ is trivially bounded by $z$. For the latter, there are only finitely many discontinuities to worry about.
Comment: ok. f restricted on $(x'-\delta, x'+\delta)$ jumps between 0 and 1. But where is the contradiction?
Comment: @induction That interval contains both a rational and an irrational so, as you observed, you have both $|c| < 1/2$ and $|1-c| < 1/2$ which is a contradiction by the triangle inequality.
Comment: thank you. The first function is ok for me now. But I'm still struggling with $h$. I really don't know if $h$ is regulated or not
Comment: Your intuition that $h$ satisfies ($\star$) is correct. Fix $\epsilon > 0$ and try to construct a step function $\tau$ with $\|h-\tau\|_{\infty} < \epsilon$. You might consider $z = \epsilon / 2$ in the hint above.
Comment: ok. Thank you. I think I understand it now. On $[0,z]$ I can choose the zerofunction as a step function, lats call it $\tau_1$. On the open intervall $(z,1)$ we have $h(x)=-x$, thus for $\epsilon>0$ one can find a step function $\tau_2$ which satisfies $\|h-\tau_2\|<\epsilon$.
Comment: And: For $\epsilon>0$ for the partition $x_0=0, x_1=z$ and $x_2=1$ I define $\tau$ as follows: $\tau(x)=0$ if $x\in [0,x_1]$; $\tau(x)=\tau_2(x)$ if $x\in (z,1)$ and $\tau(x)=h(x)=-1$ if  $x=x_2=1$ This should work if I'm not mistaken.
Comment: uhm, almost.. I have found a mistake in my definition of $\tau$. But I can repair it
Comment: You are correct for $[0, z]$, however $h(x) \neq -x$ on $(z, 1)$ if $z < 1/2$. In particular, $h(1/2) = 1/2 \neq -1/2$. It will also differ at $1/3, 1/4$, etc., depending on how small $z$ is.

Are given functions regulated?


Separability of a dual space

Question: Let $X$ be a Banach space and assume that $(B_X, \text{weak})$ is metrizable.
WLOG, we may assume that
$$U_n = \{ x \in B_X : | f(x) | < 1, \, f \in F_n \},$$ where $F_n$ is a finite subset of $X^*$ and $n \in \mathbb{N}$, form a countable base of nbhs of $0$ in $(B_X, \text{weak})$.
To see that $X^*$ is separable, it is enough to show that $\overline{\text{span}} (\bigcup_{n \in \mathbb{N}} F_n ) = X^*$
Under this situation, pick $x^{**} \in X^{**}$ that vanishes on $\bigcup_{n \in \mathbb{N}} F_n$.
According to the textbook (Fabian), there exists a net $\{x_{\alpha}\}$ in $B_X$ that $\text{weak}^*$-converges to $x^{**}$.
I think the Goldstine theorem is used here.
But then, how can we know that such $x^{**}$ lies in $B_{X^{**}}$ (to apply the Goldstine theorem)?
Any help will be appreciated!
Comment: Consider $x^{**}/\|x^{**}\|$.
Comment: @JohnGriffin WOW. Thanks!
Answer: As @John Griffin said, by considering $x^{**}/\|x^{**}\|$, we can choose a net $\{x_{\alpha}\}$ in $B_{X}$ that $\text{weak}^*$-converges to $x^{**}/\|x^{**}\|$.
Here, we assumed that $x^{**} \neq 0$.
From the fact that for fixed $n \in \mathbb{N}$ there exists $\alpha_0$ such that  $$ |\,f(x_{\alpha})| = \left| \, f\left( \frac{x^{**}}{\|x^{**}\|} - x_{\alpha}\right) \right| < 1 $$
for all $f \in F_n$ whenever $\alpha \geq \alpha_0$, we can conclude that
$x_{\alpha} \in U_n$ for $\alpha \geq \alpha_0$.
Hence $x_{\alpha} \rightarrow 0$ in weak-sense; thus $x^{**}/\|x^{**}\| = 0$, a contradiction!
Thus, $x^{**} = 0$.
It follows that $\left(\text{span}\left( \bigcup_{n\in\mathbb{N}} F_n\right)\right)^{\perp} = \{0\}$, therefore $\overline{\text{span}}\left( \bigcup_{n\in\mathbb{N}} F_n\right) = X^{*}.$
The proof is done.

Separability of a dual space


Show Coverage in github PR

Question: We run tests on each pull request in github.
It would be very cool if the coverage would be visible if you are looking at the changes of a pull request.
While looking at the diff in the PR review I would like to see which line was covered and which line not.
We use pytest and travis.
I found this: https://github.com/TheKevJames/coveralls-python
But AFAIK this shows the coverage reports somewhere else, not in the "Files changed" tab of the PR.
This question is not about reaching a overall coverage of X percent. This question is about seeing which lines where covered and which lines where not covered.
Comment: Take a look at [`coverage-diff`](https://pypi.org/project/coverage-diff/) or [`diff-cover`](https://diff-cover.readthedocs.io/). However, IMO the easiest approach is to have 100% code coverage and make tests fail if it drops below, e.g. running `pytest --cov-fail-under=100` in the CI (using the `pytest-cov` plugin).
Comment: so whats the question? You want code to show coverage in Github on a pull request?
Comment: @JulianSilvestri It would be very cool if the coverage would be visible if you are looking at the changes of a pull request.
Comment: have you tried the python package 'coverage' ?
Comment: maybe this will help ? https://about.codecov.io/blog/python-code-coverage-using-github-actions-and-codecov/
Comment: @JulianSilvestri codecov.io is a commercial service. I thought this is doable with scripting.
Comment: im sure it is, but you can attempt to learn how they do it ?
Comment: @hoefling have you ever seen a project with 100% code coverage??
Comment: @be-chiller-too yep, a lot. What's your question, exactly? You need examples of good projects to learn from? Or are you questioning the usefulness of a full code coverage in general?
Comment: Code coverage is great, but I believe that the projects that reach 90% code coverage are really rare, let alone those with 100%
Answer: You can achieve this using Github Actions and the Github Check runs API.
Generic flow of the Github Action workflow will be:

Run pytest on the changed files and then generate coverage report.
Use coverage report to annotate the commit.

The generated report looks something like this:

I have created a repository : pytest-annotate-pr to demonstrate how this can be done. Note that, this is just a demonstration of how this can be done, and it's far from production ready. For example, this currently only works when a commit is pushed in a PR :) It does not work when branches are merged into main. The above screenshot is from one of the PR raised against main.
If you work on this and make a production ready workflow, please consider making it into a Github Action.
Answer: A partial solution: In one of the projects I work in, we have a codecov action set up. This action has a feature to annotate lines that were not covered in the "Files changed" tab:

This only shows you which lines were not covered, but maybe that's better than nothing?
Edit: I just now noticed that Julian already suggested codecov in the comments and that you said you wanted something non-commercial. I can only add there that codecov is free for public repos as well as teams with up to five members. If you have a private repo and a bigger team and do not want to pay for this service, this doesn't help you.
I think I should still keep the answer here in case someone finds this question when searching for a solution for an open source project.
Comment: Thank you. This answer is better than nothing. Maybe I should switch to gitlab, since I prefer usually "do it yourself" to "buy".
Answer: You can output the code coverage into the README file, this way you will see the code coverage when you are in the "Files changed" tab AND when you access the project's page.
Comment: I don't think you can do this, it's the purpose of a coverage report, but you do this offline before pushing your code.
Comment: I updated the question to make it more clear: I want to see which lines are covered and which lines are not.

Show Coverage in github PR