Neural networks and signal-to-noise ratio

Question: My guess is that neural networks do not work very well in noisy environments, i.e. the lower the signal-to-noise ratio, the worse the result of a neural network, if compared to other statistical modeling tools. Thus, for example, neural networks are good at predicting credit cards frauds, but they get much less exciting results when trying to predict financial markets (very noisy, at least in the short term).
Any theoretical result and/or empirical evidence on that?
Comment: IMHO, this is not too broad. There should be some generic knowledge about this, after all.
Comment: This is an overly broad question. Can you be more specific?
Comment: Why would this be the case?  Why would other ML approaches be any better in situations where there is substantial idiosyncratic error?
Comment: If you overfit a network it will try to follow the noise as well as the signal, but this can happen for spline and polynomial fits too. There are techniques to avoid overfitting.
Answer: If you have lots of inputs but only a few that matter, the neural network may try to overfit to that noise. L1 regularization can help in that aspect as it leads to sparse weight vectors, resulting in noisy / unimportant variables having the weights linked to them set to 0 (you need to find a correct regularization strength). You can read L1 Norm Regularization and Sparsity Explained for Dummies by Shi Yan if you want to get an intuitive grasp of how that works.

The reason for using L1 norm to find a sparse solution is due to its special shape. It has spikes that happen to be at sparse points. Using it to touch the solution surface will very likely to find a touch point on a spike tip and thus a sparse solution.

With lots of noise, the training might require more data as you increase the search space and may require larger batch sizes and smaller learning rates to avoid going in the wrong direction.
However, adding a little noise can prove beneficial in some cases.
Train Neural Networks With Noise to Reduce Overfitting by Jason Brownlee explains how

The addition of noise during the training of a neural network model has a regularization effect and, in turn, improves the robustness of the model. It has been shown to have a similar impact on the loss function as the addition of a penalty term, as in the case of weight regularization methods.
Comment: @Avelina, I agree with the push-pull aspect and I didn't try to go into details about how this works. But taking only this into account could result in smaller weights for less important variables, just like L2 but with less punishment on magnitude. The sparsity comes  from the "pointiness" of the shape which surface has constant L1 norm, the pointy bits being sparse vectors (Hard to summarize here, please read [L1 Norm Regularization and Sparsity Explained for Dummies](https://blog.mlreview.com/l1-norm-regularization-and-sparsity-explained-for-dummies-5b0e4be3938a).
Comment: Hi, welcome to CV. Please add a reference for your link in case it dies in the future. Please also summarize the content of the link.
Comment: I wouldn't say that "it judges useful by setting the weights of the others to 0" is accurate. That's the effect of L1 regularization, but that is far from how the mechanism works. L1 regularization applies a constant penalty based on the L1 norm which 'pushes' weights towards zero uniformly (unlike L2 which penalises larger weights more than smaller ones), while the optimizer attempts to adjust weights from backpropagation (which may pull weights away from zero); this push-pull results in the 'most important' features having non-zero weights. Learning rate and L1 amount will need fine-tuning.


Reinserindo elementos em outra parte do Body, html e Javascript

Question: Estou tentando armazenar em uma var, e depois reinserir no body 3 elementos do tipo "p".

 Um

 Dois

 Tres
 ....

Ao fazer isso:

O que eu tenho impressos na tela é uma espécie de "rótulo" ou nome da lista de objetos(para o DOM, ou elementos, pro html, correto?) "p":
Imprime assim:
[object NodeList]
Agora, tem como eu buscar esses elementos "p", e efetivamente reimprimir na body o seu conteúdo??
Comment: `querySelectorAll` devolve um `NodeList` logo para utilizar cada um dos elementos precisa de um `for`. Respondi à bem pouco tempo a uma questão muito semelhante à sua se não mesmo duplicada. [Veja aqui](https://pt.stackoverflow.com/questions/290025/como-inserir-uma-nova-div-e-empilh%C3%A1-las-ap%C3%B3s-clicar-em-um-bot%C3%A3o/290043#290043)
Answer: Ao usar document.querySelectorAll irá gerar um objeto com uma coleção de nós (nodes) dos elementos selecionados no seletor, no seu caso o "p" (tag
).
Para reimprimi-los usando document.write você pode usar um laço for pegando o HTML de cada item da coleção com outerHTML. Existem outras formas de inserir os itens na página, mas usando document.write poderia ser desta forma:

var parag = document.querySelectorAll("p");

for(var x=0; x<parag.length; x++){
   document.write(parag[x].outerHTML);
}

 Um

 Dois

 Tres

Você pode também clonar os elementos e ir reinserindo na página usando o método cloneNode:

var parag = document.querySelectorAll("p");
for(var x=0; x<parag.length; x++){
   var clone = parag[x].cloneNode(true); // true quer dizer selecionar tudo do elemento
   document.body.appendChild(clone);
}

 Um

 Dois

 Tres


invalid v token expires twitter sharekit

Question: I am using sharekit to share url from my xcode app. I am getting Invalid V Token expires and can not able to share link. any idea about this? Consider am testing on simulator and previously it was worked fine but now it through an error like this. I have tried after resetting keys as well ..no luck
Answer: This is not related to Twitter. Twitter OAuth tokens never expire.
Comment: Air-dex, I did not say it is related to twitter, I was saying that when I want to share link on twitter using sharekit I am getting this error..


Storing class instances in spreadsheet's cells

Question: is there a way to store user defined class instances in a cell? i would like to store the instances' data somehow and preferably displaying the properties of the class
Comment: Cells may contain scalar values only, you should create a procedure for object properties storing into cells one by one.
Answer: Cells cannot hold objects only values. Therefore you need to save all values into cells.
Add 2 procedures to your class eg SaveToWorksheet and LoadFromWorksheet. One that saves all data to a desired worksheet and one that is able to load the data from a worksheet into a instance.
So you can do something like:
Dim MyInstance = New MyClass
'do something with that instance
MyInstance.SaveToWorksheet "SheetName"

Set MyInstance = Nothing 'kill the instance

Dim MyNewInstance = New MyClass
MyNewInstance.LoadFromWorksheet "SheetName"
'should recreate the instance with the data from the worksheet

How to save/load your data depends on how your class looks like and how you desire to save the data to the sheet.
Comment: In practice, you might want to pass a Range of an open Worksheet as the parameter to the procedures or a starting Cell, rather than a Sheet Name, but +1 for the answer.
Comment: @JonathanWillcock Yes, of course there is a lot of space for improvement. Actually I just wanted to give the idea along, that the worksheet does not have to be hardcoded in the class and should be kept variable to each instance.
Answer: Create default member of a class

Export your class module;
Save it as text file with .cls extension;
Remove original class from your project;
Open exported module in a Notepad;
Add the following line Attribute Value.VB_UserMemId = 0
Specify your desired property as a default

Example of specifying a property as a default:
Property Get Value() As Long
    Attribute Value.VB_UserMemId = 0
    Value = Whatever
End Property

Save your modified file and import it back to the project

Example of using a default property:
Dim CC As CMyClassName
Set CC = New CMyClassName
CC.Value = 123
' is the same as
CC = 123

Source


Two Bessel Functions

Question:
Let $I_k$ denote the modified Bessel function of the first kind of index $k$. Show that, if $y > x > 0$, then
  $$-I_0(x)I_1(y) + I_0(y)I_1(x) <0$$

I am looking for a source that provides this information.
Comment: You are surely aware that the derivative of $I_0$ is $I_1$. This property has hopefuly a role to play...
Comment: Please learn to use `$$...$$`.
Answer: From the integral representation
$$
I_0(x)I_0(y)=\frac{1}{\pi}\int_0^\pi I_0(\sqrt{x^2+y^2-2xy\cos\theta})d\theta,
$$
using the formula $I_1(x)=I_0'(x)$ we immediately see that
$$-I_0(x)I_1(y)+I_0(y)I_1(x)=\frac{1}{\pi}\int_0^\pi \frac{I_1(\sqrt{x^2+y^2-2xy\cos\theta})}{\sqrt{x^2+y^2-2xy\cos\theta}}(x-y)(1+\cos\theta)d\theta.
$$
For $0<x<y$ this is clearly less than $0$, thus proving $$-I_0(x)I_1(y)+I_0(y)I_1(x)<0,~0<x<y.$$
Since $I_1(x)/x$ is monotonically increasing for $x>0$, it is also possible to prove a stronger inequality

$$
-I_0(x)I_1(y)+I_0(y)I_1(x)<-I_1(y-x),~0<x<y.
$$
Comment: This is great, thanks so much. Would you happen to have a source where the integral representation comes from?
Comment: This integral representation is obtained by integrating the addition theorem for the function $I_0$, $I_0(\sqrt{x^2+y^2-2xy\cos\theta})=\sum_{n=-\infty}^\infty I_n(x)I_n(y)\cos n\theta$, (see equation 11.41.(6) on page 365, Treatise on the theory of Bessel funcions, by G.N. Watson) with respect to $\theta$
Answer: There is a useful lemma that settles the question very nicely here.

If $f(z)$ and $g(z)$ are two real analytic functions in a neighbourhood $U$
  of the origin, $$ f(z)=\sum_{n\geq 0}a_n z^n,\qquad g(z)=\sum_{n\geq
> 0}b_n z^n $$ with $a_n,b_n>0$ and the sequence
  $\left\{\frac{a_n}{b_n}\right\}_{n\geq 0}$ being an
  increasing/decreasing sequence, then the ratio
  $r(x)=\frac{f(x)}{g(x)}$ is an increasing/decreasing function over
  $U$.

The proof is not difficult: it is enough to consider $f'g-g\, f'$ in terms of power series to deduce that $\frac{dr}{dx}$ has constant sign. In our case, the coefficients of $I_0(x)$ and $I_1(x)$ have a simple structure, and the above lemma leads to:

$\displaystyle\frac{I_0(x)}{I_1(x)}$ is a decreasing function over $\mathbb{R}^+$;
$\displaystyle\frac{x\cdot I_0(x)}{I_1(x)}$ is an increasing function over $\mathbb{R}^+$.

Additionally, $\frac{I_0}{I_1}$ has a very nice representation in terms of continued fractions, that makes the above statements almost trivial.

Two Bessel Functions


Why array not working with function call?

Question: My array not taking inputs (Ex- 5.21).I am trying to take a float input for each of my array elements but this isn't doing this. and Also this function is not working with the array. please help
float maxJump (double jumpRecord[], int attempts)
{
    float max = jumpRecord[0];
    for(int i=0; i<attempts; ++i)
    {

        if(jumpRecord[i]>max)
            max = jumpRecord[i];
    }

    printf("Longest Jump = %.2f meter\n",max);
}

int main()
{
    double jumpRecord[5];

    printf("Jumping information (in meter): \n");
    for(int i=0; i<5; i++)
    {
        printf("Attempt %d. = ",i+1);
        scanf("%.2f",&jumpRecord[i]);
    }

    maxJump(jumpRecord,5);
    return 0;
}

Comment: "Also this function is not working with the array." This is not a useful description. What is not working? What do you expect, what do you get, what input do you provide? Same for "but this isn't doing this" What is it doing instead? How do you know, you do not care about return value of `scanf` and hence cannot know if it did as expected. You should always check return values of all input/output functions
Answer: When used with scanf(), %.2f (%f) format specifier is for reading float values.
The elements of jumpRecord are double, so you should use %lf (%.2lf) format specifier instead.
Note that %f should be used for printing double via printf(). %lf is allowed in C99 or later, but %f is more portable.


How can I open a new email using AppleScript, setting "To:", "Subject:", and attaching a file?

Question: To help my users report problems easily, I want to create as much of an email as possible. Most importantly, I want to attach a file (the log file).
How can I do this in AppleScript?
Answer: <code>tell application "Mail"
    set theMessage to make new outgoing message with properties {visible:true, subject:"My Subject", content:"My Body"}
    tell theMessage
        make new to recipient at end of to recipients with properties {name:"Ben Waldie", address:"email@example.com"}
    end tell
    tell content of theMessage
        make new attachment with properties {file name:pathToFile} at after last paragraph
    end tell
end tell
</code>
Comment: Thanks, that did the trick. I had to make a small change - make new to recipient didn't work in "tell content of theMessage" block"
Comment: Sorry about that and thanks for fixing it. It's still early and I was in a rush.


How to prove that if f is continuous that the exponential function satisfies the functional equation?

Question: Prove that if   $f : \mathbb{R}→\mathbb{R}$ is continuous with the property $f(x+y)=f(x)·f(y)$ for all $x,y \in\mathbb{R}$ and $f(1) = a > 0$, then $f(x) = e^x$.
I'd be greatfull for any help!
Comment: This question has appeared many many many times!
Comment: Your question title means something different from the text. The title asks to verify that $a^x$ is a solution, but "then $f(x)=a^x$" means you need to **disqualify** all functions except $a^x$ as solutions.
Answer: $f(x)=f(0+x)=f(0)f(x)$ so $f(0)=1$ or $f(x)=0$. Since $f(1)>0$ the latter is denied.
For $n\in\mathbb{N}$, $f(n)=f(1+\cdots+1)=f(1)^n=a^n$ by induction.
$1=f(0)=f(n-n)=f(n)f(-n)$ hence $f(-n)=f(n)^{-1}$.
$f(\frac{1}{n}+\cdots+\frac{1}{n})=f(1)=a$ implies $f(\frac{1}{n})=a^{1/n}$.
Hence $f(\frac{m}{n})=a^{m/n}$.
Now, since $f$ is assumed to be continuous, one can continue to the conclusion that $f(x)=a^x$ by taking rational approximations $p_n\to x$, $f(x)=\lim_{n\to\infty}f(p_n)=\lim_{n\to\infty}a^{p_n}=a^x$.
Comment: If $f(0)\ne1$ then $f(x)=0$ for all $x$, including $1$.
Comment: You cannot prove $f(x)=e^x$. For example, $2^{x+y}=2^x2^y$ is a valid possibility.
Comment: how does $f(1)=a$ deny the latter?
Comment: Thank you very much Chrystomath! It was really helpful! I now struggle with the second part of the proof where I need to show that e^
Answer: You can show inductively, that the statment holds on $\{ \frac{m}{2^n} | n,m\ \in \mathbb{Z} \}$ and this subset is dense.


field_widget_form call as a custom form

Question: I want to call my custom field_widget_form with the use of drupal_get_form()
Is it possible to do that?
simple requirement .. I want to build a form where user enter the field value and that value should be store with the node but only one field value instead of all 30 fields.
Now i have created the field with custom code so how can i call this function field_widget_form() in that or any other help full alternative.
Comment: is there any chance you can update/re-phrase the question a bit. I believe I have the same the question, but can't tell, and the answer doesn't make sense to me?
Answer: When you define a widget form, you can change it from another module with hook_field_widget_form_alter(), which requires Drupal 7.8 or higher.
<code>function hook_field_widget_form_alter(&$element, &$form_state, $context) {
  // Add a css class to widget form elements for all fields of type mytype.
  if ($context['field']['type'] == 'mytype') {
    // Be sure not to overwrite existing attributes.
    $element['#attributes']['class'][] = 'myclass';
  }
}
</code>

field_widget_form call as a custom form


Azure AD B2C & Microsoft Identity Web - Sign In with multiple policies (.net Core 3.1)

Question: I have an application using .NET Core 3.1 MVC Web App that uses Azure AD B2C to sign in users and I've just migrated it to use Microsoft Identity Web library.
We want to have two different policies for Sign In, one for regular users (B2C_1A_SignUpOrSignIn) and one for admin users (B2C_1A_SignInAdmin).
So, in the Appsettings, we have the following format:
"AzureAdB2C": {
    "Instance": "https://url.b2clogin.com/tfp/",
    "ClientId": "clientId",
    "CallbackPath": "/signin-oidc",
    "SignedOutCallbackPath": "/signout/B2C_1A_SignUpOrSignIn",
    "Domain": "url.onmicrosoft.com",
    "Domain_b2cLogin": "url", // Required by the Cookie Policy
    "SignUpSignInPolicyId": "B2C_1A_SignUpOrSignIn",
    "SignInAdminPolicyId": "B2C_1A_SignInAdmin",
    "ResetPasswordPolicyId": "B2C_1A_PasswordReset",
    "EditProfilePolicyId": "",
    "ClientSecret": key,
    "B2cExtensionAppClientId": "key"
  },

In the Startup class, I just added the following:
            services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
           .AddMicrosoftIdentityWebApp(Configuration, "AzureAdB2C");

And I also overrode the "AzureController", so, for a regular user, I use the same method obtained from
here Microsoft Identity Web - Account Controller "SignIn". However, for an admin user, I changed that method to use something similar to what is provided by the PasswordReset method, as the following:
        public IActionResult SignInAdmin()
    {
        string scheme = OpenIdConnectDefaults.AuthenticationScheme;

        var redirectUrl = Url.Content("~/");
        var properties = new AuthenticationProperties { RedirectUri = redirectUrl };
        properties.Items[Constants.Policy] = "B2C_1A_SignInAdmin";
        return Challenge(properties, scheme);
    }

So, as you can see, I'm using a different policy name for this method.
Everything seems to work fine, the user is redirected to the correct login page based on the policy and the token is issued by Azure and our application accepts the Token, in our method
options.Events.OnTokenValidated = context => {}

However, soon after that, something goes wrong with the authentication and the method
options.Events.OnRemoteFailure

is called with the following exception
"{"Message contains error: 'invalid_grant', error_description: 'AADB2C90088: The provided grant has not been issued for this endpoint. Actual Value : B2C_1A_SignUpOrSignIn and Expected Value : B2C_1A_SignInAdmin ..."
So, my question is, what do I have to do to be able to use two different policies to sign in? Or is there any configuration that I should do to be able to do that?
Thank you in advanced.
Comment: Thanks @Henrique and Could you please follow these two example and let us know if issue -error still persist - https://github.com/azure-ad-b2c/apps/blob/master/apps/spa-hellojs-popup/source-code/.Net-Core-API-RBAC/MultiBearerAPI/Startup.cs#L30
Comment: @Jit_MSFT Do you have an example of this for .NET framework (4.7.2)?
I am specifically trying to call an API which I want to access with 2 different policies and applications.
Answer: I guess the B2c Middleware validates the "tfp" claim. This should usually match the SignInPolicyId. You might have to override the TokenValidation to let both policies (tfp = trustedFrameworkPolicy) be valid


Instrumental intro on "Piano and melody with lyrics" score in LilyPond

Question: I am creating a LilyPond piano/vocal score starting with the Template A.3.2 Piano and melody with lyrics.  In my piece, there will be several bars of instrumental introduction before the vocal line comes in, and I would like the staff for the vocal to be omitted for that part.
The template produces this:

What I have been doing is:

Pad the start of the vocal line with sufficient rests to cover the duration of the introduction.
Change the \RemoveEmptyStaves option inside the \layout block to \RemoveAllEmptyStaves, since \RemoveEmptyStaves will not affect the first system in a score.
Include a \break on the vocal line after the last measure of the introduction to force the change from 2 staves to 3 to occur at the right point.

I now realize that I could also do this:

Store the notes expressions for the piano intro in separate variables, (say upperIntro = { } and lowerIntro = { }
Include two top-level \score blocks (one with 2 staves for the intro and one with 3 staves for the remainder).

Or, as indirectly suggested by Lazy's answer:

Put the 2-staff intro and 3-staff main part into their own << ... >> blocks.  (This does not seem to work inside a \score block– it throws an error and only outputs the intro part.)

As far as I can tell, the only visual difference is that the version with a single \score block and break justifies the output for each section:

while the version with two \score blocks (or two << ... >> blocks) left-aligns both sections:

Is there any other reason to prefer one method over the others, i.e. are there any side-effects to the choice that impact flexibility, modularity, appearance, etc.?
Are there other, better ways to accomplish the same thing?
Answer: Well, you can make the second version look like the first one. But I advocate for using \RemoveAllEmptyStaves, unless you really want to separate the introduction and the main part. Doing a break before the voice starts is not really nescessary, because even if the break is before the part you will just get a few rests in the voice. Manually specifying breaking might be nescessary if the break falls just slightly after the start of the vocal line. Else doing lots of breaks is just detrimental to spacing. Also a possibility instead of manual breaks is using \noBreak in places where a break is not good.
For example look at this:
<<
  \new Staff \with { \RemoveAllEmptyStaves }
    {
    R1*8 c''4 d'' e'' f'' c''4 d'' e'' f'' c''4 d'' e'' f'' c''4 d'' e'' f'' \repeat unfold 8 { c''4 d'' e'' f'' }}
  \new PianoStaff
  <<
    \new Staff \repeat unfold 20 { c' d' e' f' }
    \new Staff {\clef bass \repeat unfold 20 { c2 g }}
  >>
>>

<<
  \new Staff \with { \RemoveAllEmptyStaves }
    {
    R1*8 \break c''4 d'' e'' f'' c''4 d'' e'' f'' c''4 d'' e'' f'' c''4 d'' e'' f'' \repeat unfold 8 { c''4 d'' e'' f'' }}
  \new PianoStaff
  <<
    \new Staff \repeat unfold 20 { c' d' e' f' }
    \new Staff {\clef bass \repeat unfold 20 { c2 g }}
  >>
>>

here the vocal line would start two bars in, which is not bad. But specifying \break forces lilypond to make the piano very dense in the first system.
Comment: I suppose either way is acceptable, though the majority of the commercially-engraved piano/vocal sheet music in my library looks like second version.
Comment: Please see expanded question. Specifically, could you explain why I might or might not "really want to separate the introduction and the main part"? Thanks.
Comment: @Theodore The only time I'd see a reason for separating these two is if the introduction is a separate entity on it's own. Like if you have a organ prelude to some chorale or something.


In the integral of a function, why is it that I am able to take out a function and claim it is smaller than the integral itself?

Question: Could you please explain what happens in the second last line, where they had the integral larger or equal to the e^-s integral of 1/t. I am confused as it seems that they just pulled the e^-s out and claimed that it was smaller than the integral itself. What law are they using here? sorry I did not type in Latex, but please refer to the image below. thank you.
Comment: You are told why: "On $[0,1]$, $\mathrm{e}^{-st} \geq \mathrm{e}^{-s}$ for all $s$."  Why is that explanation unsatisfactory?
Comment: @EricTowers  OP does not understand how that is being used.
Comment: yes but  e− has no integral here
Answer: First, it is a general fact that, if $f(x) \geq g(x)$ for all $x \in [a,b]$, then
$$  \int_a^b f(x) \,\mathrm{d}x \geq \int_a^b g(x) \,\mathrm{d}x  \text{.}  $$
(To prove this, observe that $f(x) - g(x) \geq 0$.  Then subtract the right-hand side of the display form the left-hand side, observe that the new (single) integral on the left is an integral of an everywhere nonnegative function, so is nonnegative.)
Then, from $\mathrm{e}^{-st} \geq \mathrm{e}^{-s}$ on the interval of integration, we have
\begin{align*}
\mathrm{e}^{-st} &\geq \mathrm{e}^{-s}, \\
\frac{\mathrm{e}^{-st}}{t} &\geq \frac{\mathrm{e}^{-s}}{t} \qquad t \neq 0, \\
\int_0^1 \; \frac{\mathrm{e}^{-st}}{t} \,\mathrm{d}t
    &\geq \int_0^1 \; \frac{\mathrm{e}^{-s}}{t} \,\mathrm{d}t  \\
    &= \mathrm{e}^{-s}\int_0^1 \; \frac{1}{t} \,\mathrm{d}t  \text{,}
\end{align*}
because $\mathrm{e}^{-s}$ is independent of $t$ so is constant while evaluating the integral.
Comment: OP doesn't realize it's a constant in the integral.
Comment: when you say subtract the right-hand side of the display, which display do you mean?
Comment: the new single integral is f-g? thanks
Comment: @JosephRock : There is only one display prior to that sentence.
Comment: @JosephRock : Yes, the left-hand side is $$  \int_a^b \;f(x) - g(x) \,\mathrm{d}x  \text{.}  $$
Comment: i see, from here do you use the linear transformation property of integrals? apologies for the rudimentary question if it is
Comment: @JosephRock : No.  You use the inequality with which we started that paragraph.  This integrand is everywhere nonnegative, so the integral is nonnegative.  So the claimed inequality holds.
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/113215/discussion-between-joseph-rock-and-eric-towers).
Comment: u are right randall


Counter is always set to 1 and not incrementing

Question: I want Anonymous users to redirect to login page if they have read three articles as trail. I wrote this code but the counter on any article is always set to one.I guess its initializing again and again. Can anybody please help with code.
<?php
function track_node_view($node, $view_mode) {
if (user_is_anonymous()) {
  if($node->type == 'article' && $view_mode=='full'){
      static $counter = 0;
      if($counter <= 3) {
        $counter += 1;
        dsm($counter);
        drupal_set_message($counter);
      }
      else {
        drupal_goto("user/login");
      }
   }
  }

Answer: The problem is that you failed to grasp how HTTP is basically stateless.
static $counter = 0;

This value will be kept during one run. But one run is one page load. Obviously, it will be fresh and new at all page loads - ie all article reads.
If you need to keep value in one user session, see Session API, that's what it is for.
If you don't need any extra functionalities, just use $_SESSION['mymodule_counter']. If you want to make sure session is started, use drupal_session_start()
Comment: I don't know how to work with session api, i tried but not getting anything. can you suggest me how should i start.
Comment: Just write to `$_SESSION['mymodule_counter']` ?
Comment: should i need to write the code to create a session or just start with $_SESSION?
Comment: See my updated answer. got to go now, bye.
Answer: It is not working because that hook is normally invoked once for each page request, and at each page request, Drupal is getting bootstrapped; that means that once Drupal served the page request, every static variable is reset to its default value (in your case, 0). In your case, you should use variable_set() to set a Drupal persistent variable, and variable_get() to get its value.
Notice also that hook_node_view() should not redirect users like your code does; it is going to give you troubles with cron tasks. In fact, that hook is invoked even when a module is calling node_view() without any user watching a node page.
Comment: So how should i redirect users without hook_node_view?
Comment: `variable_set()` is global, and he wants per-user data, right? May you elaborate how would you distinguish anonymous users, and why not to use sessions for this?
Answer: In my mind your code does the following:
An anonymous user shows an article. Then (why static, makes no sense) a counter is declared with 0. Then the counter increments to 1.
Then any other or the same user (because there's no saving in user session) clicks anywhere and looks an article. The counter is again set to 0, then to 1 etc...
I assume you want the following scenario:
An anonymous user comes to your site, clicks 3 articles, and when clicking to the 4th he shall be redirected to the login site.
So what do you need:
You need a session handler for anonymous user. But Drupal haven't got sessions for them.
So here are the drupal functions for sessions:
drupal user session
To start a session for anonymous user:
if(session_id() == '') {
  session_start();
}

Then in your module your counter comes to play a role, so start with your hook_node_view:
function track_node_view($node, $view_mode) {
if (user_is_anonymous()) {
//Look if the counter is already set, if not create it
  if(!isset($_SESSION['track'])){
    $_SESSION['track'] = array();
  }
//Then look if the counter is in the session array(defaults to NULL)
  $tracker = track_session_get('track');
// if $tracker isn't available initialise your $counter
  if($tracker == NULL){
    $counter = 0;
  }
 // if session track is set and tracker != NULL you have to get the variable for counter
  if($tracker != NULL){
    $counter = $_SESSION['track']['counter'];
  }
 //now you should have a number between 1-3
  if($counter <= 3){
    $counter = ++$counter;
    //store this new value in session
    track_session_set('track',$counter);
  }
  else{
   //set the $counter to 0 and redirect
   $counter = 0;
   track_session_set('track',$counter);
   drupal_goto("user/login");
  }
 }
}

function track_session_get($session_name, $default = NULL) {
  if (isset($_SESSION[$session_name])) {
    return $_SESSION[$session_name];
  }
  return $default;
}

function track_session_set($session_name, $session_value) {
  $_SESSION[$session_name] = $session_value;
}

I did not tested this code, so this is only an advise, not a howto;)
Comment: Wow, thanks alot maen. Great effort. really grateful. working on it. Thanks again Sir.


Is my GAM modeling longitudinal change appropriately?

Question: My data includes the following:

Y which is a physiological measure
Age continuous
Gender factor, Male or Female
Timepoint One for each year of study visit: 1,2,3,4
subject each unique subject

Model aim:

To identify longitudinal changes of Y across Timepoints, for each subject between Gender

I am assuming a random slope and intercept is appropriate as each subject will have a different physiological change between Timepoints

I am also trying to use gratia::smooth_estimates to plot the smooth term for age between Gender

Model:
m1 <- gam(Y ~ subject + Gender + s(age, by = subject) + s(Age, by = Gender, bs = 'fs') + s(Timepoint, subject, bs ='fs') +  s(subject, bs = "re") ,
    data = DF,
    method = "REML")

However, this does not work,
Error in smooth.construct.tp.smooth.spec(object, dk$data, dk$knots) :  A term has fewer unique covariate combinations than specified maximum degrees of freedom
So I have to use:
m1.reduced <- gam(Y ~ Gender + s(Age, by = Gender, bs = 'fs') + s(subject, bs = "re"),
          method = "REML",
          data = DF_y)

Or I can also use gamm4 (which works with the plot below, though I am not sure if it is appropriately modeling what I am intending:
m2 <- gamm4(Y ~ s(Age, by = Gender, bs = 'fs') + Gender+ Timepoint,
                     random = ~ (1 + Timepoint | subject),
                     data = DF)

Plot:
sm <- smooth_estimates(m1) %>%
  add_confint()

ggplot(data = sm, aes(y = est, x = Age)) +
  geom_ribbon(aes(ymin = lower_ci, ymax = upper_ci, fill = Gender),
                alpha = 0.25) +
  geom_line(aes(color = Gender),  size = 1.5) +
  geom_point(data = DF, aes(Age, Y, color = Gender), alpha = .25) +
  geom_line(data = DF, aes(Age, Y, group = subject), alpha = 0.25) +
  ylab("Partial effect") +
  xlab("Age"))

I am trying to replicate a plot like this (source): where each color is a gender and the predicted longitudinal trajectory is plotted across age.

This is my model 2 (gamm4) plot that is providing what I want.
However, is my gam model modeling and plotting longitudinal change correctly?

sample data:
structure(list(subject= c(33714L, 35377L, 40556L, 40798L, 40800L,
40815L, 50848L, 52183L, 52461L, 53320L, 53873L, 54206L, 54581L,
55122L, 55267L, 55462L, 55612L, 55920L, 56022L, 56307L, 56420L,
56679L, 57405L, 57480L, 57725L, 57809L, 58004L, 58215L, 58229L,
59326L, 59327L, 59865L, 60099L, 60100L, 60280L, 60384L, 60429L,
60493L, 60503L, 60603L, 60664L, 60846L, 61415L, 61749L, 61883L,
62081L, 62983L, 63327L, 63329L, 64418L, 64507L, 64596L, 65178L,
65250L, 65802L, 65975L, 65978L, 66396L, 66572L, 66589L, 74034L,
74427L, 74607L, 74952L, 75732L, 76574L, 76595L, 76755L, 76759L,
77203L, 77453L, 77668L, 81064L, 81065L, 33714L, 35377L, 40556L,
40798L, 40800L, 40815L, 50848L, 52183L, 52461L, 53320L, 53873L,
54206L, 54581L, 55122L, 55267L, 55462L, 55612L, 55920L, 56022L,
56307L, 56420L, 56679L, 57405L, 57480L, 57725L, 57809L, 58004L,
58215L, 58229L, 59326L, 59327L, 59865L, 60099L, 60100L, 60280L,
60384L, 60429L, 60493L, 60503L, 60603L, 60664L, 60846L, 61415L,
61749L, 61883L, 62081L, 62983L, 63327L, 63329L, 64418L, 64507L,
64596L, 65178L, 65250L, 65802L, 65975L, 65978L, 66396L, 66572L,
66589L, 74034L, 74427L, 74607L, 74952L, 75732L, 76574L, 76595L,
76755L, 76759L, 77203L, 77453L, 77668L, 81064L, 81065L, 33714L,
35377L, 40556L, 40798L, 40800L, 40815L, 50848L, 52183L, 52461L,
53320L, 53873L, 54206L, 54581L, 55122L, 55267L, 55462L, 55612L,
55920L, 56022L, 56307L, 56420L, 56679L, 57405L, 57480L, 57725L,
57809L, 58004L, 58215L, 58229L, 59326L, 59327L, 59865L, 60099L,
60100L, 60280L, 60384L, 60429L, 60493L, 60503L, 60603L, 60664L,
60846L, 61415L, 61749L, 61883L, 62081L, 62983L, 63327L, 63329L,
64418L, 64507L, 64596L, 65178L, 65250L, 65802L, 65975L, 65978L,
66396L, 66572L, 66589L, 74034L, 74427L, 74607L, 74952L, 75732L,
76574L, 76595L, 76755L, 76759L, 77203L, 77453L, 77668L, 81064L,
81065L), Gender = structure(c(1L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 1L, 2L, 2L, 1L, 2L,
2L, 1L, 2L, 1L, 1L, 2L, 2L, 1L, 1L, 1L, 2L, 2L, 2L, 1L, 1L, 2L,
2L, 1L, 2L, 1L, 1L, 1L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 2L, 1L, 1L, 1L, 2L, 1L, 2L, 1L,
1L, 1L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
1L, 2L, 2L, 1L, 2L, 1L, 2L, 2L, 1L, 2L, 2L, 1L, 2L, 1L, 1L, 2L,
2L, 1L, 1L, 1L, 2L, 2L, 2L, 1L, 1L, 2L, 2L, 1L, 2L, 1L, 1L, 1L,
2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L,
1L, 1L, 2L, 1L, 1L, 1L, 2L, 1L, 2L, 1L, 1L, 1L, 2L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 1L,
2L, 2L, 1L, 2L, 2L, 1L, 2L, 1L, 1L, 2L, 2L, 1L, 1L, 1L, 2L, 2L,
2L, 1L, 1L, 2L, 2L, 1L, 2L, 1L, 1L, 1L, 2L, 2L, 2L, 1L, 1L, 1L,
1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 2L, 1L, 1L, 1L,
2L, 1L, 2L, 1L, 1L), .Label = c("Male", "Female"), class = "factor"),
    Age = c(15L, 15L, 9L, 11L, 16L, 9L, 16L, 16L, 14L, 8L, 6L,
    14L, 10L, 15L, 13L, 15L, 8L, 9L, 9L, 8L, 9L, 9L, 13L, 10L,
    7L, 8L, 8L, 6L, 15L, 8L, 11L, 14L, 12L, 10L, 16L, 12L, 10L,
    6L, 13L, 11L, 12L, 13L, 10L, 13L, 14L, 12L, 17L, 9L, 12L,
    11L, 10L, 12L, 10L, 10L, 14L, 16L, 15L, 14L, 14L, 13L, 10L,
    12L, 9L, 9L, 16L, 10L, 14L, 15L, 13L, 15L, 13L, 13L, 8L,
    11L, 16L, 16L, 11L, 13L, 18L, 10L, 18L, 18L, 15L, 10L, 8L,
    15L, 12L, 16L, 14L, 16L, 9L, 11L, 11L, 10L, 10L, 11L, 14L,
    12L, 8L, 9L, 9L, 8L, 16L, 9L, 13L, 16L, 13L, 12L, 18L, 13L,
    11L, 8L, 14L, 12L, 13L, 14L, 11L, 14L, 15L, 14L, 18L, 11L,
    14L, 12L, 12L, 13L, 11L, 11L, 15L, 17L, 16L, 15L, 15L, 14L,
    11L, 13L, 10L, 11L, 18L, 11L, 15L, 16L, 14L, 17L, 14L, 14L,
    9L, 12L, 18L, 18L, 12L, 14L, 19L, 11L, 19L, 19L, 16L, 11L,
    9L, 17L, 13L, 18L, 16L, 18L, 11L, 12L, 12L, 11L, 11L, 12L,
    16L, 13L, 9L, 11L, 10L, 9L, 17L, 11L, 14L, 17L, 14L, 13L,
    19L, 15L, 12L, 9L, 15L, 14L, 14L, 15L, 12L, 16L, 17L, 15L,
    20L, 12L, 15L, 13L, 13L, 14L, 12L, 12L, 16L, 19L, 18L, 16L,
    16L, 15L, 12L, 14L, 11L, 11L, 19L, 12L, 16L, 17L, 15L, 18L,
    15L, 15L, 10L, 13L), Y= c(-3.91040964443843, -4.1357309415226,
    -3.15711439697149, -5.3057953682613, -5.64125379300701, -5.14501797062188,
    0.244858660867345, 0.0760011434854038, -1.38050716781916,
    -0.765398953185992, -1.47720169386702, -1.64309892713909,
    -0.396819316883117, -1.36975793945299, -1.80785572309015,
    -0.401914887891913, -0.811404680008261, -2.4492910546021,
    -0.620903118150862, -1.10304119407832, -0.683772758549858,
    -1.47159656575735, -0.823876696667884, 0.229547683217108,
    -1.91347963071532, -1.05594355918274, -2.37962731952471,
    -1.34893889218848, -0.385827442278431, -0.980092345023242,
    -0.181058281596406, -0.832975930612162, -1.9959308225624,
    -0.12787022611412, -0.205371434695515, -0.609353157197594,
    -0.477183751078979, -1.53245224237668, -0.137187841673061,
    -1.11978378453579, -1.38506891710323, -0.665622819909006,
    -0.858065551674517, -0.935493966384356, -0.444790478237349,
    -0.0888284463372063, 0.793433276899957, -1.37118955226023,
    -1.27752810419379, 0.571630350274245, 0.38391922015976, -0.585743678190174,
    -0.639562613892594, 0.637702921021628, 1.02283103079116,
    0.509051885361466, 0.256796855802238, -0.827467860997893,
    1.25436407158129, 0.797121499725372, -1.46470541258355, 0.72639012119852,
    -1.77912640845008, 1.31912635264069, 0.135328148802092, -0.522413009938002,
    0.680529982119362, 0.068624697834574, -0.598312753345204,
    0.141370040141094, -0.392378890718311, 1.88269650621348,
    -3.0968410713246, -0.0622101539722123, -4.1452184094485,
    -0.954638754603117, -0.106347504757922, -0.716821176235476,
    0.33910245990621, -1.65370256776216, 0.666505029533247, 1.11178514183042,
    0.0370564222038934, 0.0466166840013479, -0.557572449898689,
    -0.961796818639283, 0.936133529770083, -0.283139554139275,
    -0.111006312537392, 0.655416096433155, 0.253351279215336,
    -1.27117077274472, 0.821677299062997, 1.30061244464207, -0.571112110007776,
    -0.978927643078376, -0.0939968112175565, 1.01351341523222,
    -0.482667556069396, 0.416191169882131, -1.13218300732386,
    0.0109719515636322, 0.570902411558704, 0.251628490921888,
    0.757376045856767, -0.286245425992256, -0.850640576775985,
    0.323694423760568, 0.33667599752107, 0.489882832518856, 0.0192219236731108,
    -0.267852841112888, 1.81167395220041, -0.0196014744891418,
    -0.0975879755475657, 1.11193072957353, 1.43135023795345,
    -0.0901872652728838, 0.562191411596049, 0.0591857591563773,
    1.10166679368438, -0.140293713526042, 0.0486791770287176,
    0.34512008662136, 0.818328780971503, -0.238419852381131,
    -0.368114266866903, 1.03486628422146, 0.539091489689507,
    -0.233227222876931, -0.351614322647946, -0.108579850152251,
    1.42892377556831, 0.96726504217144, -0.436055213650844, 0.565831105173759,
    -1.54960733143962, 1.100259445501, -0.614861226811863, -0.361659876922429,
    1.40706134947819, -0.552573937385301, 0.105482661464863,
    -0.374326010572864, -0.0292830594058542, 1.98545718822419,
    -1.19221368673224, 0.760991474810628, -0.539568099000947,
    -0.0739542319162944, 0.34373700306183, 0.053071073945822,
    0.0606901658351647, -0.211219209043705, 0.216590374080456,
    1.00007081361854, -0.495649129829898, 0.0867746364754286,
    -1.67723925289802, -0.826836980777755, 1.18863120556783,
    -0.795681203752549, -0.650869928607352, 0.333545861044237,
    0.63103014946249, -1.75748236397463, -0.118601139802882,
    1.51474775013074, 0.132537717059181, -0.677973513449372,
    0.467947612557183, 1.56786301174147, -0.118528345931329,
    0.554596584330558, -0.59224659738235, -0.543498968064875,
    -0.380440695783417, 0.0643541240367275, -0.210830975062082,
    0.503471021875644, -0.660672836643319, 1.50676468888363,
    0.780839937121078, -0.116635705270918, -0.240385286913095,
    0.00044110481212036, 1.4571920623552, -0.350401091455376,
    0.0850033189342734, 0.197882349091022, 0.726511444317778,
    -0.331741595713643, 0.837497833814114, 0.609264781867777,
    0.532151807268005, 0.65446977610295, -0.581909867621652,
    0.322942220421174, 1.52993740466172, -0.264455793773691,
    -0.0568476721010507, 1.05299195823846, 0.915435805624833,
    0.297609953120304, -0.114257772133482, 0.248013061968027,
    1.64822744593733, 1.26800079018578, -0.0459528559917708,
    0.720784993088846, -1.05679282101754, 1.38345187047077, -0.64414862780051,
    -1.25411274217718, 0.177548594303543, -0.257734492966852,
    -0.818028922319694, 0.409736779937656, -0.216411838547905,
    1.32536236097051, -0.500938817829506, 0.803770006660658),
    Timepoint= c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
    1, 2, 2, 3, 3, 3, 2, 3, 3, 2, 3, 3, 2, 3, 2, 2, 2, 2, 3,
    3, 3, 2, 3, 2, 3, 2, 2, 2, 3, 2, 2, 3, 3, 2, 3, 3, 2, 2,
    3, 2, 2, 2, 2, 2, 2, 2, 3, 2, 3, 3, 2, 3, 2, 2, 2, 2, 2,
    2, 2, 2, 2, 2, 2, 2, 3, 3, 2, 2, 2, 2, 3, 2, 2, 2, 2, 4,
    4, 4, 4, 4, 3, 4, 4, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
    3, 4, 4, 4, 3, 4, 3, 4, 3, 4, 4, 4, 3, 4, 4, 4, 3, 4, 3,
    4, 3, 3, 3, 4, 4, 4, 4, 4, 4, 3, 4, 3, 3, 3, 3, 4, 4, 3,
    3, 3, 3, 3, 3, 3, 4, 3, 3, 3, 3, 4, 3, 3, 3, 3)), row.names = c(NA,
-222L), class = c("tbl_df", "tbl", "data.frame"))

Comment: Also, could you *please* post actual code; you are using `ID` and `subject` interchangeably now, while you had a `gender` vs `Gender` typo before. It's very difficult to provide concrete advice when what you post has inconsistencies like this - you wouldn't believe the number of questions we get where the problem is an overlooked typo.
Comment: You don't *need* a correlation structure but you can't ignore residual temporal autocorrelation which you almost surely have if you model `Timepoint` a linear parametric term as you did in the `gamm4` model you asking about. There are many ways to model the temporal dependence, the correlation structure in the residuals (GLS style) or working correlation matrix (GEE style) are but two. The other would be to use flexible enough smooths per subject to model the temporal dependence directly. Whether you can do any of these depends on the data, etc
Comment: In what sense is it not working? Apart from the obvious problem that there is no way that figure can be produced using `smooth_estimates()` as smooths in {mgcv} are (typically) centred and hence will span positive and negative values. Is that also a typo: `gender` vs `by = Gender`?
Comment: I updated the question. The original plot is just a sample, sorry for the confusion, my plot does span negative values. I don't think the `by = gender` is a typo, but maybe an inaccuracy in the model?
Comment: You don't have `gender` in your dataset, only `Gender` so the code will produce an error if you're using `gender` in the formula.
Comment: Sorry for the misunderstanding, I thought you were asking about the `by` function, not the spelling, yes that's a typo, I will fix that
Comment: You are assuming a compound symmetric correlation structure, which ignores the forward flow of time.   Think about using a serial correlation structure instead, and using a Markov process or generalized least squares.
Comment: @FrankHarrell is this possible with `gam`, using something like `corAR1(form = ~ Timepoint)`?
Comment: @CanyonView for `gamm()` *only* you can use use `corAR1()` or `corARMA()` *if* your data are sampled regularly in time. If they aren't you can use `corCAR1()`. For `gam()` and `gamm4::gamm4()` you are out of luck. You could use `bam()` with *known* (or a priori estimated outside `bam()`) value of $\rho$ for an AR(1) process in the residuals.
Comment: I'm not sure that I need `corAR1`. My data is similar to this post: https://stats.stackexchange.com/questions/459200/syntax-of-gam-longitudinal-dataset I am just confused why timepoint is not included that model.
Answer: The first model fails because you ask for too many coefficients; you ask for Subject specific intercepts twice (!) and try to fit 9 basis functions per subject in the s(Timepoint, Subject, bs = "fs") term.

The "fs" smooth of Timepoint by Subject already contains a random intercept per Subject: you don't need to add another separate random intercept via the s(Subject, bs = "re") term. You're just trying to estimate an extra nlevels(Subject) parameters that are confounded with the "fs" terms random intercept parameters, and nlevels(Subject) of these parameters are going to be shrunk to 0.
Whenever you include "fs" smooths, you typically don't need a separate random intercept for the grouping variable of the "fs" smooth.

Looking at your plot, in many case you <<9 observations per subject. Unless you have >>9 observations for other subjects you can't estimate 9 parameters per subject; you don't have the data to support such rich smooths.

The s(Age, by = Gender, bs = "fs") really isn't doing what you think it's doing. There's no grouping variable here (so this should arguably raise an error). This just needs to be s(Age, by = Gender).

You compound the problem from 1. by adding another nlevels(Subject) - 1 parameters for the subject term by estimated parametric effects. While we say you "have" to include the by variable as parametric effect, typically what we mean is you need to have something in the model for the group means of the factor by term(s). The random intercept in either the s(Timepoint, Subject, bs = "fs") smooth or the s(Subject, bs = "re") smooth would account for these group means of Subject; you don't need yet more terms.

Get rid of Subject from the parametric effects, drop the Subject random intercept, and think through how many parameters you can afford to expend on the subject-specific time curves (in the "fs" Timepoint` smooth).
Why do you estimate time curves by subjects as having the same wiggliness but age by subject curves are each allowed their own wiggliness? You now have to choose values for nlevels(Subject) extra smoothness parameters. It also doesn't make much philosophical sense to treat one as essentially random and the other fixed for the same grouping variable.
A much simpler model for the first case would be:
m1 <- gam(Y ~ Gender + s(Age, by = Gender) +
            s(Timepoint, Subject, bs = "fs", k = 10),
          data = DF,
          method = "REML")

and shift k smaller if you can't fit even this simpler model.
If you want subject-specific Age effects, you could add that to the model as so:
m1a <- gam(Y ~ Gender + s(Age, by = Gender) +
             s(Timepoint, Subject, bs = "fs", k = 10) +
             s(Age, Subject, bs = "fs"),
           data = DF,
           method = "REML")

but at this point we probably need to start thinking more carefully as you'd want to be sure you aren't estimating another nlevels(Subject) random intercepts and check what identifiability constraints are being applied here.
With the exception of considering an auto correlated process in the model (Frank's suggestion), a GAM such as the one I show above would be a reasonable starting point for modelling longitudinal data via a GAM.
You're never going to get a plot like the one you showed from the paper by using smooth_estimates(). This is evaluating the partial effect of each smooth, while the plot from the paper seems to be on the response scale (so it would include the intercept). You could achieve this, kind of, by just adding the correct parametric effects to the est column (and the upper and lower intervals), but you'd have to get the right parametric effects added to the right rows, and you would have too-narrow an interval because you aren't really including the uncertainties in those parametric effects into the std. error.
Beyond that, you can't just pass the object of smooth_estimates() to ggplot() and expect it to do anything useful. You returned values for all smooths in your model in the object sm, so any row with an Age value that isn't NA will end up getting plotted somehow.
Better would be to get predictions from your model for the scenarios you want. To do that you would be forced to write down what you actually want to achieve, not just tell us what you want it to look like that image (I'm not reading a paper's methods section just to intuit what you want to show and what the authors of that paper did to generate their plot).
You seem to want to plot estimates of Age by Gender at the population level. So you will want something like:
new_df <- with(df,
    expand.grid(Subject = factor(levels(Subject)[1], levels = levels(Subject)),
                Age = gratia::seq_min_max(Age, n = 100),
                Gender = factor(levels(Gender), levels = levels(Gender)),
                Timepoint = median(Timepoint)))

where you are explicitly conditioning on the Age and Gender values, and we'll exclude the random smooth effects due to Subject and Timepoint and Age from my m1a (assuming you can fit that) by use of the exclude argument to predict.gam().
Top get the predicted values use gratia::fitted_values() unless you want to do everything by hand yourself, in which case use predict(m1a, new data = new_df, exclude = ...., se.fit = TRUE) etc with exclude set as below:
fv <- fitted_values(m1a, data = new_df,
                    exclude = c("s(Timepoint,Subject)",
                                "s(Age,Subject)"))

where the things we pass to exclude are the labels of the smooths that relate to Subject level that we want to ignore as you seem to want population level effects. You can get these labels from gratia::smooths(m1a) if you don't know how {mgcv} creates labels for it's smooths.
At this point you might want to rethink your model structure a bit to include population (Pedersen et al 2019 PeerJ call these "global" smooths) effects for Timepoint as the only thing account for Timepoint is a random smooth term. Even so, you'd still have to condition the predictions on the specified value of Timepoint (here I use the median time point but you can plug in whatever value works for you).
Your gamm4() makes similar mistakes (the bs = "fs" is redundant on the s(Age, by = Gender, bs = "fs") smooth and why do you now model time as a linear parametric effect of Timepoint when in the gam you had subject-specific random smooths of Timepoint for these effects? If you wanted this in the gam() why not the gamm4()?
These models are so flexible and there are so many options, one needs to restrain oneself from trying everything in a model because you think you need it. Think about the model (effects) you want fit, write it down in words, bullet point form. Then write out a model formula in R's notation that you think covers each bullet point. Then post that in your question if you want to know if this is a suitable model for your data.
Comment: Gavin would you mind expanding a bit on what correlation pattern is assumed by your `m1a`?
Comment: @FrankHarrell None, the observations conditional upon the model are assumed to be independent; it making the assumption that the curves from the `s(Timepoint, Subject, bs = "fs")` are sufficiently flexible to capture the temporal correlation in the observations. If they aren't one would need something additional to model the autocorrelation. This starts to get complicated though as a wiggly smooth can be viewed as autocorrelation so including both in the model can lead to identifiability issues. Seeing as the OP has so many other issues with their models `m1a` is a reasonable start.
Comment: Thanks Gavin.  Is this modeling subjects as fixed effects?  I can see that this would take care of certain correlation patterns, but requires estimation of a large number of parameters.
Comment: Is what modelling subjects? Frank is asking about correlation structures in the sense of his GLS/Markov process comment. Your model includes subject as a fixed effect which is redundant given the subject random intercepts (of which you have two lots) as per my post.


Postfix calculation in smalltalk

Question: I have to write a code to evaluate a postfix notation(reverse Polish evaluation) in Smalltalk. I have gone through the documentation and have also implemented a stack. This is the code I have written so far:
Object subclass: #Rpcalc
instanceVariableNames: 'anArray top'
classVariableNames: ''
poolDictionaries: ''
category: nil !

pop:
    | item |
item := anArray at: top.
top := top - 1.
^item!

push: item
 top := top + 1.
 anArray at: top put: item!

setsize: n
  anArray := Array new: n.
  top := 0!

evaluate:
       | expression aToken op1 op2 operator answer|
   Transcript show: 'Enter Expression' .
   expression :- stdin nextLine.

   | aStack |
   aStack := Array new: 10 .

   aToken := self getNextToken.
       ((aToken key) = 'operand')
       ifTrue: [push : aToken].

        aToken := self getNextToken.
       ((aToken key) = 'operator')
        ifTrue:  [op1 := pop.
           op2 := pop.
           operator := aToken.
   "if(operator := +)"
   "answer := op1 + op2"

I want to know how to tokenize each element in the expression. For e.g.,
for the expression,
    10 3 + 3 7 * -
I need to equate it to a token. If its an operand, it should push it into the stack. If an operator, pop the stack twice to get the operands and evaluate the expression. I am totally new to smalltalk, so I am clueless about the syntax.
Answer: You did not specify which Smalltalk dialect you are using. In Squeak, you could use the findTokens: method:
'336 8 4 2 1 + - * /' findTokens: ' '
==> an OrderedCollection('336' '8' '4' '2' '1' '+' '-' '*' '/')

Use isDigit to test if a token is a number:
'336' first isDigit
==> true
'+' first isDigit
==> false

To convert from a string to a number you would use asNumber:
'336' asNumber
==> 336

The whole RPN parser/evaluator can easily be implemented with less than 10 lines of code, but obviously that is your homework to do (hint: no need to implement a stack, there is already one).
Answer: I suggest you taking a look at the PetitParser. You can parse the expression you have: 10 3 + 3 7 * into a tokens, like: NumberToken(10), OperatorTocken(+) and then depending on the token you do what you need to. Also don't do
operator = '+' ifTrue: [op1 + op2]

do:
op1 perform: operator with: op2

instead,
Comment: Using `#perform:` is something you should only do if (a) you're an experienced Smalltalker and (b) you absolutely have to. It ruins introspection, and can easily make your life hell. For an interpreter like this I would say _don't_ use `#perform:with:`. With the explicit check you can see at a glance the supported operations.

Postfix calculation in smalltalk


Which correlation method to us?

Question: I am debating whether to use Pearson's correlation or Kendall's tau on a set of data. While linear relationships among subsets of the data are reasonable, I am deeply skeptical that they exist throughout the data, so I am reluctant to use Pearson's correlation. Also, while I've read on Research Gate that Kendall's tau is preferable to Spearman's rho for smaller data samples (like mine), I also read that it's used generally for ordinal data -- which brings me back to Pearson's metric.
Keeping in mind that the results from the "winning" method will be inputted into a copula that will be involved in a week-long simulation, I'd like to get it right the first time, and was hoping your advice could steer me in the right direction.
Thank you!
Comment: similar question here, https://datascience.stackexchange.com/questions/64260/pearson-vs-spearman-vs-kendall
Comment: Thank you! Didn't pop up when I was searching for an answer on Google. My challenge is that my data are not necessarily independent. For example: I have monthly hydro-generation data for weekdays from 7am to 10pm and for weekdays from 11pm through 7am. I would imagine that river flow at night is partly determined by river flow during the day and vice versa — but I need to keep the two separate, as the first set of generation during the first peak trades at different prices than that on the second peak. Any thoughts?
Answer: Correlation coefficient is not robust enough to say there's a relationship or not, as it's somehow just care about a small range of relationships, such (linear relationship) in case of Pearson's correlation coefficient.

In time-series data, the correlation coefficient is useless as there's a complex relation (i.e. there's no linear nor ranking relationship), you could try Arima model instead.
Comment: There's a lot of ways you could go through, you could try statistical test with a time-shifting, you could try to compute the correlation coefficient for weighted average - compute weighted average, then shift data and split into k-groups, compute correlation coefficient for each group separately. (I guess it's a terrible approximation, but i believe it's quite good ), but  a statistical test is the best way, consider using **ADF Test** too.
Comment: In case of you really want use correlation directly, then I guess Spearman or Kendall's Tau, over k-subset of the data set (i.e. shift data, selection based window size), is the way to find least reasonable results. The idea behind data splitting, is trying to approximate data to (n-lines or monotonic function), then compute correlation coefficient.
Comment: Thing is, I need to somehow capture the relationship among variables, in order to generate my copulas. Is there an alternative to a correlation matrix of which I'm not aware?

Which correlation method to us?


Erro ao tentar atribuir um novo valor de um objeto de repetição

Question: Preciso atribuir ou mesmo criar um novo atributo em um objeto dentro de um laço, porém está dando o seguinte erro, que não faço ideia do que seja:

TypeError: Cannot create property 'is_content_hide' on boolean 'false'

EmailService.setMail(data)
     .then(function(result){
         if (result.status) {
            if (angular.isDefined(result.em_massa) && result.em_massa) {
                for (var i in $scope.content.dados) {
                    if ($scope.content.dados[i]) {
                       $scope.content.dados[j].is_content_hide = true;
                    }
                }
            }
         }
     });

Answer: Você esta comparando se existe o if ($scope.content.dados[i]) utilizando a variavel i mas atribui o valor utilizando a variavel j


Why is -rf used?

Question: So I'm learning the linux command line and I come across in the book I'm reading the definition for -rf.
I've always seen -rf used when people rm -rf is explained as
"Same as above, except that if either file1 or dir1 do not exist, rm will continue silently."
So is all -rf doing is continuing on with deleting something if it doesn't exist?
Why do people always use this then when doing commands?
Comment: Those are Linux command line options, specifically for `rm` but other commands may have them as well. I suggest reading `man rm`. In this case, for `rm`, the `r` means "recursive" (it will remove files and all subdirectories if the given item is a directory), and the `f` means "no flack" (it won't complain if it's read-only or whatever, it will just try to remove it without asking you). The `mv` command also allows `f`.
Comment: '-rf' are options specific to the 'rm' command.  The 'r' means recursive.  When you're deleting directories, you must provide the '-r' option to delete things recursively (delete all of its contents).  The 'f' option stands for 'force'.  If you run the command "rm file1 file2 file3 file4..." and file2 doesn't exist, the rm function will complain and not remove anything.  You can ignore that warning with the 'f'.
Answer: For the "rm" command anyway,
-f tells the command not to prompt at each file
-r tells the command to recurse through subdirectories
As mbratch says, the man page ("man rm") tells the whole story.
Answer: while "rm" just removes a file rm -rf removes file(s) that are within a directory. So for example. if your current location is $HOME and you would like to delete everything in $HOME/robert/files you could simply do rm -rf robert/files/* if you are not a privileged user you would have to use sudo in front of your command so it would look like "sudo rm -rf robert/files/*"

Why is -rf used?


Nested resources Rails

Question: criei dois models na aplicação Rails, e fiz o relacionamento deles através da declaração nas classes e no banco também.
Adicionei a configuração para ativar o nested resources no arquivo routes, as rotas foram criadas porém não trabalham da forma como deveriam, vou colocar o exemplo abaixo:
Model Project
has_many :steps

Model Steps
belongs_to :project

resources :projects  do
    resources :steps
end

Porém, quando eu bato a url /projects/1/steps ele sempre cai na ação index do controller Steps, que retorna um Steps.all.
A forma correta de fazer isso, é só reescrevendo a ação index mesmo??
Comment: Rapaz, ficou confuso... você definiu a action `index` do Steps? Ou tá perguntando se tem que definir...
Answer: O que está acontecendo é exatamente o que deveria.
Com o comando rake routes é possível ver o seguinte resultado:
       Prefix Verb   URI Pattern                                    Controller#Action
project_steps GET    /projects/:project_id/steps(.:format)          steps#index

Ou seja, sua URL /projects/:project_id/steps está apontando para a controller stepse a action index.
Caso você queira direcionar para outra ação específica que não a index, você pode criar uma rota para ela como:
resources :projects  do
  resources :steps do
    get :minha_acao, on: :collection
  end
end

Que irá responder a:
/projects/:project_id/steps/minha_acao

Se você trocar collection por member a URL será:
/projects/:project_id/steps/:step_id/minha_acao

Note também que o verbo criado foi o GET, mas poderia também ser POST, PUT ou DELETE, dependendo da sua necessidade.

Nested resources Rails


The productrelation matrix is equivalent to the product of the matrix of the relations

Question: I didn't get this thing. Let $R, S$ be two relations from $A \to A$ with $A$ being an arbitrary set. And $M_R$ and $M_S$ their relation matrixes defined as:
$(M_R)_{ij}=\left\{
  \begin{array}{l l}
    1 & \quad \text{if $iRj$}\\
    0 & \quad \text{otherwise}\\
  \end{array} \right.$
$M_S$ and $M_{R \circ S}$ are defined analogue. Why it holds that $M_{R \circ S}=M_R \cdot M_S$ (no, not exactly, but I don't know how I can express this in a nice way)? This is my incomplete sort of proof. Can you give me some advice how to improve it?
Now $iR \circ Sj \Leftrightarrow \exists x : iRx \wedge xSj$.
Suppose $\exists x : iRx \wedge xSj \Rightarrow (M_R)_{ix}=1, (M_S)_{xj}=1 \Rightarrow (M_R)_{ix} \cdot (M_S)_{xj} = 1$.
On the other hand, suppose $\neg(\exists x : iRx \wedge xSj) \Leftrightarrow \forall x: \neg iRx \vee \neg xSj \Rightarrow (M_R)_{ix} = 0 \vee (M_S)_{xj} = 0 \Rightarrow (M_R)_{ix} \cdot (M_S)_{xj} = 0$.
Therefor, if we take the product it will sum up all those things. If there exists such an $x$ for any $i,j$, there will turn up a number greater than $0$. Otherwise, if there does not exists such a number it will be a $0$. If we replace all those integers greater than $0$ by $1$, it is the matrix of the productrelation $R \circ S$.
Comment: Can you be more specific about the relation (?) you denote by "R;S"? This is not standard notation as far as I know.
Comment: Ah, I learned it as being standard notation. But more common is I guess $R \circ S$. It are arbitrary relations, from a arbitrary set A to itself.
Answer: Your proof is fine.  In case it helps, here is how I would have written it up.  To simplify notation, let's assume that $A$ is the set $\{0,1,\ldots,n-1\}$.
We have $(M_R M_S)_{i,j} = \sum_{k<n} (M_R)_{i,k} (M_S)_{k,j}$ and because $(M_R)_{i,k}$ and $(M_S)_{k,j}$ are always zero or one, this sum is greater than zero if and only if there is a $k<n$ such that both $(M_R)_{i,k}$ and $(M_S)_{k,j}$ are equal to one.
In other words, if and only if there is a $k< n$ such that $i \mathbin{R} k$ and $k \mathbin{R} j$. This is equivalent to $i \mathbin{(R \circ S)} j$, which in turn is equivalent to $(M_{R \circ S})_{i,j} = 1$.
I don't know if there is standard terminology for describing two matrices such that one is obtained from the other by replacing all the nonzero entries with ones, by the way.  I have never heard "equivalent" used for this.  It is probably best just to say explicitly what you mean, that is, $(M_{R \circ S})_{i,j} = 0$ if and only if $(M_R M_S)_{i,j} = 0$, and $(M_{R \circ S})_{i,j} = 1$ if and only if $(M_R M_S)_{i,j} > 0$.


Reading AWS EC2 tag from Code Deploy hook during Auto Scaling scale-in event

Question: I'm really enjoying working with AWS Code Deploy integrated with Auto Scaling, but am struggling with an issue around reading tags during a scale-in event.
I have setup my Auto Scaling Group to tag any new instances created with a tag name 'Environment'.  Then as part of my Code Deploy after-install script I read the tag and use it to conditionally configure my apps. This works perfectly if I deploy a revision to existing instances in an auto scale group. However during a scale-in event it seems that the tags are not assigned to the new EC2 instance before Code Deploy is called from Auto Scaling, so my after-install configuration fails.
Any ideas on a way around this? Can I force the tags to be associated with the EC2 instance earlier in the pipeline?
Answer:
Then as part of my Code Deploy after-install script I read the tag and
  use it to conditionally configure my apps

Instead of using tags you can try using the environment variables exposed by CodeDeploy during the deployment lifecycle events. Please check out https://blogs.aws.amazon.com/application-management/post/Tx1PX2XMPLYPULD/Using-CodeDeploy-Environment-Variables for more information.
Thanks,
Surya.
Answer: You have to add custom life cycle hook in your auto-scaling group and work with them within CodeDeploy.
Remember that the order of hook execution is not predictive.
Comment: Unless you are suggesting that I add a custom lifecycle hook to Auto Scaling which adds the tags before CodeDeploy is called .... but as you said order of execution is not predictive so that wouldn't be reliable.
Comment: From my tests it seems that Auto Scaling only adds the tags to the new EC2 instances AFTER Code Deploy has completed its actions, so I don't see how any custom life cycle hooks will help?
Comment: @rossh - Correct.  Tags are not created until the ReadyToInstallSofware lifecycle hook is completed/continued. Not sure why that is. I makes a tag based CodeDeploy impossible
Answer: We had the exact same issue regarding CodeDeploy and custom lifecycle hooks.  The tags were not generated until AFTER the lifecycle completed, which was too late as we wanted to do a CodeDeploy during the time the lifecycle was pending.
Our solution was to build a userdata script what would allow the instance to tag itself at launch time.  The script is installed into each AMI, and accepts two parameters: Environment and Function.
<code><script>
PowerShell -ExecutionPolicy Bypass -NoProfile -File c:\tools\server_userdata.ps1 --function Reg -environment production
</script>
</code>
So we can attach that userdata to a Launch Configuration, and the instance tags itself on launch.  As soon as the instance is stable, but not yet through its lifecycle we can do a CodeDeploy successfully.
We had to use a Role policy to give the instance permission to list and create tags:
<code>{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeTags",
                "ec2:CreateTags",
                "ec2:DeleteTags"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
</code>
The userdata script is generic and parameter driven, so we can launch any instance with any environment and function setting. The same approach would work in Linux of course.
This approach solved the problem you are describing.
Comment: Thanks Rodrigo ... will try this approach


Do you know a flash barcode scanner?

Question: For my web project i require a flash barcode scanner.
I did some research but didn't find any usable software.
Do you know an asset written in flash to scan barcodes what i can include into a html document and access with JavaScript?
Answer: Just googled a bit.
http://zxing.riaforge.org/
It's a flex project, but I think you can easily embed the generated SWF or AS3 file into your flash project.
Comment: tanks. i dont know anything about flash programming. so i didn't search for actionscript libraries.

Do you know a flash barcode scanner?


Is there a closed-form expression for $\int (a-b\ln(cx))^{-1} \mathrm{d}x$?

Question: Is there a closed form expression for
$$\int\frac{1}{a-b\ln(cx)}\,\mathrm dx\ ?$$
I was wondering how to integrate the above function. I have spent a lot of time on it. First i did an integration by parts until I had to integrate $\ln|a – b \ln(c x)|$ again. Once more, I did an integration by parts and substituted this value back into the original equation. However, now all the terms cancel each other out, effectively giving $0 = 0$. How do I go about integrating $1/(a-b \ln(cx))$? Thanks for the assistance.
Edit: $a$, $b$ and $c$ are constants.
Comment: integrals.wolfram.com suggests that this doesn't have a closed form: it gives an answer in terms of the exponential integral (http://en.wikipedia.org/wiki/Exponential_integral).
Comment: [Read this first](http://en.wikipedia.org/wiki/Logarithmic_integral_function)
Comment: Please make the bodies of your posts self-contained.
Comment: @Arturo: That may be my fault. The original title was: `integrate 1/(a-b*ln(cx)) with respect to x` which did not seem descriptive of what the OP appeared to be asking in the question. Sorry if my edit resulted in *more* confusion rather than less.
Comment: @cardinal: It wasn't your fault, because the original post did not include the integral he wanted to integrate in the body either, relying instead on information on the subject line. As I've said many times before: think of the subject line like the spine of a book containing its title. What would you think of a book that asks you to go look at the spine to get some basic information without which the text is incomprehensible?
Answer: There is no elementary antiderivative.  Note that $a - b \ln(c x) = -b \ln(c e^{-a/b} x)$ (assuming $a$, $b$, $c$, $x$ positive), so (after a change of variables) it all reduces to the case $a=0$, $b=c=1$. You can also prove that
the integral is non-elementary using the Rothstein-Trager theorem.
Comment: For completeness: the nonelementary integral you'll bump into is $\int\frac{\exp\;u}{u}\mathrm du$, which is the exponential integral $\mathrm{ei}(x)$ that Qiaochu mentions.


Flowchart always shows all nodes collapsed, any way to expand them?

Question: We are using a flowchart activity as base for our workflow. All the activities that I drop on the designer surface of the flowchart are shown in collapsed state. Even if I click on "Expand all" in the top right of the designer.
First I thought it's a problem with my activity designer code, but the same holds true for standard activities like Sequence: They can not be shown in expanded mode when inside a flowchart. They do not show their regular Expand/Collapse icon in the top right, too.
A little sequence inside another sequence is expandable:

But inside the flowchart, the same sequence is not:

Is that by design, intentionally, or am I just missing some very basic piece? I would like to expand individual activities (we have a lot with elaborated designers), but at the moment this seems to work only in Sequence-like base activities, not in flowchart.
Answer: I think it will be so complicated if your Flow chart has a lot of items so they prefer levels which you can see it in the navigation menu when you hit double click  to the Sequence Item
and also the Same if you have Sequence WF and you added Flow Chart item will be the Same
Comment: Might be the reason, yes. Basically you say that it's by Design and no way to escape, right? Sometimes it would be pretty helpful  to expand all and printout a Big Poster, then post it to the Wall of the Teamroom...


Prove that $\bigcup_{U \in \mathfrak{F}} U$ is a subspace of $V.$

Question: Let $V$ be a vector space, $\mathfrak{F}$ a collection of subspaces of $V$ with the following property: If $X,Y \in \mathfrak{F},$ then there exists a $Z \in \mathfrak{F}$ such that $X \cup Y \subset Z.$ Prove that $\bigcup_{U \in \mathfrak{F}} U$ is a subspace of $V.$
I'm having trouble starting this problem, some hints to get me started and nudge me in the right direction (no full answers please) would be appreciated. Thank you!
Comment: Quibble: $\mathfrak{F}$ has to be nonempty, otherwise the union is empty, and $0 \notin \emptyset$ :)
Answer: Name $U^\prime=\bigcup_{U \in \mathfrak{F}} U$.
For $x,y \in U^\prime$, $x$ and $y$ belong to subspaces $x \in X$ and $y \in Y$ with $X,Y$ subspaces of $U^\prime$. By hypothesis, there is a subspace $Z$ such that $X \cup Y \subset Z \subset U^\prime$.
As $Z$ is a subspace, $x+y \in Z \subset U^\prime$.
You can proceed in a similar way to prove that if a vector belongs to $U^\prime$, then it is also the case for its multiplication by a scalar. This allows to conclude as desired that $U^\prime$ is a subspace.
Comment: Interestingly, any union of subspaces would be closed under scalar multiplication, and they'd all contain $0$. It seems the hypothesis that the collection is directed is only needed to handle the binary operation $+$.


Confused as to why one scale fits four different chords

Question: I'm learning the bass guitar. I'm looking at a song called "Bling Thing" in the Rockschool Grade 4 book. On YouTube in the bass guitar backing track this part begins at 0:48.

In this piece of music, there are four bars which are to be improvised.
The piece is in the key of E minor.
The chords are Em Am D7 B7
My music teacher said that with the exception of one note of B7 I could just play the E minor scale and it would fit all four chords.
I am baffled as to how this could be possible, surely they have different notes in them? For example, E minor contains E F# G A B C D but A minor does not have F#.
But when I play just the E minor scale up and down through the whole part it sounds OK to me!
This revelation came at the end of the lesson and there wasn't time to go into a lot of detail, but his explanation kinda overwhelmed me, so I will ask him to go into it more, but explaining the theory and getting my head around it is hard work that I would rather do away from the lesson as much as possible.
I wonder if anyone could explain or direct me to the relevant topic so I can begin to get my head around all of the theory for this. I play classical piano which never touched on chords and scales beyond rote memorisation, so some parts I am strong and some basic things I am weak at as I try to pick up this second instrument.
Comment: Aaron nailed it as usual.  My contribution: I'd refine what your teacher is saying by adding one more criterion-- you are mainly moving by STEP.  If you move by step over a scale run, then every note is either a chord tone, or leads to a chord tone or from one.  If you were playing D chord and ONLY played the notes of an a-minor chord, that might sound strange.  But if you play a scale run from A up to E and back again, it will sound fine.
Comment: To simplify things still further: The very first, most basic way to approach this song is to play the notes E, A, D, and B over and over, those being the "roots" of the chords. Your teacher is talking about what you can do when that becomes musically unsatisfying; you can elaborate by including other notes, especially other members of each chord.
Comment: @AndyBonner Dm doesn't have an E in it though
Comment: When did anyone mention Dm? What does that have to do with the question? If there is information missing from the question, perhaps you could edit it to include the missing parts.
Answer: You seem to have confused the concepts of (1) chord, (2) scale, and (3) key. Your teacher gave you the E minor scale because of the E minor key, not because of the E minor chord. :) Confusing? Let's see:
The E minor chord, abbreviated as Em, has three notes: E, G and B.
The "E minor scale"... you probably mean the E natural minor scale. It has the notes E, F#, G, A, B, C and D. There are other minor scales like melodic, harmonic and jazz minor. Scales are just helper grids to help you reason about things.
The key of E minor means that the tonic chord i.e. the home chord is the E minor chord. If a song is said to be "in E minor", it just means that the E note is a central pitch in the song's harmony like the number 0 is a center in mathematics, and around that central E, notes are played so that an E minor chord would feel like a natural ending or resting state of that song. Because the notes of the E natural minor are so widely used in songs in the key of E minor, a key signature of one sharp (F#) is used in songs in the key of E minor (and its relative major key, G major). The E natural minor scale is a default scale in the key of E minor. Deviations from the default scale are marked with accidentals in staff notation. I assume you've seen accidentals and key signatures when playing piano.
Being in the key of E minor doesn't dictate any note or chord anywhere in the song, even though certain chords tend to be used very often. You can have a chord progression like Em - Am - B7 - Em - Dm6 - E7 - Am - F#7 - B7 - Em6, in a song that's in E minor, even though some of those chords contain notes that cannot be found in the E natural minor scale, or even any E minor scale at all.
The E natural minor scale is just a helper grid. Basic safe default pitches that are often used in songs in the key of Em. For technically oriented people, it could be called the factory default setting of notes for a song in E minor. But harmonic progressions can make chromatic alterations, like in the case of the B7 chord, which has a D# note. That means that the D note has to be made sharp, at least for the duration of that chord. In staff notation, this would be marked with an accidental.
But once more. A scale is a pitch grid that helps you locate cultural default expectations and often-used pitches. But a scale doesn't "sound" and it's not a law of nature. If the backing track plays a plain Em chord, you can play a loud and clear C# note over it with your melody instrument, and then the sounding total chord becomes Em6: E, G, B, C#. You changed the harmony. If you keep playing the C# for a long time, then it can even become an expected note. But if you then play a C natural instead, it will change the situation. But these changes or chromatic alterations do not change the key, if they do not change your feeling of what chord is a home chord. As long the E minor chord feels like home, you're in the key of E minor, regardless of what pitches the air is filled with.
In my opinion, to really learn how note combinations interact in harmony, you have to play chords by ear and try to reproduce the chord progressions you hear. For example, if you play a B major chord or a B minor chord in a song that's in the key of E minor, how it changes the feeling. You have to try both alternatives and learn to feel the difference.
The question "what notes fit where" is often answered with rules-of-thumb like "use this scale", for simplicity and to get beginners something to play that sounds sensible and isn't discouraging. I think that's what your teacher did. You're now supposed to just play notes from the scale and listen to what they sound like over different chords, without over-thinking it. Play and listen. Playing and listening is 90% of learning music. You learn by doing, not by thinking. When the time comes, you will encounter more concepts and different things to play and listen to. The more you've played and listened, the more you'll get out of the next lesson.
Comment: I saw a great video of Victor Wooten explaining the concept of a "wrong note", or rather explaining the concept away. The gist was something along the lines of "how wrong a note is or whether it is wrong at all, depends on what note you play after".
Answer: Chord tones and not chord tones
When improvising against any chord, the "primary" notes are those in the chord itself. Other notes are more or less decorative: adding color, passing from one chord note to another, or serving as embellishments to a particular note.
An example of chord tones and not chord notes and how to use them
Take an A minor chord as an example. It contains the pitches A, C, and E. In the abstract, when improvising and wanting to move from A to C, I could play A B C or A Bb C, among other possibilities. Which "in-between" notes I choose is heavily influenced by the overall key of the piece of music. So, when the piece is in E minor, I would likely choose A B C, because the B is part of the larger context: E minor. But for a piece in F major, I would more likely play A Bb C for the same reason.
Why can a chord work with a scale of a different name?
Any key/scale encompasses many chords. E minor, for example, contains the following triads (three-note chords):

       E F# G A B C D E ...
Em     E    G   B
F#dim    F#   A   C
GM          G   B   D
Am            A   C   E
Bm              B   D   F#
CM                C   E    G
DM                  D   F#   A  (C) ← (D7)

Thus, when a piece is in E minor and a sequence of these chords is encountered — such as Em Am D7 — one can "just play E minor, knowing that the chord tones will be contained within that scale, as will the intermediate tones.
Comment: If you play a so-called non-chord tone prominently enough, the chord becomes a different chord and then the note is a chord tone! :) After years of watching these "what notes work" questions online, I'm inclined to think that soloists are being taught wrong. They should be taught to provide the chords as well - maybe even as a first priority. If someone else plays backing chords, it just means that the soloist doesn't have to do everything, so there's less work, but on the other hand they're a bit  restricted with regards to the total harmony they can create.
Answer: Let's look at this a different way.
You are thinking that the notes from one scale maybe won't fit four different chords. How about if notes from that same scale won't even fit the root chord in question?
Key is Em. Play the 2nd, 3rd, 4th and 5th, or 4th, 5th, 6th and 7th - all in simple order, on the 4 beats, over the chord of Em. You may agree that they don't work too well either. But they're all notes from the very scale, over the very chord in question!
One of the main things a bassist does, in general terms, is to outline the prevalent chord. To do that, the chordtones get played, and they sound more convincing when they happen on particuar points in a bar. Namely the 1st and 3rd beats, speaking broadly. When you did what I suggested in para.2, neither of those were met.
Yes, you could just play up and down a scale - in fact, were it the pentatonic minor scale, it would work quite well. As proven by many a beginner guiarist/bassist. That's an improvement note-choice-wise, on the whole scale, as it omits two 'iffy' notes.
However, to play something which fits the sequence better, and sounds more musical, playing 1 on 1 is a good (often expected) place to start. So, if you played in a scalar manner (going sequentially up the scale, note by note), starting each bar on the appropriate scale note, it will start to sound like you know what's going on. On an Em bar, play E F♯ G A, on a D7 bar, play D E F♯ G, on an Am bar play A B C D, etc. Note that all the notes are still from that Em scale. And E natural minor will be fine.
I'm keeping it simple so far, so using notes 1 3 5 poss. 3 again), instead is another way. They fit perfectly, as they outline the notes from the triad of each chord.
There are going to be many more options than merely playing straight 4 in the bar, but the above tenets will still be the bare bones of what gets played. So when you said you do use the Em scale notes they fit, I say yes - and no. Those notes are the ones most likely to fit and sound o.k. Why wouldn't they? They all belong to the key of E minor - tha's why we put them all in the same scale - but rest assured that other notes may fit better. As you rightly say, A minor doesn't have F♯. (It actually can have, but that's for another day!). But since the piece is in key Em, the notes from that key will always work best.
A bit like talking to someone in English, and suddenly throwing in a Swahili word - often going to throw the listener!
Of course there will be times when a note that's out of key will work best, but at grade IV level, I wouldn't be too bothered about that yet. As an aside, have a look at RGT bass grades - they have a different take on things, which may shine some lights on queries you may have.


Navigation Link in bar Items goes back to top of NavigationView

Question: struct Testing: View {

    var body: some View {
        NavigationView{
            VStack {
                Text("View 1")
                Text("View 1.3")

                NavigationLink(destination: TestView(),
                                    label: {
                                        Text("View 1 navigation")
                                    })
            }
        }
    }
}

struct TestView: View {

    var body: some View {
            VStack {
                Text("View 2")
            }
            .navigationBarItems(trailing:NavigationLink(
                                    destination: Text("View 3"),
                                    label: {
                                        Text("Navigate")
                                     }))
    }
}

When clicking the NavigationLink inside of TestViews navigationBarItems it navigates to View 3. When I click the back button it does not go back to TestView it goes all the way back to Testing instead.
How do I make it go back to the previous view instead of the first view?
I am using Xcode 12.
Comment: It works fine for me on Xcode12, maybe there is something else?
Comment: I don't have anything else inside this file besides the `PreviewProvider` which is just used for the live preview
Answer: Put navigation link into body of navigation view (bar is not in navigation view)
So here is possible approach
struct TestView: View {
    @State private var isActive = false

    var body: some View {
            VStack {
                Text("View 2")
            }
            .background(
               NavigationLink(destination: Text("View 3"), isActive: $isActive,
                label: { EmptyView() })
            )
            .navigationBarItems(trailing:
               Button("Navigate", action: { self.isActive = true }))
    }
}

Comment: NavigationLink should be in body to be placed in stack of NavigationView, how you design it for reuse, is up to you. Moreover, keep in mind that navigationBarItems modifier is deprecated in favour of toolbar since iOS 14. Just be aware.
Comment: I have tried this and it currently does not navigate to View 3 when pressed. I put a print statement inside of the `Button` action with the isActive toggle and it is getting pressed, but the navigation is not working.
Comment: Fixed - typed here and forgot one argument. See updated. Tested with Xcode 12 / iOS 14.
Comment: This works exactly how I want. Would I have to do this type of logic for every view I want to have a `NavigationLink` be triggered when a bar button is clicked? Seems like the code would get pretty messy with the more bar buttons you have.


Apache redirects directories

Question: So, I'm trying to redirect any pages to a file, but avoid redirecting anything thats an existing file or directory.
RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} -d
RewriteRule ^(.+)$  $1 [L]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME} !-f
RewriteRule ^(.+)$  /index.php?page=$1 [QSA]

However, any directories still get redirected (existing files are fine and can be located.)
Comment: are you asking how to redirect any file (but not directory) requests that don't exist?
Answer: You're using REQUEST_FILENAME to mean REQUEST_URI, and while that is documented to work in some cases (like when used on a vhost), I have a feeling that in this case it's already determined into the final name. See if RewriteCond %{REQUEST_FILENAME} -d makes a difference, or just always use REQUEST_URI.
If not, well, one option is to rewrite to add index.htm/html/php/pl etc to the end, test each for existence and finalize the rule if so, otherwise go to the last step? That's all that I can think of off the top of my head.

Apache redirects directories


How to view a character with glyphshow?

Question: <code>/Helvetica findfont setfont
/a glyphshow
</code>
The interpreter give out a nocurrentpoint error. How to show a character with glyphshow in PostScript?
Thanks.
Answer: You have to indicate where to put it. (This is the meaning of the "nocurrentpoint" error.)
Oh, you'll also need to scale the font before setting it. The default font size is 1pt (1/72 inch) [very small]. (Not an error per se, but a foreseeable obstacle in your path.)
As pipitas (our SO Postscript top-dog) recommends, here's a complete example:
<code>%!
/Helvetica findfont 24 scalefont setfont   %or: /Helvetica 24 selectfont
306 396 moveto    %move to center of 8.5x11 page
/a glyphshow
showpage
</code>
Comment: @pipitas Thanks. It looks much better fleshed-out. ... Looks like Ken just took your Postscript crown! You're still top in `gs`, tho.
Comment: To make it fully clear: `/Helvetica findfont 24 scalefont setfont 306 396 moveto /a glyphshow` should do it.
Comment: Hehe. I'll happily hand over the GS crown too. KenS has just to keep contributing, and I'll keep upvoting him  :-)  -- It's good to see GS developers participating here...

How to view a character with glyphshow?


ZendSkeleton Issues

Question: I have downloaded .zip for zendskeleton and it is having composer.phar and composer.json.
when I run below given command in Git CLI from zendskeleton folder it gives some error.
php composer.phar self-update

Error:
$ php composer.phar self-update
[Composer\Downloader\TransportException]
  The "https://getcomposer.org/version" file could not be downloaded: failed
  to open stream: A connection attempt failed because the connected party did
   not properly respond after a period of time, or established connection fai
  led because connected host has failed to respond.
self-update
php composer.phar install

Error:
$ php composer.phar install
Warning: This development build of composer is over 30 days old. It is recommend
ed to update it by running "composer.phar self-update" to get the latest version
.
Loading composer repositories with package information
[Composer\Downloader\TransportException]
  The "https://packagist.org/packages.json" file could not be downloaded: fai
  led to open stream: A connection attempt failed because the connected party
   did not properly respond after a period of time, or established connection
   failed because connected host has failed to respond.
install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-c
ustom-installers] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--opt
imize-autoloader]
Edit: with php composer.phar update

$ php composer.phar update
Warning: This development build of composer is over 30 days old. It is recommend
ed to update it by running "composer.phar self-update" to get the latest version
.
Loading composer repositories with package information
[Composer\Downloader\TransportException]
  The "https://packagist.org/packages.json" file could not be downloaded: fai
  led to open stream: A connection attempt failed because the connected party
   did not properly respond after a period of time, or established connection
   failed because connected host has failed to respond.
update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock]
 [--no-custom-installers] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [
-o|--optimize-autoloader] [packages1] ... [packagesN]
Comment: FYI, to configure composer behind http proxy, (http://getcomposer.org/doc/03-cli.md#http-proxy-or-http-proxy)
Comment: have you tried `php composer.phar update` ?
Comment: what will happen with this command?
Comment: This will update the dependencies to latest version.
Comment: not working...see i have updated my quest with it as well
Comment: try expanding the process timeout of your composer json by adding this, `"config":{ "process-timeout":4000}`. or try by exceeding the `default_socket_timeout` time of php.ini file. may be its a host issue.
Comment: I have added config in json and default_socket_timeout = 60 but still same issue
Comment: let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/39261/discussion-between-paulraj-and-poonam-bhatt)
Comment: not having access to chat page ... not possible for me
Comment: this error may happen due to network errors. Check if the proxy or firewall settings are okay, and use ping to detect where the connection is blocked.
Comment: how can i check proxy and firewall settigns
Comment: i have ping the blocked ...it giving replay
Comment: can you run `php composer.phar diagnose` and see the output for the problem ?
Comment: yes...I did that as well
Comment: Warning: This development build of composer is over 30 days old. It is recommend
ed to update it by running "composer.phar self-update" to get the latest version
.
Checking platform settings: OK
Checking http connectivity: FAIL
[Composer\Downloader\TransportException] The "https://packagist.org/packages.jso
n" file could not be downloaded: failed to open stream: A connection attempt fai
led because the connected party did not properly respond after a period of time,
 or established connection failed because connected host has failed to respond.
Comment: Checking composer.json: OK
Checking disk free space: OK
Checking composer version:

  [Composer\Downloader\TransportException]
  The "https://getcomposer.org/version" file could not be downloaded: failed
  to open stream: A connection attempt failed because the connected party did
   not properly respond after a period of time, or established connection fai
  led because connected host has failed to respond.
Comment: enable the SSL extension if not in php.ini file. `extension=php_openssl.dll` and restart the server.
Comment: it is alreday enable and showing in php info
Comment: the problem is with your proxy settings then. check this page to change your proxy and try with that. (http://stackoverflow.com/questions/15268093/composer-cannot-download-files)
Comment: add answer to this question so that I can accept it
Answer: The error occurred due to the proxy settings. You need to set the http_proxy environment variables since you are on windows OS. You can set the environment variables as below,
set http_proxy=<your_http_proxy:proxy_port>
set https_proxy=<your_https_proxy:proxy_port>

To configure composer behind http proxy, Http Proxy

ZendSkeleton Issues


What shape is homeomorphic to a sphere attached with a handle and a cross cap?

Question: The classification theorem for closed surfaces states that a orientable closed surface is homeomorphic to a sphere with m handles with m >= 0; while a nonorintable closed surface is homeomorphic to a sphere with n cross caps with n > 0.
Then, what surface is homeomorphic to a sphere with both a handle and a cross cap? Is this surface orientable? What is its Euler number?
Thanks.
Comment: Look up Dyck's theorem.
Comment: The sphere with three crosscaps is homeomorphic to a sphere with one torus and one crosscap. So more generally, a sphere with $n$ crosscaps and $m$ tori is equivalent to one with $n+2m$ crosscaps and no tori.
Comment: the genus three non orientable surface: it is like a torus with a crosscap
Comment: compare: https://www.mathcurve.com/surfaces/dyck/dyck.shtml
Answer: Dyck's theorem  shows that although a torus by itself can't be turned into crosscaps, if there is another crosscap around you can send the torus through it and turn it into two crosscaps, so  the sphere with one torus and one crosscap  is homeomorphic to a sphere with
three crosscaps, and more generally, a sphere with $n$ crosscaps and $m$ tori is equivalent to one with $ n+2m $ crosscaps and no tori.


Will compiling PHP 5.4 work on a server that has PHP 5.3 installed?

Question: I have a dev machine with Ubuntu 12.04.  I installed AMP by following directions from http://www.howtoforge.com/installing-apache2-with-php5-and-mysql-support-on-ubuntu-12.04-lts-lamp.
Will it work if I download PHP5.4 source code and compile it?  Will it break packages such as php5-tidy, php5-xmlrpc, etc, etc?
Is this the best way to upgrade and maintain PHP?
Is there a good book for learning web server maintenance?
Thanks in advance.
Answer: You're probably better off in the Ubuntu world using a PPA (Personal Package Archive) to install a package not in the official repos.
http://www.mellzamora.com/installing-php5-4-apache2-and-mysql-on-ubuntu-linux/
Comment: I researched PPA and they seem to be maintained by individuals.  Can I jump back to Ubuntu packages if the PPA is abandoned?  Also, the directions only show a use case where PHP is not installed.  Will this be a problem?
Comment: I think the idea is that they are maintained by individuals, yes.  You can make your own to add your own packages to ubuntu.
Comment: Because it's a package it's trivial to remove it and go back to the official ones. I don't know how it will differ between systems with or without PHP already installed -- I would test it on a non-production box first.
Comment: Ok I tried the PPA. It upgraded PHP and everything seems to be working (so far).  Can you guys recommend a good book for web server management?  I feel that I can't be a better programmer until I understand server management.
Comment: That's a pretty broad subject, but I think the de facto standard GNU/Linux book is: [UNIX and Linux System Administration Handbook](http://www.amazon.com/UNIX-Linux-System-Administration-Handbook/dp/0131480057/ref=sr_1_1?s=books&ie=UTF8&qid=1342118316&sr=1-1&keywords=linux+administration+handbook)
Comment: That's one thick book.
Answer: You can compile it from source if you want.  Doing so won't break anything.  Installing it is likely to, as distributions tend to change the paths make install installs to, and they also (usually) test that particular versions work as dependencies.  You may have to compile other packages against the new PHP.
If you are using ubuntu, I find it is usually better to wait for ubuntu to release the new PHP as a package, because ubuntu-izing things can be a bit painful (I wouldn't want to test it on a production server, for instance).
Comment: So to clarify, compiling will replace the PHP 5.3. However, since it may get installed in a different directory other packages that rely on PHP such as php5-xmlrpc may not work.  Am I getting this right?