Handling observable errors inside a component when subscription / observable is defined external to component

Question: I have written an example below similar to my scenario (simplified).
Currently, I have a generic form component the emits a submit event. When this is handled outside of the component we subscribe to a request (defined outside of form).
However, I would like the form to know if there was an error with the request so it can reset. I want to avoid handling the error outside of the component (as someone may not handle it).
I was wondering how I could provide the subscription to the component or alternatively how the form can know if there was an error in an external request.
Can an Output receive a subscription / observable as well as emit? Or is there a way of providing an observable via Input (would need to be callable as it is created with external / dynamic data).
Event:
interface SubmitEvent {
   id: string;
   data: any;
}

Component A: A general form with submission. Not aware of specific contexts/ subscriptions.
@Component({
   selector: 'my-form'
   template: `<button (click)="doSubmit()">Submit</button>`
})
export class MyForm {
   @Input() id: string;
   @Output() formSubmitted = new EventEmitter<SubmitEvent>();

   formValues: any;

   doSubmit(){
      this.formSubmitted.emit({ id: this.id, data: this.formValues });
      /*
      after this emit I would like to know if there was an error with the external request.
      */

   }
}

Component B: One instance of many forms. Defines the observable / request that happens on submit.
@Component({
   selector: 'form-b'
   template: `<my-form [id]="'my-id'" (formSubmitted)="handleSubmit($event)"></my-form>`
})
export class FormB {

   @Input() somethingElse: string;

   constructor( private readonly httpService: HttpService ){}

   handleSubmit( event: SubmitEvent ){
      httpService.customPost( event.id, event.data, this.somethingElse ).subscribe();
      /*
      This request could be any observable. It may or may not use arguments from the event.
      */
   }

}

Answer: If I understand the problem right, I would approach it using a service.
More specifically:

First you create a service with a public method submitForm(params) which performs the submit logic, a private Subject that emits when the submit logic completes, and a public Observable that exposes the Subject as an Observable, i.e. something like this

public class MyService {
private _submitFormSubject = new Subject()
public submitForm$ = _submitFormSubject.asObservable()
public performSubmitForm() {
httpService.customPost( event.id, event.data, this.somethingElse )
.subscribe(this._submitFormSubject);
}
}

Then you inject MyService into the components MyForm and FormB

In the ngOnInit method of MyForm you subscribe to submitForm$ and in the subscription logic you define whatever logic you need, e.g. you deal with the errors that may be emitted

In FormB you react to the event fired by MyForm in the method handleSubmit by calling the method performSubmitForm of MyService

The service method performSubmitForm does the submit and, via the private Subject _submitFormSubject you notify either the response (in case of success of the submission) or an error (in case the submit operation fails)

Since MyForm has subscribe to the Observable submitForm$ it will be able to react to either the success of the failure of the submit operation


mgcv - gamm model evaluation

Question: After running a different model using gamma() in the mgcv package, when I do
anova(model.ar1.int$lme, model.ar1.orderedfactor.int$lme)

I got the following:
Model df      AIC      BIC     logLik   Test  L.Ratio p-value

model.ar1.int\$lme      1 13 18828.83 18918.72  -9401.414

model.ar1.orderedfactor.int\$lme     2 14 22783.59 22880.39 -11377.795 1 vs 2 3952.761  <.0001

I don't understand how come the 2nd model's R-square adj is significantly lower and a lot higher AIC and BIC, and yet the p-value is still <.0001 significant — doesn't that mean the 2nd model is favored?
Answer: The anova test will conduct a likelihood ratio test on the two models. This involves comparing the log likelihood of two nested models and test whether the alternative model is better than the null model. The null model is the one with less parameters and the anova() command will detect which model this is. It doesn't matter if you type anova(m1, m2) or anova(m2, m1), the results will still be the same.
In your case, I'm wondering if your models really are nested. If they were nested, the models with more paramters (model.ar1.orderedfactor.int) shouldn't have a lower log likelihood than the other model. So are your models really nested? Is it possible to set some of the parameters in model.ar1.orderedfactor.int to 0 to get the first model? An example:
Imagine you have a dependent variable y and two independent variables x1 and x2. Now compare these models
modelA <- lm(y ~ x1)
modelB <- lm(y ~ x1 + x2)

ModelA is nested within modelB because if the estimated coefficient of x2 in modelB is set to 0, you will get model A. You can thus compare them with the anova test. Now consider this model:
modelC <- lm(y ~ x2)

ModelC is also nested within modelB and you can compare them with the anova test, but modelC and modelA are not nested within each other, since you cannot transform one into the other by setting an estimated coefficient to 0. You therefore cannot use the anova command to compare the models.
I suspect you have non-nested models which is the reason that the model with more parameters (degrees of freedom) has a much lower log likelihood.

mgcv - gamm model evaluation


Como permitir que o input() receba notação científica em python 3?

Question: Tenho um coeficiente de dilatação térmica que o usuário deve fornecer, exemplo 1.6*10e-5, exatamente este valor, como permitir que o usuário escreva este valor em notação científica?
entrada do coeficiente pelo usuário
notação = input('digite a notação')

operação
multiplica_n = notação * 2

Print(str(multiplica_n))

Comment: Não faz sentido modificar a pergunta já que uma pequena variação da resposta soluciona o novo problema: `str_value = str_value.replace("*10", "")`.
Answer: A chamada normal para transformar uma string em número decimal - float, já converte qualquer número válido em Python para um objeto "float" que é tratado como um número.
E os números válidos incluem números com . decimal, sinal de "-", expoente de notação científica com "e" ou com "E", e os literais "inf" e "nan" - mas não incluem o sinal de vezes com a letra "X" (nem com o caractere unicode "⨉" (N-ARY TIMES OPERATOR"), que é diferente da letra "X"). Isso é, você pode passar "1.6e-5", mas ele não entende "1.6X10e-5" - isso já seria uma conta de vezes explicita.
É importante ter em mente que no caso de um número inválido a chamada a floatvai dar um erro do tipo "ValueError" - nesse caso é importante combinar um bloco try/except junto com um bloco while para repetir a entrada.
Então, como exemplos:
a = '2e5' cria uma string - se seguiros isso com print(float(a)), vamos ver o Python interpretar o "e" como o "e" da notação científica:
In [97]: a = "2e5"

In [98]: print(float(a))
200000.0

(As marcações 'In[...]' e 'Out[...]' são o prompt do Ipython no modo interativo)
A chamada a input sempre retorna uma string (em Python 3 - nas versões antigas da linguagem, até a 2.7, era necessário usar o raw_input)
Se você quiser que o símbolo "X" como operador de vezes possa ser usado, pode fazer uma transformação antes da chamada ao float para tirar a letra da string - então mesmo que o usuário digitar o número, a entrada continua válida. Para isso, é possível usar o método .replace das strings.
Por fim, se for validar a entrada com o while/try/except vale a pena colocar o trechinho todo de código numa função - de forma que esse código possa ser reusado para qualquer entrada de float.
def float_input(msg=""):
    retry = True
    while retry:
        str_value = input(msg)
        str_value = str_value.replace("X10", "").replace("x10", "")
        try:
            value = float(str_value)
        except ValueError:
            pass
        else:
            # Esse else é do bloco "try": só entra aqui quando não ocorrer erro na chamada ao `float()"
            retry = False
    return value

E dá para usar assim:
In [104]: v = float_input("Digite a notação: ")
Digite a notação: 1.6X10e-5

In [105]: print(v)
1.6e-05

In [106]: print(v * 2)
3.2e-05


Show a menu block only on nodes that belongs to this menu

Question: I am trying to accomplish something similar to the book navigation module.
Say I create a block containing a menu. I want this block to become visible for nodes that belong to this menu.
So if NodeA belongs to the menu "My SubMenu 1", which is included in a block called "My Submenu 1 Block", I want this block to appear everytime NodeA comes up.
This could be accomplished with some pathauto tricks, or assigning a certain taxonomy term to all nodes of "My Submenu 1". But I realise this is redundunt, there is no need to force the user to create content with specific terms--the information is all there in menu-->node connection.
Answer: Take a peek at http://drupal.org/project/menu_block. I think it might do what you want, or, if not, at least maybe its code might give you an idea on how to do so yourself.
Comment: Menu block does exactly this. In your menu block's configuration, for the 'Fixed parent item' option select 'the menu selected by the page.' Then be sure to enable which menus can be enabled for that option at admin/config/user-interface/menu-block.
Comment: thanks jimajamma @DaveReid and @mtro! great advice, you saved me from a lot of headaches..
Answer: Context module does exactly this. If you chose a "menu" condition, it will allow you to show blocks, menus, etc only for content that is listed in the menu.
Comment: thanks for your answer--I haven't tried either menu_block or Context yet, but it seems that menu-block is a bit more specialized towards my end. I could be wrong, but I will grant the answer to menu_block for now


The function that draws a figure eight

Question: I'm trying to describe a counterexample for a theorem which includes the figure eight or "infinity" symbol, but I'm having trouble finding a good piecewise function to draw it. I need it to be the symbol, except at the "crossing point" the function jumps (not continuous) so that we still have a manifold.
Comment: Odds are you'll get a much better response if you accept a response from your previous five questions!
Comment: See [the FAQ](http://math.stackexchange.com/faq#howtoask): "When you have decided which answer is the most helpful to you, mark it as the accepted answer by clicking on the check box outline to the left of the answer. This lets other people know that you have received a good answer to your question. Doing this is helpful because it shows other people that you're getting value from the community."
Comment: I'm sorry, I don't know how to "accept" a response on here :(
Answer: You can use the function $$t\in(-\tfrac12\pi,\tfrac32\pi)\mapsto(\cos t,\sin t\cos t)\in\mathbb R^2.$$
The resulting curve is (I'm omitting a little bit on the left and a little bit on the right from the domain in this picture):
Comment: This is the lemniscate of Gerono (alias "eight curve").
Comment: Well, it is the one you get if play 1 minute with a CAS which can do parametric plots... I guess Gerono did not have one, so +1 to him!
Comment: @Jim: To be more explicit, you can exploit the periodicity of the underlying trigonometric functions such that when traversing the curve, you only hit the origin once.
Comment: But wouldn't you hit (0,0) at both pi/2 and -pi/2?
Comment: @JimJones: indeed. It is trivial to fix that.
Answer: There are no functions that describe the "lemniscate", but there are parametric and polar equations for the lemniscate of Bernoulli and the lemniscate of Gerono, to name two of the more famous lemniscates.

Might as well share this, since the question already has an accepted answer. Here is my favorite way of generating the lemniscate of Bernoulli, as an envelope of circles centered on a rectangular hyperbola, and passing through the origin:
Comment: There's also the generalization of the lemniscate of Bernoulli: the [lemniscate of Booth](http://www.mathcurve.com/courbes2d/booth/booth.shtml).
Comment: ...and the [hippopede of Proclus](http://mathworld.wolfram.com/Hippopede.html) as well.
Answer: Is the lemniscate what you want?  I don't know what you need as a jump at the crossing point, but maybe you can get that with a trigonometric parameterization.

The function that draws a figure eight


Problema na impressão de um vetor

Question: Pessoal estou com um problema básico que não consigo entender o erro:
Classe Prova:
public class Prova {

public static void main(String[] args) {

    Vetor lista = new Vetor();

    Scanner scan = new Scanner(System.in);
    int codigo, idade;

        Dados dado = new Dados();

        System.out.println("Digite o código da idade: ");
        codigo = scan.nextInt();
        dado.setCodigo(codigo);

        System.out.println("Digite a idade da pessoa: ");
        idade = scan.nextInt();
        dado.setIdade(idade);

        lista.add(dado);

        System.out.println(lista);
    }
}

Classe Vetor:
public class Vetor {

private Dados[] dados = new Dados[100];
private int total = 0;

public void add(Dados item){

    for(int i = 0; i < this.dados.length; i++){
        this.dados[this.total] = item;
        this.total++;

    }
  }
}

Classe Dados:
public class Dados {

private int cod, idade;

public int getCodigo(){
    return this.cod;
}

public int getIdade(){
    return this.idade;
}

public void setCodigo(int cod){
    this.cod = cod;
}

public void setIdade(int idade){
    this.idade = idade;
}

}

O problema é que na minha saída está assim:
prova.Vetor@89cf1e
Não sei como fazer para sair com string. O que está faltando, como resolvo?
Comment: Você precisa de um laço pra percorrer item a item um vetor e exibi-los de forma individual. Mas esse metodo add, se você controla o total separado, pra que usar for? Basta checar se total alcançou o tamanho -1 do vetor
Comment: Da uma olhada nessa pergunta https://pt.stackoverflow.com/questions/126968/fazer-um-vetor-gen%C3%A9rico-em-java
Comment: Ok, entendi que não é necessário o uso do for no método add, mas não entendi o uso do for para percorrer os valores
Comment: Veja o exemplo da resposta que eu postei o link. Você precisa criar métodos na classe vetor que retornei um item informado um indice, e outro que retorne o tamanho dele.
Answer:
O problema é que na minha saída está assim: prova.Vetor@89cf1e

Quando você manda imprimir o vetor com o System.out.println(lista) é o nome do objeto e o identity hash code que costuma ser o seu endereço de memória. Isso ocorre porque o seu método toString não foi implementado para se comportar de forma diferente. Para que você imprima os valores do seu vetor, basta sobrescrever esse método na sua classe Vetor. Ficaria da seguinte forma:
class Vetor {
    @Override
    public String toString() {
        String vetorTexto = "";
        for (Dados dado : dados) {
             vetorTexto += dado.toString() + " ";
        }
        return vetorTexto;
    }
}

A classe Dados implementaria o toString para retornar as informações no formato desejado (e.g. codigo, codigo + idade, ...):
class Dados {
   @Override
   public String toString() {
       return codigo + "-" + idade; //Exemplo: formata o objeto um código-idade.
   }
}

Comment: A solução é essa mesmo, mas fazer no `toString()` eu não gosto não. Não tenho certeza se é normal fazer isso em Java, se for, achei mais um motivo para não gostar da linguagem. Em outras linguagens se considera um abuso do mecanismo já que esse método é para imprimir a identidade mesmo e não uma representação dos dados.
Comment: Interessante sua observação. Conceitualmente seria errado abusar disso, mas sugeri o toString por ser algo mais transparente para o print. Se eu fosse implementar faria uma classe utilitária para realizar a impressão dos dados.
Answer: Como você está implementando uma classe Vetor, seria interessante que a torne capaz de retornar essas informações de forma individual. Como eu disse nos comentários e está sugerido este link, você pode adicionar um método que retorne um item e outro que retorne o tamanho, para que a opção de exibir um ou todos seja de quem está usando a classe:
public int tamanho(){
    return total;
}

public Dados retornarElemento(int indice){
   if(indice >= total){
     throw new ArrayIndexOutOfBoundsException("Indice fora do intervalo do vetor");
   }
  return dados[i];
}

No main, você faz um laço pegando o tamanho e varrendo até o ultimo indice adicionado:
for(int i = 0; i < lista.tamanho(); i++){
    //retornarElemento() retorna um tipo Dados, por isso a chamada direta
    System.out.println(lista.retornarElemento(i).getCodigo());
    System.out.println(lista.retornarElemento(i).getIdade());
}

Pode parecer mais complexo, mas dá mais flexibilidade de uso da sua classe Vetor, permitindo saber a qualquer momento quantos itens foram adicionados, e recuperar um item individual(quando já se conhece seu indice) sem ter que usar laço.

Vale lembrar que seu método add pode melhorar, trocando o laço por verificações condicionais em relação a sua capacidade(se está cheio demais).

Problema na impressão de um vetor


What is "the space consisting of two stacked torus surfaces, stacked as one would stack two inner tubes"?

Question: This is a question in Fraleigh's abstract algebra book asking for computing the homology groups of this shape. But I do not understand that sentence and English is not my mother language...
Could someone please explain to me what is "stacked as one would stack two inner tubes"?
Comment: I don't have the book on me, but it sounds like what would happen if you stacked two donuts on top of each other. Then, the two tori intersect in a circle.
Comment: Two tori  contacting on their flat circles
Comment: @TakumiMurayama  thanks for comment, now I can understand that sentence.
Answer: Something like this. ${}{}{}{}$
Comment: The space is homeomorphic to $(S^1\vee S^1)\times S^1$, so it's fundamental group is $(\Bbb Z*\Bbb Z)\times \Bbb Z$.
Comment: It's kind of an interesting space topologically.  What is it's fundamental group?  I'm guessing $\Bbb Z\times\Bbb Z\times\Bbb Z$?
Comment: @iwriteonbananas Is $\Bbb Z * \Bbb Z$ the free group on two generators?
Comment: Indeed it is, @GregoryGrant.
Comment: @iwriteonbananas I see.  Thank you.  So I stand by my original contention.  This is an interesting space.  :-)
Comment: Sure it is, @GregoryGrant :)


Panning objects on a UIScrollView

Question: I'm working on an app where I have several UIView objects that are subviews on a UIScrollView object. I create the subviews programmatically and place them on the scroll view according to the properties of associated objects. The user is allowed to move these subviews around on the scroll view. Usually this works, but sometimes the scrollview grabs the pan gesture.
What I'd like to do is to suppress the scroll view gesture recognizer if the touch location is inside one of the subviews.
I can find the scroll view gesture recognizer by looking through the scroll view's array of gesture recognizers and looking for a UIScrollViewPanGestureRecognizer object. I assume there can only be one.
An idea I have is to make my view controller be a delegate of this gesture recognizer and then have the delegate suppress it if the touch is within the bounds of one of the subviews.
Is this the best way to handle this scenario, or is there a better way?
I've done something similar, described in my answer to my own question here.
How to get stepper and longpress to coexist?
Hmmm. Looks like it will be more difficult than I anticipated to recognize the scrollview's UIScrollViewPanGestureRecognizer. Any hints on doing this would be appreciated.
My idea doesn't work. In order to code my idea, I had to make my VC be the delegate of the scrollview's pan gesture recognizer. However, when I do that, I get this error:
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'UIScrollView's built-in pan gesture recognizer must have its scroll view as its delegate.'
Here is the code I used. In viewDidLoad I called a method which got the scrollview's pan gesture recognizer and set self as delegate (self.scrollViewPanGestureRecognizer is just a property to store it):
self.scrollViewPanGestureRecognizer = [self.scrollView panGestureRecognizer];
self.scrollViewPanGestureRecognizer.delegate = self;

I then implemented this delegate method:
-(BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldReceiveTouch:(UITouch *)touch
{
   //Disable touch if touch location is in a subview.
   BOOL enableGestureRecognizer = YES;

   if (gestureRecognizer == self.scrollViewPanGestureRecognizer) {
      CGPoint touchLocation = [touch locationInView:self.scrollView];
      for (UIView *s in self.scrollView.subviews) {
         if (CGRectContainsPoint(s.frame, touchLocation)) {
            enableGestureRecognizer = NO;
         }
      }
   }
   return enableGestureRecognizer;
}

Seemed like a good idea, but it looks like I can't make my VC be the delegate.
Just tried setting scrollEnabled to NO on the scroll view. That successfully disabled scrolling, but it did not fix the problem. Views still occasionally do not respond to gestures. Thinking that perhaps some bug caused the gesture recognizer to fall off the object, I asked the debugger to display the gesture recognizers for the problematic views. They were still there. I'm stumped.
UPDATE: New information. I finally realized that the subviews that aren't responding are the ones on the right side of the screen. After carefully testing, it seems that this happens only in landscape orientation and only when the finger location is to the right of the right edge in portraite, i.e. 320 points. Apparently, something is not being handled property when rotating to landscape. Everything appears normal, but the gestures aren't being recognized.
Just for grins, I decided to display the frames and bounds and content area in the method viewDidLayoutSubviews. What I get is:
self.view.frame             is {{0, 0}, {480, 320}}
self.view.bounds            is {{0, 0}, {480, 320}}
self.scrollView.frame       is {{0, 0}, {480, 320}}
self.scrollView.bounds      is {{0, 0}, {480, 320}}
self.scrollView.contentSize is {480, 320}

I seem to have missed something. What else needs to be set when rotating?
Answer: use requireGestureRecognizerToFail: method.
you want your scroll view pan gesture (scrollViewGesture) to be failed when one of the gestures happen on its subView.
So, when you add pan gesture to your subView  (subViewGesture),  set below property as
scrollViewGesture.requireGestureRecognizerToFail =subViewGesture;

Comment: A preliminary test seems to have confirmed this to work. I need to do more testing though before accepting this answer, since the problem was just an occasional problem to begin with.
Comment: One potential problem with this method is that it could introduce a delay in the scroll view's pan gesture recognizer's performance. So far, though, I don't see a significant delay. An example delay would be where a tap requires a double-tap to fail. That results in a default 0.2 second delay, I believe. Apparently, the failures I'm looking at return nearly instantly, though.
Comment: After further testing, it looks like this doesn't work.
Comment: I suspect the reason this doesn't work is that the method sets a relationship between two recognizers. In order for this to work, the scroll view's recognizer would need to require all the others to fail. To get it to work, I'd probably need to attach the recognizer to the scrollview instead of the subviews and fail it unless the touch location is inside one of the subviews. Now this is looking as complicated as the other offered solution.
Comment: possible to `requireGestureRecognizerToFail` with multiple other gestures? or just one?
Comment: you can use UIGestureRecogniserDelegate methods to work with multiple gestures. See this method - (BOOL)gestureRecognizer:(UIGestureRecognizer *)gestureRecognizer shouldRequireFailureOfGestureRecognizer:(UIGestureRecognizer *)otherGestureRecognizer NS_AVAILABLE_IOS(7_0);
Comment: santhu's answer works for me in Swift 4. I have a parent scrollview that the user can pinch/zoom, and pan. Within that parent scrollview, I have a child scroll view which contains text that the user can scroll up and down. Sometimes when the user tries to scroll up / down, the parent scrollview's pan gesture happens instead. Inside the child scroll view, I did this: `parentscrollView.panGestureRecognizer.require(toFail: childScrollView.panGestureRecognizer)`, and now the user can scroll up and down on the child without the parent view gesturing being triggered instead.
Answer: I found the solution. I'd forgotten that the subviews are not placed directly into the scroll view. There is a view originally occupying the bounds of scrollview onto which the subviews are placed. The hierarchy is like this:
self.view
    scroll view
        UIView (fills whole scroll view)
            subview1
            subview2
            subviewn

In my code to handle rotation, I was not resizing the UIView into which the subviews are placed. Correcting this issue solved the problem.
I'd originally tried placing the subviews without their UIView superview in between them and the scroll view, but it didn't work for some reason. Adding this extra layer solved that problem, but I forgot to handle the resizing when rotating.
So I guess the gesture recognizers did not respond because although they were visible, they were outside the bounds of their superview.
Answer: I'm making this answer a community wiki because I haven't completely worked out this solution yet. The main thing is to take advantage of this from the documentation:

Subclasses can override the
  touchesShouldBegin:withEvent:inContentView:, pagingEnabled, and
  touchesShouldCancelInContentView: methods (which are called by the
  scroll view) to affect how the scroll view handles scrolling
  gestures.
Answer: One solution: instead of playing around with gesture recognizers, just disable all of them and use touchesBegan, touchesMove and touchesEnded directly. It might be a bit of work, but pretty sure it will work exactly the way you want.
You need to disable user interaction on the subviews, disable scrolling on the scrollview, and modify the scrollview's contentOffset directly.
Comment: I think I'm going with the other answer so I don't have to do all the work involved with this option. So far so good.

Panning objects on a UIScrollView


Amount of reward I get until the first heads

Question: I play a simple game which is modeled as follows:
I've a coin with heads probability $p$. I flip it and then give myself 1 coin with probability $p_w$ (if tails, I get nothing). Then I check the coin, if it's heads I stop, otherwise I continue until I get heads.
Let $R$ denote the number of coins I got when the game terminates.
I'd like to calculate $E(R)$ and $Var(R)$. My attempt was as follows.
Let $N$ denote the number of rounds until I get a heads and $X$ denote the number of coins I get in a round. We have $R = N\cdot X$.
We further have $N \sim Geom(p)$ and $X \sim Bern(p_w)$. Since $N$ and $X$ independent we can write,
$$ E(R) = E(N)\cdot E(X),$$
$$Var(R) =  E(X)^2\cdot Var(N) + E(N)^2 \cdot Var(X) + Var(X)\cdot Var(N).$$
Finally, plugging the values yield.
$$ E(R) = p_w / p $$
$$ Var(R) = \frac{p_w(2 - p - p_w)}{p^2} $$
I think the expected value is sound however, variance does't make any sense to me. I feel that the only thing that matters is the ratio $\frac{p_w}{p}$ but value of variance changes with actual values of probabilities.
In particular, we can have $E(R) - StdDev(R)$ negative in some cases which doesn't make any sense.
Am I overlooking something?
Comment: @spiderrico It is possible for a distribution that can only be nonnegative to have negative numbers within one standard deviation of the mean. In fact it's the case for the Bernoulli distribution with $p<1/2.$ This is ok... it's just soreness.
Comment: Even for a process that never returns negative numbers, there's nothing inherently contradictory about $E(X) < StdDev(X)$ (or, equivalently in this case, $[E(X)]^2 < Var(X)$) so long as the distribution for $X$ is skewed.  For example, if $X$ is selected from $0, ..., n$ such that 0 is selected with probability $\frac{1}{2}$ and $1, ..., n$ are selected with probability $\frac{1}{2n}$, then we have $Var(X) = \frac{1}{48}(n+1)(5n+1)$ while $E(X) = \frac{1}{4}(n+1)$.  In this case, $Var(X) > [E(X)]^2$ for $n > 1$.
Comment: The reward is always positive. How can a negative one be bounds within std.dev? Isn't that illogical?
Comment: @spaceisdarkgreen:  That is a much nicer example than mine.  Another better-known one is the Poisson distribution with $\lambda < 1$.
Comment: @spiderrico I mean skewness! (Of course it autocorrected and was just in time for the 5 minute window to edit comments.)
Comment: I see. Seemed pretty counter-intuitive initially.
Answer: Yes, you are missing something. Your equation $R=NX$ is not coherent. You say $X$ is "the number of coins you get in a round" but what does that even mean? You have a random number $N$ different rounds and on each round you get some number $X_i$ of coins. Thus you should write $$ R= \sum_{i=1}^N X_i.$$
For help doing calculations where the number of elements in the sum is random, perhaps google Wald's theorem, which states that in fact your answer for expected value is correct. The secret is basically using conditional probability carefully.
Comment: I see my error now. In a round, you either get 1 coin or 0 by the way.
Comment: Also, how can I compute the variance of R in this case? How many terms are there r.h.s when I take Var of both sides ?
Comment: Ok, got that here also: http://www.math.unl.edu/~sdunbar1/ProbabilityTheory/Lessons/Conditionals/RandomSums/randsum.shtml
Comment: @spiderRico That link you found answers your question, right?


time mismatch after convert with format_date

Question: In my custom module, I converted time from database like ('17:00:00') with format_date(). After I converted this I got a wrong time (17:10). Please check my code, I am also naive drupal develper.
$form['departure_time'] = array(
  '#type' => 'textfield',
  '#maxlength' => 10,
  '#title' => t('Departure Time'),
  '#default_value' => format_date(strtotime($data -> departure_time),'custom', 'H:m'),
  '#description' => t("You Need To Add Departure Time.<< 00:00"),
  '#required' => TRUE,
);

Answer: As listed in the documentation http://php.net/manual/en/function.date.php lowercase m is for month. For minutes you should use i.
$dep= format_date(strtotime($data -> departure_time),'custom', 'H:i');

Answer: Just of note, you don't need to pass the date through t() as format_date() handles this itself through _format_date_callback() for reference.  You just have to pass in the langcode yourself.
Comment: This could have been a comment or a suggested edit to the answer
Comment: It's a completely different comment, not related to the accepted answer post - it's optional and more about coding standards than how to solve the posters issue.  If I were to add something related, it'd be more to do with that it's fairly clumsy to ask for a date via a textfield - a select box is a much better (and safer option).


Nginx: Redirect https://www.domain.tld/target to https://nocache.domain.tld/destination

Question: I have a Cloudflare/Nginx(v1.8.0)/Apache server config with lots of caching.
Goal is to cache the main domain but allow access to a site via a subdomain on which all caching is bypassed.
So, I'm looking for a generic way of redirecting "https://www.domain.tld/target" to "https://nocache.domain.tld/destination"
A simple location match against the request_uri is enough for the non-www domain variant, but if the request comes in over the www subdomain, like many of the sites will, then it redirects to nocache.www.domain.tld which doesn't exist.
I've tried all kinds of different ways to remove the 'www.' substring, but nothing seems to work.
EDIT
Here is my non-working Nginx server config:
server {

  listen 80 default_server;

  server_name localhost;

  set $PROXY_DOMAIN_OR_IP $host;

  if ( $request ~* ".*(www\.)(.*)\/target" ) {
     set $host_without_www $2;
     return 302 $scheme://nocache.${host_without_www}\/destination;
  }

Comment: Could you post your Nginx config? You may have had a working rule, but had a rule which matched it before hand.
Comment: Thanks varlogtim.  Only just joined, so limited in the number of urls I can post.  Also my nginx config is quite large (Engintron).  That said, the problem can be boiled down to needing a way to create a $host_without_www variable, and then redirect to that instead of $host.
Answer: Here is a location block that you can use for the redirection
server {
    server_name www.domain.tld;
    location /target {
        rewrite /target(.*)$ /destination$1 last;
        return 301 https://nocache.domain.tld$request_uri;
    }
}

The server_name limits the redirection to the www.domain.tld domain
The location matches against all URIs that start with /target
the rewrite directive converts /target to /destination and keeps anything that appears after
The return pushes the redirect to the other domain
you can also do this with proxy pass if you don't want the user to be redirected
Comment: Thanks, however I need the rule to cope with multiple domains.  I've got a similar location block to yours except that I want the return to be to something like: https://nocache.${domain}.${tld} with any 'www.' removed from $domain.
Comment: You do realize that you can't use multiple domains in your server block because you are using SSL, right?  So why are you redirecting to multiple domains?  In any case, it is super simple to right the regex that you want using the same format suggested above.  Good luck!
Comment: See server config above.  The server block serves multiple domains (not my config and works very well).  On the SSL matter, Cloudflare Flexible SSL sends https requests to port 80 of the origin server (nginx).
Answer: Finally got it working as follows:
location ~* "/target" {
  if ( $host ~* "www\.(.*)" ) {
    return 302 https://nocache.$1/destination;
  }

  return 302 https://nocache.$host/destination;
}

The 302's were for testing and will be changed to permanent redirects.
Thanks to all who responded.


Elevation em Botão não está mostrando a Sombra

Question: Olá,
Estou tentando usar Elevation nos botões da seguinte tela:

Mas não está funcionando, o Elevation não está formando sombra.
Segue o XML:
<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context="kiraitami.krtbeta03.MainActivity">

<android.support.v7.widget.Toolbar
    android:id="@+id/toobar"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:layout_marginTop="23dp"
    android:background="#13000000"
    android:elevation="3dp"
    android:minHeight="?attr/actionBarSize"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toTopOf="parent" />

<ImageView
    android:id="@+id/bg_mountain"
    android:layout_width="462dp"
    android:layout_height="777dp"
    android:scaleType="centerCrop"
    android:src="@drawable/mountain"
    app:layout_constraintBottom_toTopOf="parent"
    app:layout_constraintEnd_toStartOf="parent"
    app:layout_constraintHorizontal_bias="0.0"
    app:layout_constraintStart_toStartOf="parent" />

<ImageView
    android:id="@+id/bg_diagonal"
    android:layout_width="match_parent"
    android:layout_height="485dp"
    android:scaleType="centerCrop"
    android:src="@drawable/diagonal"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toStartOf="parent"
    app:layout_constraintHorizontal_bias="0.231"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintTop_toBottomOf="@+id/bg_mountain"
    app:layout_constraintVertical_bias="0.818" />

<Button
    android:id="@+id/btn_damage"
    android:layout_width="311dp"
    android:layout_height="38dp"
    android:elevation="20dp"
    android:clipToPadding="false"
    android:layout_marginBottom="212dp"
    android:layout_marginEnd="8dp"
    android:layout_marginStart="8dp"
    android:background="@drawable/damagebtn"
    android:onClick="startDamageActivity"
    android:text="BOTAO 1"
    android:textAppearance="@style/TextAppearance.AppCompat"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintHorizontal_bias="0.14"
    app:layout_constraintStart_toStartOf="parent" />

<Button
    android:id="@+id/btn_audio"
    android:layout_width="311dp"
    android:layout_height="38dp"
    android:layout_marginBottom="80dp"
    android:layout_marginEnd="8dp"
    android:layout_marginStart="8dp"
    android:background="@drawable/audiobtn"
    android:elevation="10dp"
    android:text="BOTAO 3"
    android:textAppearance="@style/TextAppearance.AppCompat"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintHorizontal_bias="0.14"
    app:layout_constraintStart_toStartOf="parent" />

<Button
    android:id="@+id/btn_notes2"
    android:layout_width="311dp"
    android:layout_height="38dp"
    android:layout_marginBottom="144dp"
    android:layout_marginEnd="8dp"
    android:layout_marginStart="8dp"
    android:background="@drawable/notesbtn"
    android:elevation="8dp"
    android:text="BOTAO 2"
    android:textAppearance="@style/TextAppearance.AppCompat"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintHorizontal_bias="0.14"
    app:layout_constraintStart_toStartOf="parent" />

<Button
    android:id="@+id/button4"
    android:layout_width="50dp"
    android:layout_height="50dp"
    android:layout_marginBottom="8dp"
    android:layout_marginEnd="8dp"
    android:layout_marginStart="8dp"
    android:background="@android:drawable/ic_lock_power_off"
    android:elevation="8dp"
    app:layout_constraintBottom_toBottomOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintStart_toStartOf="parent" />
</android.support.constraint.ConstraintLayout>

PS.: O único lugar em que a elevation está funcionando é na Toolbar
O que devo fazer para consertar isso?
Comment: Já tentou colocar tudo dentro de um LinearLayout ?
Answer: Ao atribuir um background ao botão este irá perder algumas das sua características.
Entre elas estão a elevação e a animação ao ser clicado.
Por outro lado, mesmo sem atribuir um background não é possível alterar a elevação apenas atribuindo um valor através de android:elevation.
A elevação, nos diferentes estados de um botão(Enabled, Disabled, Focused, Pressed) é controlada por um StateListAnimator.
Assim, para alterar a elevação terá de construir o seu próprio StateListAnimator ou atribuir null a android:stateListAnimator.
android:stateListAnimator="@null"
android:elevation="20dp"

No seu caso sugiro que em vez de atribuir a imagem ao backgroud use android:drawableLeft.
Qualquer coisa assim:
<Button
    android:id="@+id/btn_notes2"
    android:layout_width="311dp"
    android:layout_height="wrap_content"
    android:textAppearance="@style/TextAppearance.AppCompat"
    android:backgroundTint="#ffffff"
    android:drawableLeft="@drawable/notesbtn"
    android:text="BOTAO 2"/>

Nota: O efeito da elevation não é visível em dispositivos Android com API inferior à 21.
Comment: Vou tentar!
Muito Obrigado!!!


Turn  redirects ON and OFF?

Question: I have a legacy ColdFusion system with over 200 <cflocation> tags, and would like to be able to easily turn them all off for testing purposes from within my Application.cfm.
What would be the best way to do this other than having to add a conditional statement to every file?
The thought of extending the <cflocation> tag to first check a true/false flag came to mind, but I have never done anything that advanced before, and have no idea where to start.
Comment: This is not possible. You cannot just globally override tags. You will need to deal with them conditionally like any other code.
Comment: Can't extend tags either. You can create your own custom tag and try a search/replace, but who knows what kind of developer nightmare you'd have at that point.
Answer: You cannot extend/inherit native tags in CF. You could go with a custom tag and then search-replace all occurences of <cflocation (native) with <cf_location (custom) having your redirect flag inside of it.
Application.cfm
<cfset APPLICATION.allowRedirects = false>

location.cfm (wrapper for native cflocation)

<cfif (
    (not isDefined("APPLICATION.allowRedirects")) or
    (isBoolean(APPLICATION.allowRedirects) and APPLICATION.allowRedirects)
)>
    <cflocation attributeCollection="#ATTRIBUTES#">

</cfif>

<cfif structKeyExists(ATTRIBUTES, "url")>
    <cfoutput>Attempted to redirect to: #encodeForHtml(ATTRIBUTES["url"])#</cfoutput>
</cfif>

<cfabort>

Another approach would be <cfmodule>. Search-replace all occurences of <cflocation with <cfmodule template="/mytags/location.cfm". Advantage here: You don't need to register location.cfm via customTagPaths since cfmodule works via template path.

Even simpler: Just search-replace all of your <cflocation> tags in one go and wrap them in a conditional. You won't have to go through your 200 files manually.
(This is for Sublime Text, but any proper editor can do this for you.)
Find: (<cflocation[^>]+>) (enable RegEx!)
Replace: <cfif APPLICATION.allowRedirects>\1</cfif>
(\1 is the backreference, some editors use $1 instead)
Comment: `how my hosting company disables specific tags so they are not available` Probably using the CF's sandbox security. IIRC when enabled, the server throws an error when a restricted tag is encountered in code.  Probably not what you need here.
Comment: Thank you for the great detailed explanation!

I am actually surprised to hear that the core CF tags cannot be extended.  I thought pretty much ALL languages could do that.  I guess that is why I have never done it.

I was also hoping there might be a way to simply DISABLE it, but I am guessing that is not possible either.  I wonder how my hosting company disables specific tags so they are not available.  It must be something super CUSTOM.

I am already very familiar with Find & Replace, but I didn't want to COMMIT that many files to the repo for something irrelevant to the system.
Comment: I guess I am going to just write a custom CFC, and perform the logic in there.

Again, thank you for taking the time to respond!

-Sergio

Turn  redirects ON and OFF?


Juniper SRX IPSec tunnel to Microsoft Azure Dropping

Question: I'm a bit stumped and was hoping to find some guidance here.
I've configured an IPSec tunnel to Microsoft Azure from my Juniper SRX240 (12.1X44-D45.2). The tunnel works fine but phase 2 drops when there is no traffic running across the tunnel (doesn't matter from which side traffic originates).
I've tried playing around with DPD but Azure doesn't support it. I've also configured VPN monitor to a destination on the other end of the tunnel but this also didn't work. In my "show log kmd" I am seeing P2 no proposal chosen messages after the drop occurs. I should add that phase 1 never drops.
This would be ok but unfortunately I have to statically route the remote ranges over the tunnel and since the tunnel doesn't (and can't) have an IP address, my next hop is st0.2. When phase 2 drops, so does the static route and routing follows the next more specific route. So there's no way to bring the tunnel back up automatically at this time.
I would greatly appreciate any advice or assistance on the matter. I need the tunnel to stay up even when there's no traffic running over it. Please see my config below.
<code>set groups GENERIC_GROUP security ike proposal IKE_PROPOSAL authentication-method pre-shared-keys
set groups GENERIC_GROUP security ike proposal IKE_PROPOSAL dh-group group2
set groups GENERIC_GROUP security ike proposal IKE_PROPOSAL authentication-algorithm sha1
set groups GENERIC_GROUP security ike proposal IKE_PROPOSAL encryption-algorithm aes-256-cbc
set groups GENERIC_GROUP security ike proposal IKE_PROPOSAL lifetime-seconds 28800
set groups GENERIC_GROUP security ipsec proposal IPSEC_PROPOSAL protocol esp
set groups GENERIC_GROUP security ipsec proposal IPSEC_PROPOSAL authentication-algorithm hmac-sha1-96
set groups GENERIC_GROUP security ipsec proposal IPSEC_PROPOSAL encryption-algorithm aes-256-cbc
set groups GENERIC_GROUP security ipsec proposal IPSEC_PROPOSAL lifetime-seconds 3600
set groups GENERIC_GROUP security ipsec policy IPSEC_POLICY proposals IPSEC_PROPOSAL
set groups CUSTOMER_GROUP interfaces st0 unit 2 family inet
set groups CUSTOMER_GROUP security ike policy IKE_POLICY mode main
set groups CUSTOMER_GROUP security ike policy IKE_POLICY proposals IKE_PROPOSAL
set groups CUSTOMER_GROUP security ike policy IKE_POLICY pre-shared-key ascii-text omitted
set groups CUSTOMER_GROUP security ike gateway IKE_GATEWAY ike-policy IKE_POLICY
set groups CUSTOMER_GROUP security ike gateway IKE_GATEWAY address omitted
set groups CUSTOMER_GROUP security ike gateway IKE_GATEWAY external-interface vlan.457
set groups CUSTOMER_GROUP security ike gateway IKE_GATEWAY version v2-only
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN bind-interface st0.2
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN vpn-monitor optimized
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN vpn-monitor destination-ip 192.168.183.2
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN ike gateway IKE_GATEWAY
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN ike ipsec-policy IPSEC_POLICY
set groups CUSTOMER_GROUP security ipsec vpn IPSEC_VPN establish-tunnels immediately
set groups CUSTOMER_GROUP security policies from-zone AZURE_ZONE to-zone CUSTOMER_TRUST policy default-allow match source-address AZURE_ZONE-RANGE
set groups CUSTOMER_GROUP security policies from-zone AZURE_ZONE to-zone CUSTOMER_TRUST policy default-allow match destination-address CUSTOMER-PRIVATES
set groups CUSTOMER_GROUP security policies from-zone AZURE_ZONE to-zone CUSTOMER_TRUST policy default-allow match application any
set groups CUSTOMER_GROUP security policies from-zone AZURE_ZONE to-zone CUSTOMER_TRUST policy default-allow then permit
set groups CUSTOMER_GROUP security zones security-zone INTERNET interfaces vlan.457 host-inbound-traffic system-services ike
set groups CUSTOMER_GROUP security zones security-zone INTERNET interfaces vlan.457 host-inbound-traffic system-services ssh
set groups CUSTOMER_GROUP security zones security-zone INTERNET interfaces vlan.457 host-inbound-traffic system-services snmp
set groups CUSTOMER_GROUP security zones security-zone INTERNET interfaces vlan.457 host-inbound-traffic system-services telnet
set groups CUSTOMER_GROUP security zones security-zone INTERNET interfaces vlan.457 host-inbound-traffic system-services ping
set groups CUSTOMER_GROUP security zones security-zone AZURE_ZONE address-book address AZURE_ZONE-RANGE 192.168.183.0/24
set groups CUSTOMER_GROUP security zones security-zone AZURE_ZONE interfaces st0.2 host-inbound-traffic system-services all
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address CUSTOMER-PRIVATE-RANGE1 10.0.0.0/8
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address CUSTOMER-PRIVATE-RANGE2 172.16.0.0/12
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address CUSTOMER-PRIVATE-RANGE3 192.168.0.0/16
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address-set CUSTOMER-PRIVATES address CUSTOMER-PRIVATE-RANGE1
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address-set CUSTOMER-PRIVATES address CUSTOMER-PRIVATE-RANGE2
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST address-book address-set CUSTOMER-PRIVATES address CUSTOMER-PRIVATE-RANGE3
set groups CUSTOMER_GROUP security zones security-zone CUSTOMER_TRUST interfaces vlan.456 host-inbound-traffic system-services all
set groups CUSTOMER_GROUP routing-instances CUSTOMER_GROUP routing-options static route 0.0.0.0/0 next-hop 18.104.22.168
set groups CUSTOMER_GROUP routing-instances CUSTOMER_GROUP routing-options static route 192.168.183.0/24 next-hop st0.2
set groups CUSTOMER_GROUP routing-instances CUSTOMER_GROUP routing-options static route 192.168.0.0/16 next-hop 172.31.0.2
set groups CUSTOMER_GROUP routing-instances CUSTOMER_GROUP routing-options static route 10.0.0.0/8 next-hop 172.31.0.2
set groups CUSTOMER_GROUP routing-instances CUSTOMER_GROUP routing-options static route 172.16.0.0/12 next-hop 172.31.0.2
</code>
This is what the kmd logs look like.
<code>[Jul  9 13:56:40]Added (spi=0xffa48b1d, protocol=0) entry to the spi table
[Jul  9 13:56:40]Construction NHTB payload for  local:184.108.40.206, remote:126.96.36.199 IKEv2 P1 SA index 1241218 sa-cfg IPSEC_VPN
[Jul  9 13:56:40]Peer router vendor is not Juniper. Not sending NHTB payload for sa-cfg IPSEC_VPN
[Jul  9 13:56:40]ikev2_packet_allocate: Allocated packet db4000 from freelist
[Jul  9 13:56:40]Received authenticated notification payload No proposal chosen from local:18.104.22.168 remote:22.214.171.124 IKEv2 for P1 SA 1241218
[Jul  9 13:56:40]ikev2_decode_packet: [db4000/dfe400] Received packet: HDR, N(NO_PROPOSAL_CHOSEN)
[Jul  9 13:56:40]ikev2_state_child_initiator_in: [db4000/dfe400] Error: Mandatory payloads (SAr,Ni,TSi,TSr) missing
[Jul  9 13:56:40]ikev2_process_notify: [db4000/dfe400] Received error notify No proposal chosen (14)
[Jul  9 13:56:40]ikev2_state_error: [db4000/dfe400] Negotiation failed because of error No proposal chosen (14)
[Jul  9 13:56:40]IPSec negotiation failed for SA-CFG IPSEC_VPN for local:184.108.40.206, remote:184.108.40.206 IKEv2. status: No proposal chosen
[Jul  9 13:56:40]   P2 ed info: flags 0x82, P2 error: Error ok
[Jul  9 13:56:40]IPSec SA done callback with sa-cfg NULL in p2_ed. status: No proposal chosen
[Jul  9 13:56:42]ikev2_packet_allocate: Allocated packet db4400 from freelist
[Jul  9 13:56:42]ikev2_decode_packet: [db4400/dfe400] Setting ed pkt ctx from VR id 4 to VR id 4)
[Jul  9 13:56:42]ikev2_decode_packet: [db4400/dfe400] Received packet: HDR
[Jul  9 13:56:42]ikev2_packet_allocate: Allocated packet db4800 from freelist
[Jul  9 13:56:43]ikev2_packet_allocate: Allocated packet db4c00 from freelist
</code>
Like I said, it works perfectly well until there's no traffic and I have no idea what else to try.
Thanks in advance!
Answer: That issue sounds like an issue I had in a IPSec VPN tunnel between Vyatta and Juniper SRX.
Have you tried to configure in your juniper and in azure the dead peer detection under the IKE configuration in the first phase of the VPN negotiation?
In Juniper I know that it is enabled by default but for example in Vyatta I had to configure manually and it looks like something like that:
<code>    ike-group <IKE-GROUP> {
        dead-peer-detection {
            action restart
            interval 15
            timeout 30
        }
        lifetime 3600
        proposal 1 {
            encryption aes256
            hash sha1
        }
        proposal 2 {
            encryption aes256
            hash sha1
        }
    }
</code>
Please let me know if it does work for you.
Saul


Obtaining $A$ from $A$$A^t$$A$

Question: Let the matrix $B$=$AA^TA$ be given to us where A is a mxn real matrix.Than how can we obtain $A$ from $B$ ? Can we do the same thing if A is a complex matrix ?
I have no idea how to do this problem.Please help and thanks in advance.
Comment: It certainly cannot be done when $A$ is complex -- then if $A$ is the $1\times 1$ matrix with element $e^{2\pi/3i}$, $B$ will be $I$.
Answer: Suppose $A$ has a singular value decomposition $U\Sigma V^T$ where $U$ and $V$ are orthogonal, and $\Sigma$ positive semidefinite diagonal.
In that case,
$$AA^TA=U\Sigma^3 V^T$$
Now think reverse. A SVD decomposition of $B$ has to share $U$ and $V^T$ with $A$, because the decomposition is unique up to permutation of singular values (that we usually sort). You must simply split
$$B=U\Gamma V^T$$
and compute
$$\Sigma=\sqrt[3]{\Gamma}$$
where this is meant as taking the cube root of every singular value. Because they are positive, this is not ambiguous.
Note that this gives you one solution, but it isn't unique, because the equation $\sigma_i^3=\gamma_i$ has $3$ solutions (for each singular value, a different root can be chosen).
Answer: Orion's method works when $m=n$. Yet a similar method works for any $m,n$. In such a case, $\Sigma$ is a $m\times n$ matrix and $AA^TA=U\Sigma\Sigma^T\Sigma V^T$. Fortunately, the matrix  $\Sigma\Sigma^T\Sigma $ has exactly the same form than $\Sigma$ ; indeed it suffices to change each $\Sigma_{i,i}$ with ${\Sigma_{i,i}}^3$. Thus $U\Sigma\Sigma^T\Sigma V^T$ is "the" SVD of $B=AA^TA$. We finish the construction as Orion did.

Obtaining $A$ from $A$$A^t$$A$


Where does the nprogress configuration go in an Aurelia Decorator?

Question: Having worked through the Aurelia Contact-Manager tutorial I'd now like to change the HTML element that the nprogress progress bar is rendered against.
I've updated the src/app.html by adding an id to the div.container like so:
<template>
  <require from="bootstrap/css/bootstrap.css"></require>
  <require from="./styles.css"></require>
  <require from="./contact-list"></require>

   ...

  <loading-indicator loading.bind="router.isNavigating || api.isRequseting"></loading-indicator>

      <contact-list class="col-md-4"></contact-list>
      <router-view class="col-md-8"></router-view>

</template>

.. and according to the nprogress documentation, I need to call the .configure() method like so..
NProgress.configure({ parent: '#main' });

But where does this line of code go?  I assume somewhere in src/resources/elements/loading-indicator.js..
Answer: Your assumption is correct, simply amend resources/elements/loading-indicator.js, adding a constructor() method to the class where you can call the .configure() method (line 8)..
import * as nprogress from 'nprogress';
import {bindable, noView, decorators} from 'aurelia-framework';

export let LoadingIndicator = decorators(
  noView(['nprogress/nprogress.css']),
  bindable({name: 'loading', defaultValue: false})
).on(class {
  constructor() {
    nprogress.configure({ parent: '#main' });
  }
  loadingChanged(newValue) {
    if (newValue) {
      nprogress.start();
    } else {
      nprogress.done();
    }
  }
});


Validações do Command - CQRS

Question: Estou aprendendo sobre CQRS e em um Hangout promovido por um MVP da Micrososft ele apresentou uma implementação de Command em que há validações dos dados do modelo dentro da validação do Command. Validações de datas ou se há campos obrigatórios em branco, por exemplo, não estavam no modelo tornando a entidade de modelo quase anêmica. Aí me veio as dúvidas, que apesar da explicação do MVP que pra mim não foi muito convincente.
É realmente uma boa prática implementar validação do modelo dentro da validação do Command ?
Exemplo da abordagem usada no Hungout:
public abstract class UserCommand : Command
{
    public Guid Id { get; protected set; }

    public string Name { get; protected set; }

    public string Email { get; protected set; }

    public DateTime BirthDate { get; protected set; }
}

public class RegisterNewUserCommand : UserCommand
{
    public RegisterNewUserCommand(string name, string email, DateTime birthDate)
    {
        Name = name;
        Email = email;
        BirthDate = birthDate;
    }

    public override bool IsValid()
    {
        ValidationResult = new RegisterNewUserCommandValidation().Validate(this);
        return ValidationResult.IsValid;
    }
}

public class RegisterNewUserCommand : UserCommand
{

 public class RegisterNewUserCommandValidation : UserValidation<RegisterNewUserCommand>
{
    public RegisterNewUserCommandValidation()
    {
        ValidateName();
        ValidateBirthDate();
        ValidateEmail();
    }
}

O que realmente é feito dentro da validação de um COMMAND?
Permissão para execução do Command, existência de duplicidade seriam exemplo de validação ?
Comment: Pode dar mais detalhes?
Comment: @Maniero adicione trechos de códigos que exemplificam a questão...
Answer: As validações variam de acordo com a aplicação e com os dados a serem validados.
Se, por exemplo, tu tens um formulário para cadastro de cliente em um e-commerce, tu precisarias, por exemplo, validar o CPF (documento) da pessoa, se as datas estão no formato correto, se os campos obrigatórios foram preenchidos. Tu ainda poderias validar se o endereço cadastrado existe com uma integração com alguma API (Correios, Google Maps, etc). Também, na hora de efetivar uma compra, é necessário validar cartão de crédito, cupom de desconto, etc...
Essas validações se extendem - poderias validar permissões de acessos a determinados dados, credenciais de usuários, criptografia, etc.
Em resumo - são validações para ver se os dados contidos no objeto estão corretos para prosseguir com o comando...
No exemplo de código dado, a validação é se os dados estão corretos para criação de um usuário. Poderiam estar vendo se não há duplicatas, se os campos estão em branco, etc...

Validações do Command - CQRS


Como consultar valores de Data em uma conexão C# para excel usando OleDb?

Question: Segue abaixo a linha de código
private void button7_Click(object sender, EventArgs e)
        {
            string conex = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + textBox1.Text + ";Extended Properties=\"Excel 8.0;HDR=YES;\";";
            OleDbConnection conn = new OleDbConnection(conex);

            OleDbDataAdapter myDataAdapter = new OleDbDataAdapter("SELECT * FROM  [" + textBox2.Text + "$] where DATA ="+ dateTimePicker1.Value , conn);
            DataTable dt = new DataTable();

            myDataAdapter.Fill(dt);
            dataGridView1.DataSource = dt;
}

tento retornar um valor fazendo busca pela Data, porem só encontro erros.
Sem aspas simples(como no codigo), retorna erro =

System.Data.OleDb.OleDbException: 'Erro de sintaxe (operador faltando)
na expressão de consulta 'DATA =18/12/2020 08:20:04'.'

Quando ponho aspas simples na consulta em torno da dateTimePicker da erro =

System.Data.OleDb.OleDbException: 'Tipo de dados imcompatível na
expressão de critério.'

Comment: Não, DATA é o nome da coluna os esta todas as dates, e que quero buscar os valores.
Answer: Seu banco de dados pode não está entendendo o formato da data. Normalmente se trabalha com o formato YYYY-MM-DD.
Inclua o método ToString em seu dateTimePicker1.Value explicitando em qual formato a data deve ser escrita na consulta.
OleDbDataAdapter myDataAdapter = new OleDbDataAdapter("SELECT * FROM  [" + textBox2.Text + "$] where DATA ="+ dateTimePicker1.Value.ToString("yyyy-MM-dd HH:mm:ss") , conn);

Comment: O meu banco de dados que estou usando é uma planilha do excel, o formato é dd-MM-yyyy HH:mm:ss .

agora ele até roda, mas não retorna nenhum valor
Comment: Então o erro foi resolvido. Agora a atenção tem que ser dada aos valores do banco e do critério. Ao usar o operador igual, garanta que o valor procurado é idêntico aos que existem no banco. No caso de DateTime isso inclui até os milésimos de segundo.
Comment: Consegui, inclui um "like" na consulta para que buscasse os demais valores após a data.
Muito Obrigado!!!
Comment: Legal! Se essa solução lhe ajudou contornar o erro, marque essa resposta como aceita que ela seja concluída. Abraço!


The test case is not recognized by the Simpletest module

Question: I am using Drupal 7, and the latest version of the Simpletest module. I have written a test case like the following one:
class SimpleTestDemoTestCase extends DrupalWebTestCase {
  function setUp($modules = array()) {
    parent::setUp('simpletestdemo');
  }

  public function getInfo(){
    return array(
      'name' => 'Simpletestdemo testcase',
      'description' => 'Simpletestdemo testcase desc',
      'group' => 'Simpletestdemo'
    );
  }

  public function testBlockView() {
    $simpletestblockview = module_invoke('simpletestdemo','block_view);
    $this->assertTrue(t('Simpletest Demo'),$simpletestblockview['title']);
  }
}
?>

This does not get recognized by the Simpletest module. Is there anything wrong in this test case?
Comment: did you include this file in .info files[] ?
Answer: As srihita pointed out in a comment, you need to add the file which contains the test class to the files[] array in your module's info file like this:
files[]=yourmodule.test

Then, let it rebuild by submitting the modules page or use the "Clear information" (or similar, not sure about the exact label right now) at the bottom of the Testing page.
Also, you're code is missing a ', but that might be a copy/paste error.
Comment: Saw this question when looking for a related problem. For the next person looking for an answer:
If the simpletest contributed module (version 2.x) is installed and enabled (as opposed to the D7 simpletest core module), you need to add the testing_api = 2.x to the module.info file, otherwise simpletest will not recognize your test.
Answer: I am also facing same problem.
The test case is not recognized by the Simpletest module.
i have added files[] = jobs.test in info file
Jobs Tests is visible in http://localhost/drupal7/admin/config/development/testing
 'Jobs Tests',
            'description' => 'Unit Tests for this module',
            'group' => 'Jobs',
            );
  }
function setUp() {
    parent::setUp();
  }
/**
    public function jobrecivedapplicationTest() {
    $result = jobs_recieved_applications();
    $this->assertNOTNULL($result, 'should retrieve all the informtion');
  }
*/
  function jobrecivedmailTest() {
    require_once(jobsssssss.module);
    $result = simpletest2();
    // $this->assertNOTNULL($result, 'should retrieve all the informtion');
    // assertMail($name, $value = '', $message = '')
    $this->assertMail($result, "firstname.lastname@example.com", 'The last message subject was "My latest email"');
  }
}
Comment: This is not the answer. Please use comments.


C# in the key signature

Question: I am looking at a piece of music and there is a C# in the key signature. On the cello there is a C note on the G string and on the A string. I know that in a C# key signature the C on the A string is a C# but I want to know if the C on the G string is a C# as well.
Comment: If there's a C# in a key signature, that means when reading the music, whenever you see a C notated, play a C# instead, unless there's a natural sign in front of the C. It doesn't matter what string you are playing or even what instrument you are playing.
Comment: @ToddWilcox  -  I could post an answer but there is really not much I could add to your comment.  Why don't you just make it an answer before someone else copies and pastes your comment into an answer.  Maybe in the answer version you throw in the term "accidental" to describe how a composer would notate a deviation from what is indicated in the key signature.
Comment: I feel like this question is a duplicate actually, I just haven't taken the time to find the dupe.
Answer: Yes, if the key signature included C#, it affects ALL C notes.  No need to look for esoteric cases where a composer may have invented his own private system where something that LOOKS like a key signature has a non-standard meaning, though I'm sure some could be found.  A key signature affects all octaves.
I guess there's an F# in the key signature too?
Comment: Yes. There is an F# in the key signature as well. The songs I play at my current level don't usually use the C string where the other F is where I would play the F# on that string
Answer: The position of the sharps and flats on the key signatures is traditional and, generally, seems to have been "designed" to fit within the stave. If C# is indicated in the key signature then all C notes are sharp unless indicated otherwise during the piece.
On the cello you can play a C# on one or more strings - depending on the pitch. Which one you use will depend on the left-hand position and the preceding and following notes. There will be a difference in tone though and this will be more and more detectable with experience.
Answer: There must be an F# at the beginning followed by C# so we're on the key D. Whenever you see a C note, it means to play it in sharp unless you see a natural sign. And when you see such, it will only be effective for 1 bar.

C# in the key signature


Como adicionar um calendário na Frame ou Painel no Java

Question: Estou desenvolvendo um projecto em Java que lida com datas. A idéia principal é colocar um calendário de tamanho médio na tela principal, as datas que já tiverem um evento agendado deveram possuir uma coloração (azul, laranja , etc) e ao passar o mouse sobre o menu deverá mostrar algo como um pop-up e mostrar uma descrição breve do evento.
O projecto está a ser desenvolvido usando JavaFx combinados com elementos da Swing. As minhas pesquisas só me deram o JCalendar que não satisfaz as minhas necessidades.
Qualquer dica é bem vinda.
Comment: Sobre componentes relacionados a data/calendário: JavaFX tem o [`DatePicker`](https://docs.oracle.com/javase/8/javafx/user-interface-tutorial/date-picker.htm), não precisa utilizar nada de terceiros. Pode utilizar os métodos para colorir os dias com eventos agendados e o evento de "click" para buscar o evento agendado em determinado dia.
Comment: Vou tentar usar o DatePicker, mas será que você poderia me apontar alguns links com bons exemplos, ou poderia elaborar algum?
Answer: Basicamente você terá que implementar um dayCellFactory, isso te permitirá manipular como as celulas do calendário serão renderizadas. Exemplo da documentação comentado:
<code>DatePicker dp = new DatePicker();

dp.setDayCellFactory(new Callback<DatePicker, DateCell>() {

    @Override
    public DateCell call(DatePicker arg0) {
        return new DateCell() {
            @Override
            public void updateItem(LocalDate item, boolean empty) {
                // Chamada obrigatória ao renderizador da superclasse
                super.updateItem(item, empty);

                // Aqui estamos trocando a cor do background de um dia específico
                if(MonthDay.from(item).equals(MonthDay.of(9, 15))) {
                    setStyle("-fx-background-color: #ff4444;");
                }
                // Aqui estamos desabilitando a data do dia seguinte
                if(item.equals(LocalDate.now().plusDays(1))) {
                    setDisable(true);
                }
            }
        };
    }
});
</code>
Fica assim após a execução:

Usando essa lógica você pode implementar uma classe chamada Evento, com um LocalDate e uma descrição como atributos, e percorrer um array de eventos dentro da factory pintando-os de acordo.
Já a questão do PopUp você pode usar o método setTooltip que este componente herda da classe Control (O tooltip é meio bugado). Ou pode usar o PopOver do ControlsFX, uma biblioteca bem difundida de componentes customizados para JavaFX.
Comment: Obrigadão Gustavo. Vou ler a documentação dos elementos que te referiste e em caso de problemas vou deixar meu comentário. E ainda não vou marcar sua resposta como melhor resposta para que mais pessoas respondam.


How can I filter elements in two lists with C# and .NET 6?

Question: I have 2 lists like this, what I want to do is to take the non-identical values in these 2 lists into a third list, how can I do this with C#?
List<int> listOne = new List<int>() { 112, 113, 114 };
List<int> listTwo = new List<int>() { 112, 113, 114, 115, 116 };

List<int> filteredList = new List<int>();

Exactly what I want to do is to pass the unmatched values in listOne and listTwo into filteredList. So 115 and 116 values need to be passed into the filteredList. How can I do this?
Comment: @Sasha: it depends on what should we do with duplicates. If we don't have any or if we should operate with distinct values, then sets can be a solution.
Comment: What should we do with *duplcates*? E.g. what is the expected result for `listOne = {1, 1, 2, 2, 2, 3}` and `listTwo = {2, 2, 3, 3, 4}`?
Comment: `list2.Except(list1)`?
Comment: Then it will be a little more complex. Like maybe `list1.Except(list2).union(list2.Except(list1))` or something similar. At which point it may be worthwhile to think about a non- linq solution.
Comment: Maybe rewrite with Sets and use Except? https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/linq/set-operations#except-and-exceptby
Comment: You have asked 12 questions in 2 months. Can you try searching before asking a new one, and show what you have tried?
Comment: The expected result for this list is that there are only `4` in the `filteredList`.
Comment: @imuzz01: since you don't want values in `listOne` and you want *distinct* values only, then use *Fildor's* solution: `filteredList = listTwo.Except(listOne).ToList();`
Comment: Is it possible that list1 contains more items than list2 and then you want those? What about "1, 2, 3" and "1, 2, 4" - would then "3,4" be expected?
Comment: Yes, these lists will come as variables and may contain different items from each other. When I use Except, we filter from the list with excess items, but in the opposite case, this code does not work, it returns me an empty list. @Fildor
Answer: Scanning for duplicates is a very costly operation, something like O(n^2) which is extremely bad. As such you would want to employ a HashSet for this which would reduce the time complexity to something close to O(n) which is far better:-
            List<int> listOne = new List<int>() { 112, 113, 114 };
            List<int> listTwo = new List<int>() { 112, 113, 114, 115, 116 };

            //Create a HashSet with the values from the bigger list
            var filteredList = new HashSet<int>(listTwo);

            //Remove items from the HashSet that also exist in the smaller list
            filteredList.ExceptWith(listOne);

            // Print the filtered list to the immediate window
            foreach (var i in filteredList) Debug.WriteLine(i.ToString());

The above code outputs this:-
115
116


How to improve rare event binary classification performance?

Question: I am building a binary classification model to predict patient admission with respiratory issue in R. Each row in my data set is a patient record. The dependent variable is admit or not(1 or 0), and the features including age, gender, weather info, and air quality info. All the variables are in numeric type. The data set contains 70,000 records with admit rate around 3%.
I searched online for possible technique to deal with rare event problem. Like using xgboost, or resampling data set and combine with ML algorithms.
I am using three algorithms to compare the results.
1. xgboost. My codes are:
   pos_weight <- sum(training_df$resp_admit==0)/sum(training_df$resp_admit==1)
   xgb_mod <- xgboost(data = dtrain,
                      eta = 0.01,
                      max_depth = 9,  nround=3000, nthread = 2,
                      subsample = 0.9, colsample_bytree = 0.9,
                      eval_metric = "error", eval_metric = "auc",
                      objective = "binary:logistic",
                      max_delta_step = 6,
                      scale_pos_weight = pos_weight,
                      verbose = 1)

2. logistic regression and decision tree with resampling method(ovun.sample with method = both)
However, with parameter tuning and resampling method, the classifier performance is still bad for me.
   metric      logistic regression     xgboost
Sensitivity       0.0240480962         0.34482759
Specificity       0.9703949693         0.61143868
Pos Pred Value    0.0278422274         0.02945508
Neg Pred Value    0.9657548696         0.96465116
Precision         0.0278422274         0.02945508
Recall            0.0240480962         0.34482759
F1                0.0258064516         0.05427408
Balanced Accuracy 0.4972215327         0.47813313
AUC               0.4901495211         0.45190509

I am new to rare event problem. My questions are:

I know the standard for the performance is different for different problem. But is there a overall standard? Like at least recall or precision should above 0.5?
What other techniques can I try to improve the performance?
Did I miss anything I should pay attention when I built the model?
I randomly picked some basic features that I thought are important to build a simple model. Should I also do feature engineering? Does this step matter a lot to the model performance?

The following is how I constructed my data set.

From all the patient admission records, picked out the ones admitted with respiratory issues. These records are associated with admission date. One patient may have multiple records at different admission dates. (Patients with multiple admissions count 15% among all the patients.) Labeled dependent variable for these records as 1.
Group the above records by year-month. For each day in a month, if the patient didn't admit on that day, replicates patient info, and label as 0. Do this for all the patients fall in that month, and repeat the procedure for each different year-month.
The reason I didn't generate 0 records across the whole time period is that if I did so, the rare event rate will be around 0.1%.
Combine all the 1 and 0 records, left join the weather and air quality info by date.

I am also concerning about the way I constructed my data set. Discussions about this are welcomed.
The final goal of this model is to make respiratory admission risk predictions on a patient list, based on patient info and current day weather and air quality info.
Comment: Did you try SMOTE? There is a related question about ROSE vs SMOTE. https://stats.stackexchange.com/questions/166458/rose-and-smote-oversampling-methods
Comment: @LmnICE Thank you for the article. I used to deal with unbalanced problems with the methods mentioned in the article. However, these methods didn't solve my issues with this problem.  I assume that my problem is more complicated, other than a simple imbalanced problem.
Comment: Thinking about it you are also only looking at re-hospitalizations within 1 month, if you only consider days wi t hin 1 month (i.e. first discharge day + 28 or 30 or so days - I hope you are not looking at the calendar month of the hospitalization date). If that's what you want each new first hospitalization outside that period cannot count, but starts a new at risk period. I suspect you really have a serious issue with having a clear question and a data selection that is relevant to the question.
Comment: Is your data really binary? If we have weather and air quality, that's presumably for a particular day or week? Or are intervals of unequal length?  If so, do you have more than one record per patient?
Comment: @Björn I have added the description of how I constructed my data set at the end of my question.
Comment: [This article](https://elitedatascience.com/imbalanced-classes) compares 5 methods to handle unbalanced classification problems. It's very didactic.
Comment: What about patients that never get admitted? Are you somehow taking into account that records are from the same patient?
Comment: @Bjorn I don't have information for people that never get admitted. (But I do have information for patients admitted for other issues. ) For patient admitted more than once, I treated each admit as a record in my data set. Could you explain your concern and suggestion?
Comment: I fear your risk set is completely weird - a first hospitalization is guaranteed! This also makes the people you look at be at a more similar risk, making your task harder. When you think of who the estimated risk would apply to, I suspect it is nobody you could identify up-front. What may make sense is to look for the risk of re-hospitalizations after the first one (counting days after being released from the hospital as at risk).
Comment: Secondly, some patients will be intrinsically at a higher risk than others due to things you have not measured. If you ignore that and your model treats days at risk from the same patient as independent observations (rather than explicitly allowing for a correlation - perhaps patient history variables would also work?), this is another factor that almost must lead to difficulties assessing the risk.
Comment: @Bjorn Yes, my data set needs to be reconstructed. Thank you for the explain. It helped me to learn how to construct a proper data set.
Comment: There is something called "Imbalance Data" in Machine Learning, in that we would be using techniques like SMOTE... etc., to the training data before applying ML algorithms for more details, please take a look at this [link](https://www.analyticsvidhya.com/blog/2017/03/imbalanced-classification-problem/), a detailed explanation is provided there.
Answer: Casting this as a classification problem was a major misstep.  This is inherently a "tendency estimation", i.e., probability estimation problem.  That is what logistic regression is all about.  And you've chosen improper accuracy scores - scores that are optimized by choosing the wrong features and giving them the wrong weights.  For details see http://www.fharrell.com/2017/01/classification-vs-prediction.html and http://www.fharrell.com/2017/03/damage-caused-by-classification.html
Comment: I have read the two articles. They are very inspiring to me. I can't fully understand  the knowledge and techniques in the articles currently, but I will keep learning. You mentioned different logistic regression methods in your article's comments. I was wondering for imbalanced data set, is penalized likelihood logistic regression better than resampling method? Could you suggest more on this kind issue?
Comment: When I performed logistic regression and xgboost, the output probability are very small. The median of the probability is around 0,05. For this case, how to define a standard for high/low risk properly?
Answer: In addition to Frank Harrell's important point about classification versus prediction, you might need to consider that you don't have the information needed to judge the probability of admission. AUC is the one measure in your list that isn't subject to arbitrary choices of cutoffs for classification, and it is very close to the value of 0.5 seen when your model is no better than random. It might just be too hard to determine the precise day when an individual will be admitted, particularly for a first admission, which seems to be what your data-set construction and analysis are examining.
Dealing with re-admission characteristics might be possible with a survival model. The starting date in each case would be the date of discharge from the first admission. Then you could incorporate weather and air-quality measures as time-dependent covariates in a model of time-to-readmission.
Comment: Could you explain the reason _you don't have the information needed to judge the probability of admission_?
Comment: Logistic regression provides perhaps the best approach for predicting admissions. Yet the AUC with the predictors you included in the logistic model showed no better performance than chance. The simplest explanation is that the predictors you included in your model don't really help predict admission probability, at least in the way that you modeled it.
Comment: If I perform analysis on readmission problem, besides survival analysis, is logistic regression still an option(if the AUC score is much better than 0.5)?
Comment: For analysis of the time it takes for an event to occur (like time to re-admission), survival analysis is typically better and provides more power. See [this answer](https://stats.stackexchange.com/a/250868/28500) for example.


How to use awk to extract columns from a file based on a key?

Question: I have a file of keys, file1, (each key on a new line) that I need to use to extract certain columns from a second file, file2.  File 1 is 46 lines long, while file2 is much larger, >20,000 lines long.  Not all keys that appear in file1 appear in file2, and vice versa.
file1:
322510472
322510472
322510472
322510484
322510484
322510484
322510493

file2:
109287879,Invertebrate_iridescent_virus_3,109287879,148,1,148,NCVOG0391,0
109287880,Invertebrate_iridescent_virus_3,109287880,458,1,458,,
109287881,Invertebrate_iridescent_virus_3,109287881,156,1,156,,
109287882,Invertebrate_iridescent_virus_3,109287882,451,1,451,NCVOG1423,0
109287883,Invertebrate_iridescent_virus_3,109287883,217,1,217,NCVOG4910,2
109287884,Invertebrate_iridescent_virus_3,109287884,494,1,494,NCVOG0211,0
109287885,Invertebrate_iridescent_virus_3,109287885,447,1,447,NCVOG1077,0
109287886,Invertebrate_iridescent_virus_3,109287886,347,1,347,NCVOG0967,2

Both file1 and file2 are sorted by the key, which appears in columns 1 and 3 of file2.
I need to produce a third file, file3, which contains my keys from file1, as well as columns 2 and 7 from file2, and which does not omit any keys present in file1, even if there are no matching entries in file2.
I know that I have 46 entries in my file of keys, file1.  However, when I use the following awk script,
awk -F"," 'NR==FNR {a[$1]=$1 FS $2 FS $7; next} $1 in a {print a[$1],$2,$7}' file2 file1

I only see 44 lines of output.
I need to not delete any of my keys in my awk output, as they correspond to actual data I need to keep in other files also containing those keys.
Any suggestions?  Thanks for any and all help; I've been reading stack overflow for a while but this is my first time asking a question!
Comment: `$1 in a` condition will only be true for entries in both files, you need to print also the ones that didn't match.  If keys are unique, you can remove them from `a` after each match and iterate over the remaining keys in the `END` block.  You may want to sort the results afterwards as well.  However `join` is a better solution here since data is already sorted.
Comment: You need to provide sample data that actually can represent your problem
Answer: your data doesn't have any matches, so I modified your keys to include matching entries
$ join -t, -a1 -o1.1,2.2,2.7 file1 file2

109287879,Invertebrate_iridescent_virus_3,NCVOG0391
109287880,Invertebrate_iridescent_virus_3,
109287882,Invertebrate_iridescent_virus_3,NCVOG1423
109287884,Invertebrate_iridescent_virus_3,NCVOG0211
109287886,Invertebrate_iridescent_virus_3,NCVOG0967
322510472,,
322510472,,
322510472,,

here is the file1 I used instead.
109287879
109287880
109287882
109287884
109287886
322510472
322510472
322510472


What model to use for a 'spraying experiment'?

Question: I separate my plants into four groups of 16, and I spray them to keep the bugs away:
group 1: I spray daily
group 2: I spray every 2 days
group 3: I spray every 3 days
group 4: I spray every 4 days

at the end of the 12 days, I check number of plants that's bug free
group 1 is: 12
group 2 is: 10
group 3 is: 7
group 4 is: 6

How should I fit a model to this?
Comment: We need informative title, so I updated yours; feel free to choose another one. It looks like you're after some kind of a dose-response relationship (with counts or % as an outcome).
Answer: First you should get your data into R! so create a variable for your response (number of bug free plants - or you might want to consider having this as a bug free proportion of the total), and a variable for your factor (ie sprays per day is probably the best way).
Then plot those two variables.  There's probably an obvious relationship.  The trick then is to somehow model that...
If your response is a proportion the best model is probably some form of logistic regression.  You should also think carefully about the best form of your explanatory variable; with most ways of going about modelling, you would like this to have a linear relationship to some transformation of the response variable.  Read up on logistic regression, and good luck!


Performance concerns about monitor incoming email for multiple accounts on the server

Question: I want to write a Server side script / daemon which would monitor multiple email accounts (might become quite a big number) and then send push notifications . My conceptual idea till now is: have a database with accounts and passwords. Iterate through that, check if any new messages are there and then react on that by doing smth with an email and sending push notification to the mobile device of the Client. My biggest concern is perfomance. Looping through thousands accounts doesnt seem right to me , but I cant come up with better solution. Registering an Observer for each account doesnt sound any better..
Any ideas? Im open to any languages (Scripting or programming). Not asking for code, just trying to wrap my head around the concept.
Thank you!
Comment: Why do you think registering an Observer is no better than looping through all accounts on schedule?
Comment: Sorry, didn't get notification before. Well, there could be a loot of accounts and registering an observer for each sounds a lot.. what is important to me is that accounts don't wait too long - "real time" behavior kinda. And when each observer starts doing smith one by one - how is it different? Unless I use multi-threading its all the same. But with multi-threads I would probably go with answer from santimviejo. Open to suggestions. Thanks!
Answer: You could do it by blocks. Going one by one through all your database entries may take a long time if we are talking of thousands of accounts, maybe you could divide it on several scripts or script executions, taking, lets say, blocks of 100 accounts. So you would have an environment like this: script/thread 1 checking accounts from 1 to 100, script/thread 2 checking accounts from 101 to 200...
This could be done with threads on the same script/program, by using different scrips, or by using a "wrapper" to call the script many times as needed, depending on the amount of entries/blocks. You may need to keep an eye on you server resources, but performance of the checks should improve.
Hope this helps.
Comment: Thanks! Was trying to avoid multi threading, but seems like that's all that is left ... main script starting n/100 or so times. Definitely now speed vs server power..


Does the following relation holds for infinite sums?

Question: Hey i am only wondering if it is possible that the equation below holds:
$$
\sum\limits_{i=0}^{\infty} a_i(b_i - c_i) = \sum\limits_{i=0}^{\infty} (a_ib_i - a_ic_i) =\sum\limits_{i=0}^{\infty} a_ib_i - \sum\limits_{i=0}^{\infty} a_ic_i
$$
Thanks!
Comment: You appear to have a typo: I think that you mean $$\sum_{i=0}^\infty a_i(b_i-c_i)=\sum_{i=0}^\infty(a_ib_i-a_ic_i)=\sum_{i=0}^\infty a_ib_i-\sum_{i=0}^\infty a_ic_i\;.$$ Yes, provided that the three series all converge.
Comment: No, you haven't. $a(b-c)=ab-ac$ and not $ab-bc$.
Comment: Should still be $a_ic_i$ not $b_ic_i$.
Comment: It was a typo yes. I fixed it.
Comment: I did fix that thanks!
Answer: Recall the definition of a series is: $$\sum_{i=1}^\infty a_i=\lim_{n\to\infty}(a_1+\ldots+a_n).$$
The answer to your question follows from the fact that: $$\lim_{n\to\infty}a_n(b_n-c_n)=\lim_{n\to\infty}a_n b_n-\lim_{n\to\infty}a_n c_n,$$ whenever all the limits have a finite value.
Comment: @GitGud: Finite in the sense that the actual limit of the series is finite. Not the index set.
Comment: A small misunderstanding on my part. Sorry.
Comment: @GitGud: You should have left the comment, it was a good remark that made the answer slightly better understood. But I suppose a small edit would be even better.
Answer: It holds when
$$\sum_{i=0}^\infty a_i b_i$$
and $$\sum_{i=0}^\infty b_i c_i $$
ar both unconditional convergent.
Or $$
\sum_{i=0}^\infty a_i (b_i - c_i)$$ is unconditional convergent.
This is in the finite dimensional the case when they are absolute convergent, that means when
$$\sum_{i=0}^\infty |a_i b_i|$$ is convergent.
Comment: Do you really need absolute convergence?
Comment: Do you have an example for when this fails for a conditionally convergent series?
Comment: Interesting. I thought that $0=0$.
Comment: but $\infty -\infty$ is not zero, it is indeterminate
Comment: But then $\sum a_ib_i=\infty$. This is not about conditional convergence anymore. It's just not a convergent series.
Comment: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/7977/discussion-between-dominic-michaelis-and-asaf-karagila)
Comment: I have no intentions using the chat system.
Comment: $$0=\sum\frac{(-1)^n}n(1-1)=\sum\frac{(-1)^n}n-\frac{(-1)^n}n=\sum\frac{(-1)^n}n-\sum\frac{(-1)^n}n=0$$
Comment: @Asaf oh right sry take a_i = 1/n and b_n = 1,0,1,0,1,0,1, ... and c_n = 0,1,0,1,0,1,0,1,... yeah sry my fault
Comment: Okay. I see your point now. But it is enough to require that all three sums are evaluated to a finite number, rather than two of them are unconditionally convergent.
Comment: normally at such a stuff you need unconditional convergence, would be surprising for me if this one is different, let me think of a counter example