How to send mail for every X months using rules?

Question: In my user profile there is date of orientation(date field) from date(current date) and end date.
When user fills the date field, it should generate an Mail for every X months.
                  example-according to screen shots i have two date field when user have given from date(current date or future date)based on that date for both date fields should trigger a mail every X months.
Comment: `"When user fills the date field, it should generate an Mail for every two months."` two months from which date? from date? or end date?
Comment: @ NoSssweat - thanks for your response,based on from  date(from date is current date)
Comment: how did u add the from & to date fields? did u install a special module to do that? or are they two seperate fields with separate machine names?
Comment: @ NoSsweat - it is two seperate date fields,when i have fill  from date automatically end will be generate.
Comment: @DINE Please don't add random images to your posts - if the images you recently added somehow enhance your question, you need to let people know what that is. They just look like random image dumps of your browser console window, which obviously won't relate to sending an email via rules
Comment: @ Clive - sorry clive to post here my images.
Answer: Requirements:
Rules, Rules UI and Rules Scheduler must be installed and enabled.
Step 1) Create a new component, I named it send mail every 2 months

Step 2) Add an action of Send Mail, fill out the form, see picture below.

Step 3) Add an action (again)
This time select Schedule Component Evaluation and select your component (send mail every 2 months)

Note: this will re-schedule your e-mail to be sent 2 months after its
  sent for the 1st time

Step 4) For SCHEDULED VALIDATION DATE, erase everything and put +2 Months
For Identifier, put something like I did (see picture) so you know who the schedule is for.
For ACCOUNT, put account

Step 5) Create a new rule, for React on Event, select After saving a user account.
Step 6) Add an action, select Schedule Component Evaluation and select your component (send mail every 2 months)
Step 7) Click on the switch to data selection button

Step 8) Put your field in Scheduled Evaluation Date. My user date field is called From Date with a machine name of field_from_date Therefore, it's account:field-from-date
Click on Add Offset, put 2 and select Months. (This means it will send the e-mail 2 months from the from date.
For Identifier, put something like I did (see picture) so you know who the schedule is for.
ACCOUNT, put account
Comment: Please don't vandalise your posts, regardless of whether you think it's received enough votes; that's not something you get to decide - either provide the answer with good nature and intention, or don't provide it at all. Thanks
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackexchange.com/rooms/29709/discussion-on-answer-by-no-sssweat-how-to-send-mail-for-every-x-months-using-rul).
Comment: @DINE [click here](http://chat.stackexchange.com/rooms/29709/discussion-on-answer-by-no-sssweat-how-to-send-mail-for-every-x-months-using-rul) to chat
Answer: Refer to my answer to the question about "Rule to store user's last post date and scheduled rule based 30 day reminder", which is a variation (though not a duplicate) of your question here. Below are some more details about the steps you need to complete to make it work for your case also.
Step 1
Your to date already corresponds to field_next_article. Let's assume the fieldname of your to date is field_to_date.
Step 2
Copy the Rules component example from Step 2 (in export format already) to some temporary file, and perform these changes:

change field name field_next_article to field_to_date.
change the eMail details (such as Subject, etc) to fit your requirements.

Then import the edited version in your own site.
Step 3
Copy the Rules example from Step 3 (in export format already) to some temporary file, and perform these changes:

change field name field_next_article to field_to_date.
change 30 days to 2 months, and review (correct) the value for date_offset (to also make it match 2 months).
Replace the "Rules Event" (= the line starting with "ON", by "some event that fits your exact requirements".

Then import the edited version in your own site.
Step 4
To actually sent such eMail every 2 months, you need to add another action at the end of the Rules component you created in Step 2. That action is to repeat the execution of the Rules component itself (which was created in Step 2), similar to what is explained in the video about "Repeating scheduled events" (especially what is shown from about 7:00 to 8:00 in that video).
Note: If you're not familiar (enough/yet) with Rules, checkout the 32 (!!!) great, and free, video tutorials Learn the Rules framework.
PS: I'm assuming this question is about D7.
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackexchange.com/rooms/29224/discussion-on-answer-by-pierre-vriens-how-to-send-mail-for-every-two-months-usin).


Clarinet extension in Wagner's Tannhauser

Question: A.S. I'm sorry if this question may seem stupid. I've been playing a the viola since years in many orchestras but I'm still a complete ignorant in the field of woodwinds.

I've been trying to transcribe Wagner's Tannhauser in East West Symphonic Orchestra.
This VST limits the extensions of every instrument by default: while writing the clarinet part I've noticed I couldn't play any note below D2. In the VST, instruments extensions are very accurate when it comes to string, so I assume they're correct for woodwinds also and therefore D2 is the lowest note you could play with a clarinet.
However, take a look at this:

Let me explain the problem. The whole ouverture is in E major and the first note is a B2 for all the instruments that play.
As soon as I saw the notes, I noticed they need to be transposed. In the case of the photo above, they need to go down 3 semitones.
However, if you do that you'll notice that the lower part will go below D2 (and I can't play below D2!).
Why not transpose the clarinet by +9 instead of -3? Perhaps you're just getting the wrong octave? Nope. Unless I'm becoming deaf, there's just one note being played at the beginning and that note is B2 (not B3) so the octave I'm trying to achieve is the correct one. Here, check by yourself: Tannhauser on Youtube
So, what's going on? Is Wagner fooling with the woodwinds? Is the VST fooling with me? Is the score wrong?
Being a string instrument player I've always been in a what you see is what you play mindset. Can someone explain me how does the transposition above works?

EDIT: Yep, according to this page I'm right. Clarinet in A means -3 semitones. How then is even possible to play the part above if a clarinet doesn't go below D2?
Comment: The A clarinet goes down to Db. Are you perhaps notating for a Bb clarinet?
Comment: It looks like East West VST provides you with sample for a not better specified "Clarinet". This sample only goes down to D2 unless hacked into thinking he's playing above D2...
Answer: This is a good question as transpositions notoriously trip people up.
First off, Wagner's not playing any tricks, and you've got the right octave, so rest easy. :)
The "normal" Bb clarinet's lowest written note is E2 which will sound as you described one whole-step lower than written pitch (D2.)
The A clarinet has the same written range as the Bb clarinet, but sounds a minor third (or three semitones) lower than written pitch.  Thus, with an A clarinet, a low C#2 is possible that is otherwise not possible on the Bb clarinet.
It is considered standard practice for an orchestral clarinettist to own (and bring) both a Bb and an A clarinet with them.  Many also have bass clarinets (especially if they are 2nd or 3rd chair) and Eb soprano if necessary (again, if they sit second chair.)
Unfortunately, the VST doesn't know any of this, and doesn't distinguish between different transpositions since it's not a real instrument anyway.  The problem therefore lies in the fact that the music is written correctly but due to transposition, it is out of the range for the VST, so the VST is incapable of playing the notes.
As a composer, I run into these issues all the time with different instruments, sounds, and effects on a daily (and hourly) basis.  I think that composers do just as much "imagining" currently as composers of old, but that's a subject for another time.
My solution to this problem is to use a MIDI patch instead of a fancy VST.  Sometimes labeled something like "Smartmusic Synth," the MIDI will play any note entered - regardless if it is actually possible on a real instrument.  I find such changes to be helpful when writing things that I know are possible, but are beyond the capability of the computer.
Also, make sure that your staff / instrument's transposition is configured correctly - that will also help with playback.
Hope that helps.


Number of permutations with given cyclic structure

Question: If $\sigma$ is a permutation made up by the disjoint cycles $\tau_1, \dots, \tau_r$ (including those of length $1$), we call structure of $\sigma$ $$(l_1, \dots, l_r),$$ where $l_1, \dots, l_r$ are the lengths of $\tau_1, \dots, \tau_r$ (supposing we ordered them in such a way that $l_1 \ge l_2 \ge \dots \ge l_r$).

The number of permutations $\sigma \in S_n$ with structure $$(\underbrace{l_1, \dots, l_1}_{k_i \text{ times}}, \dots, \underbrace{l_m, \dots, l_m}_{k_m \text{ times}} ),$$ where $l_1 > \dots > l_m \ge 1$ and $h_1l_1 + \dots + h_ml_m = n$, is
$$\frac{n!}{\prod_{i=1}^m { {k_i}! {l_i}^{k_i}}}.$$

How does one prove this statement? I'm only aware of the proof that there are $$\frac{1}{l}\frac{n!}{(n-l)!}$$ $l$-cycles in $S_n$.
Answer: Suppose you are given a blank structure
$$
( \cdot \cdot \dots \cdot) \cdots ( \cdot \cdot \dots \cdot)
$$
of the given shape (partition).
Start filling the blanks in all possible ways. There are clearly $n!$ ways of doing this.
But of course you have overcounted.
First, a cycle of length $l$ can be written in $l$ different ways
$$
(1 2 3 \dots l) = (2 3 \dots l 1) = (3 4 \dots l 1 2) = \dots = (l 1 2 \dots l-1).
$$
You can do this for every cycle, so this accounts for the factor
$$
\prod_{i=1}^{m} { {l_i}^{k_i}}
$$
in the denominator.
And then, the $k_{i}$ cycles of length $l_{i}$ can be permuted in all the possible $k_{i} !$ ways without changing the permutation, as they are disjoint and thus commute. This accounts for the factor
$$
\prod_{i=1}^m { {k_i}! }
$$
in the denominator.


Passing list as an argument

Question: I seem to have an unexpected issue with objects' references. There is a draft of what my code does:
// file 1
List list1;
List list2 = function(list1);
//file 2
public static List function(List input)
{
    list output = new List();
    for (int i ...)
    {
        output.Add(new B(input[i]));
    {
    return output;
}
// file 1
list1[1].FieldOfAObject = 10; // I change an object from list1
WriteLine(list2[1].objectA.FieldOfAObject); // it is not changed

It would seem to me, that as list1 is a class and A is a class and they are passed by a reference, objects of type B created in a functionand assigned to list2 should keep a reference on objects of list1.
EDIT
Real code:
public void PrintCylinder(Camera camera, Cylinder cylinder, Bitmap Bitmap, double bitmapWidth, double bitmapHeight, PrintArguments pa)
    {
        ...
        List<AffineCoordinates> verticesToPritOnScreen = new List<AffineCoordinates>(); // out list1
        foreach (var vertex in cylinder.Mesh.MeshVertices)
        {
            AffineCoordinates newVertex = PVM * vertex;
            verticesToPritOnScreen.Add(newVertex); // list1 is being filled here
        }

        List<Face> FaceList = MeshGridGenerator.GenerateCylinderGrid(verticesToPritOnScreen, cylinder.D); // our list2

// CHANGES BENEATH ARE NOT VISIBLE IN FaceList objects
        for (int i = 0; i < verticesToPritOnScreen.Count; ++i)
        {
            // project to NDC
            verticesToPritOnScreen[i] = verticesToPritOnScreen[i] / verticesToPritOnScreen[i].W;
            // project to screen space
            verticesToPritOnScreen[i].X = (verticesToPritOnScreen[i].X + 1) * bitmapWidth / 2;
            verticesToPritOnScreen[i].Y = (verticesToPritOnScreen[i].Y + 1) * bitmapHeight / 2;
            verticesToPritOnScreen[i].Z = (verticesToPritOnScreen[i].Z + 1) / 2;
            verticesToPritOnScreen[i].W = 1;
        }

        ...
    }
// our function (from MeshGridGenerator class)
public static List<Face> GenerateCylinderGrid(List<AffineCoordinates> vertices, int D)
    {
        List<Face> FaceList = new List<Face>();

        for (int i = 1; i <= D; ++i)
        {
            // upper base
            FaceList.Add(new Face(vertices[0], vertices[i], vertices[i % D + 1]));
            // bottom base
            FaceList.Add(new Face(vertices[vertices.Count - 1], vertices[i % D + 1 + D], vertices[i + D]));
            // side wall's upper triangle
            FaceList.Add(new Face(vertices[i], vertices[i + D], vertices[i % D + 1]));
            // side wall's bottom triangle
            FaceList.Add(new Face(vertices[i % D + 1], vertices[i + D], vertices[i % D + 1 + D]));
        }

        return FaceList;
    }
// our B class
public class Face
{
    public AffineCoordinates[] vertices;

    public bool AreVerticesClockwise
    {
        get
        {
            double sum = 0.0;
            for (int i = 0; i < 3; ++i)
            {
                sum += (vertices[(i + 1) % 3].X - vertices[i].X) * (vertices[(i + 1) % 3].Y + vertices[i].Y);
            }
            return sum < 0;
        }
    }

    public Face(params AffineCoordinates[] ver)
    {
        vertices = new AffineCoordinates[3];
        vertices[0] = ver[0];
        vertices[1] = ver[1];
        vertices[2] = ver[2];
    }
}

I know it is plenty of ode, that's why I didn't want to post it.
Comment: Nobody here can guess how that `B` type behaves, so if you want help figuring out why it doesn't behave as you expected you will have to post it. Also, post *real* code, since the code doesn't compile (incomplete for definition, incorrectly placed braces) etc. it makes it hard to believe this is even a cut-down version of your real code. If you want help with your actual code, post your actual code, or *even better*, post a [mcve].
Comment: Can you show the constructor for B?
Answer: This code prints '20' which is what I think you're after?
 public class A
 {
      public int Value;
 }

 public class B
 {
      public A objectA;
      public B(A obj)
      {
            objectA = obj;
      }
 }
 class Program
 {
      public static List function(List input)
      {
            List output = new List();
            foreach  (var i in input)
            {
                 output.Add(new B(i));
            }
            return output;
      }
      static void Main(string[] args)
      {
            List listA = new List();
            listA.Add(new A { Value = 10 });
            List listB = function(listA);
            listA[0].Value = 20;
            Console.WriteLine(listB[0].objectA.Value);
      }
 }

Comment: The point of the code is to demonstrate that B contains a reference to A which is the problem the OP is having. It can be improved in a lot of ways... :)
Comment: Why are you instantiating `listB` to replace it with `function(listA)`? Also, `A.Value` and `B.objectA` should both be properties, not fields
Comment: If you demonstrate bad practices to a person who's just learning, they'll learn bad practices. I've seen way too many times production code with things like `a = new List(); a = GetData();`
Comment: Feel free to post a better answer
Comment: @HansKilian your code is a simpler version of what I have and I have no idea why my code fails.

Passing list as an argument


How is the Dominican ID verification digit determined?

Question: I have been exploring how the "Check Digit" that is added to the end of the cédula is calculated. I understand that this "Verifying Digit" consists of the last two digits of the cedula plus an additional digit that is somehow calculated from the first nine digits of the cedula.
Example:
ID: 001-0052268-9 Verifier code: 895.
I have tried Luhn's algorithm, commonly used for ID number and credit card verification, but it seems that this algorithm does not apply to calculate the last digit of the "Verifying Digit" in the case of the Dominican cedula.
Therefore, I would like to kindly request any information you can provide on the algorithm or formula that is used to calculate this last digit of the "Verifying Digit".
Comment: According to [Wikipedia](https://en.wikipedia.org/wiki/Cédula_de_Identidad_y_Electoral#Technical_details) it should be a Luhn checksum: "The cédula number is broken up into three pieces: a series number (3 digits), a document number (7 digits), and a Luhn checksum."
Answer: This works for the given ID:
$$
1\cdot0 + 2\cdot0 + 1\cdot1 + 2\cdot0 + 1\cdot0 + 2\cdot5 + 1\cdot2 + 2\cdot2 + 1\cdot6 + 2\cdot8 \\
= 0 + 0 + 1 + 0 + 0 + (1+0) + 2 + 4 + 6 + (1+6) \\
= 21
$$
So the check digit should be $9$ since $21+9 \equiv 0 \pmod {10}$.
Comment: That's right but I'm looking for the last number of 895. In this particular case, how did I get to the digit 5?
Comment: @InveniumBusiness. It wasn't clear that "Verifying Digit" referred to "Verifier code".


Is the union of a countable collection of lines countable?

Question: I know that if sets $A=\{m_1x+b_1, m_2x+b_2,...\}$ and $B=\{l_1x+c_1, l_2x+c_2,...\}$ are countable, so is $A \cup B$; however, can I extend this out to $A \cup B \cup C \cup D...$ (all the lines in $\mathbb{R^2}$?) Thanks.
Answer: Considered as lines, as long as your union has only countably many items the whole collection is countable.  Each line corresponds to a pair of numbers, the slope and intercept, as you have shown.  The union of those sets is a countable set of ordered pairs.  As there are uncountably many lines in $\Bbb R^2$, your countable union cannot cover all of them.
Comment: I'm sort of getting it. What would be a counterexample of this? Or is it because I've defined those lines to be countable that I can make this argument and my counterexample would depend on the set that I've defined as countable?
Comment: Any countable union of countable sets is countable, at least in $ZFC$.  I don't know what you are looking for a counterexample of.


How to use boost::bind to bind a member-function to ANY object

Question: I'm trying to achieve something with boost::bind boost::function but can't make it work.
I see how to bind a member function of an object with some arguments to be used later as a void/void function. But how to bind it to any object.
I'd like to have a pointer to a member function of my class (that could be used with any object of the class) with binded arguments:
#include "boost/function.hpp"
#include "boost/bind.hpp"

class A
{
public:
    A() {}

    void func1( int i );
    void func2( const std::string& s );

    static void dispatch( std::vector& vect, /* boost::function parameter "func" */ ) // what parameter should I set?
    {
        for ( std::vector::iterator iter = vect.begin();
              iter != vect.end();
              ++iter )
        {
            // what syntax should I use?
            //(*iter).(*func)();
        }
    }
};

int main()
{
    std::vector vect;

    // have func1(3) be called for all objects contained in vect
    // this does not compile because boost expects an object to bind the function to
    A::dispatch( vect, boost::bind( boost::mem_fn(&A::func1), 3 ) );
    // have func2("hello") be called for all objects contained in vect
    // this does not compile because boost expects an object to bind the function to
    A::dispatch( vect, boost::bind( boost::mem_fn(&A::func2), "hello" ) );
}

I tried that:
static void dispatch( const std::vector& vect, boost::_mfi::mf0<void, A> func )
...
boost::_mfi::mf1<void,A,int> func( boost::mem_fn(&A::func1) );
boost::_mfi::mf0<void,A> binded( boost::bind( func, 3 ) );
A::dispatch( vect, binded );

But it fails to compile:
Erreur  1   error C2664: 'boost::_mfi::mf0<R,T>::mf0(void (__thiscall A::* )(void))' : impossible de convertir le paramètre 1 de 'boost::_bi::bind_t<R,F,L>' en 'void (__thiscall A::* )(void)' b:\dev\vobs_bci\public\lib\btle\src\btle\notifierdispatcher.cpp 99

Note: I'm unfortunatley not using C++11 yet, so please no auto...;-)
Comment: Editing the question (to remove const) is not very classy after you got an answer. Of course, just drop `const` in various places from my answer, then
Comment: @sehe: Sorry, I was just commenting this on your answer. Itried to remove all the `const` in your code but it does not compile, reports `impossible de convertir le paramètre 2 de 'boost::_bi::bind_t' en 'boost::function &'`
Answer: Many subtle tweaks, most simplifications:
Live On Coliru (c++03)
#include "boost/function.hpp"
#include "boost/bind.hpp"
#include <vector>
#include <iostream>

class A
{
public:
    A() {}

    void func1(int i) const                { std::cout << __PRETTY_FUNCTION__ << "(" << i << ")\n"; }
    void func2(const std::string& s) const { std::cout << __PRETTY_FUNCTION__ << "(" << s << ")\n"; }

    static void dispatch(const std::vector& vect, boost::function<void(A const&)> const& func)
    {
        for ( std::vector::const_iterator iter = vect.begin();
              iter != vect.end();
              ++iter )
        {
            func(*iter);
        }
    }
};

int main()
{
    std::vector vect(3);

    A::dispatch(vect, boost::bind(&A::func1, _1, 3));
    A::dispatch(vect, boost::bind(&A::func2, _1, "hello"));
}

Notes

Use boost::function<void(A const&)> because you dereference const iterators
Use A::dispatch(vect, boost::bind(&A::func1, _1, 3));
Mark func1 and func2 const
void func1(int i) const;
void func2(std::string const& i) const;

Output:
void a::func1(int) const(3)
void a::func1(int) const(3)
void a::func1(int) const(3)
void a::func2(const std::string &) const(hello)
void a::func2(const std::string &) const(hello)
void a::func2(const std::string &) const(hello)

Comment: Thanks sehe, this works pretty well when functions are `const`, but I can't make it work when they are not `const` (it was the case in my OP, except I was passing a const ref to the vector which was a mistake when I created my MCVE for the OP). Really sorry about that.
Comment: Literally just erased `const`: http://coliru.stacked-crooked.com/a/a335b444d44d6a08 (well, `const_` in the case of `iterator`)
Comment: Got it. I had other problems actually with includes (funny thing is that objects passed to the function in const reference must be defined (no forward declaration), else `boost::bind` refuses to bind. It works now. Thanks a lot for your help and sorry for the `const` ;-)
Comment: Well, you take _the address of a member_. It makes sense that the compiler needs to know what the class looks like
Comment: I meant function parameters (like `std::string` for `func2`). See this question I just asked: http://stackoverflow.com/questions/33667356/why-boostbind-incompatible-with-forward-declaration


Testing significance between a linear model and a linear mixed effects model

Question: I have a number of fish measurements I want to test, for example Condition, HSI, and GSI. These are to be tested against Parasitism (A factor, 0 being non-parasitized and 1 indicating parasitized). I am hoping to see no effect from parasitism.
I did preliminary T-Tests for four lakes (pooled over three years) comparing Parasitized and Non-Parasitized fish for the three metrics. All but one lake yielded no significant difference in Parasitized and Non Parasitized fish.
Overall I want test this as a model, and since I am seeing differences in one of the lakes, I believe this means I would have to treat Lake as a RANDOM effect, and Parasitism as a FIXED effect. Please correct me if I am wrong.
My question is... how do I test for significance (where I can obtain a P-value) by comparing a linear mixed model and a simple linear model?
Can this be done similarly using likelihood ratios? For example:
anova(LMM1, LM1)
All help, suggestions and links are much appreciated...
Comment: What fixed linear model do you have in mind and why do you think you need it? Do you want to find out whether including the random effect for LAKE is warranted?
Comment: Yes, I want to see if there is significant differences between lakes with respect to parasitism. So ultimately I want to test lake as an effect or interaction I believe for one model, and then test my metrics with parasitism with lakes not having an effect.
Comment: For example: lm1 <- lm(K ~ Parasites) for my linear model and lmer1 <- lmer(K ~ Parasites + (1|Lake)) for my mixed model
Answer: You can certainly do it, i.e. you won't get an error:
data("mtcars")
library(lme4)

## Loading required package: Matrix

lme1 = lmer('mpg ~ 1 + hp + (1|vs)', data = mtcars)
summary(lme1)

## Linear mixed model fit by REML ['lmerMod']
## Formula: mpg ~ 1 + hp + (1 | vs)
##    Data: mtcars
##
## REML criterion at convergence: 181.4
##
## Scaled residuals:
##     Min      1Q  Median      3Q     Max
## -1.4603 -0.6050 -0.2401  0.4277  2.1212
##
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  vs       (Intercept)  1.379   1.174
##  Residual             14.577   3.818
## Number of obs: 32, groups:  vs, 2
##
## Fixed effects:
##             Estimate Std. Error t value
## (Intercept) 29.33120    2.03135  14.439
## hp          -0.06254    0.01207  -5.183
##
## Correlation of Fixed Effects:
##    (Intr)
## hp -0.849

lm2 = lm('mpg ~ 1 + hp', data = mtcars)
summary(lm2)

##
## Call:
## lm(formula = "mpg ~ 1 + hp", data = mtcars)
##
## Residuals:
##     Min      1Q  Median      3Q     Max
## -5.7121 -2.1122 -0.8854  1.5819  8.2360
##
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept) 30.09886    1.63392  18.421  < 2e-16 ***
## hp          -0.06823    0.01012  -6.742 1.79e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.863 on 30 degrees of freedom
## Multiple R-squared:  0.6024, Adjusted R-squared:  0.5892
## F-statistic: 45.46 on 1 and 30 DF,  p-value: 1.788e-07

anova(lme1, lm2)

## refitting model(s) with ML (instead of REML)

## Data: mtcars
## Models:
## lm2: "mpg ~ 1 + hp"
## lme1: mpg ~ 1 + hp + (1 | vs)
##      Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)
## lm2   3 181.24 185.64 -87.619   175.24
## lme1  4 183.24 189.10 -87.619   175.24     0      1          1

The AIC that is returned by anova is also commonly used to compare nested models.
However, I would urge you to reconsider why you want to test the effect of the random effect for LAKE in the first place. If the variable was part of your experimental design (as it appears to be the case to me), then some would argue that you must include it in your model no matter the difference its inclusion to the model produces (see Testing the significance of random effects for more discussion). If what you are after is to examine the effect in specific lakes, then perhaps the BLUPs are what you are after:
ranef(lme1, condVar = TRUE)

If instead, you initial question was if some lakes are more conductive for parasitism, then I would advise you to include the variable LAKE as a fixed factor and include an interaction with your other variables.

[...] since I am seeing differences in one of the lakes, I believe this means I would have to treat Lake as a RANDOM effect...

No, this is not a valid reason of why you want to select a variable as a random effect. This issue has been discussed elsewhere, for example: What is the difference between fixed effect, random effect and mixed effect models?
In my opinion, you treat a variable as a random effect if you care about its influence but the specific instantiations are not important. In your case, if it is true that you want to include the effect of LAKE in the model but you wouldn't expect a difference whether the specific lakes in your sample were {Tanganyika, Victoria} or {Caspian, Baikal}, then I would include the variable LAKE as a random factor regardless of if it contributed anything to the model based on any test or metric.
Comment: +1 but this threads looks a duplicate of https://stats.stackexchange.com/questions/141746 (which does not have a very good answer btw, so you might want to post one there too).
Comment: @amoeba [This even earlier Q&A](https://stats.stackexchange.com/questions/8428/comparing-an-lme-to-a-lm) appears to be very related to the subject as well. For future reference, the leading answer there suggests to bootstrap the likelihood ratio test statistic.
Comment: That one is a bit different because the models are not nested, no? It's comparing random effect to fixed effect, not random effect to an absence of random effect.
Comment: (But the leading answer is indeed about testing the random effect, so does not really answer that question :-)


How can I show that $X_n=e^n I_{\{Y>n\}} \to 0$ in probability?

Question: Let $Y$ be a continuous random variable with density function $f_Y(y)=e^{-y}, y > 0$. Consider the sequence $\{X_n\}$, given by $X_n=e^nI_{\{Y>n\}}, n =1,2,\cdots$
How can I show that $\{X_n\} \to 0$ in probability?
My work:
So, I want to show that $P(|X_n-0|>\epsilon) \to 0$ as $n \to \infty$.
$P(|X_n| > \epsilon)=P(X_n > \epsilon)=1-P(X_n \le \epsilon)=1-\int_0^\epsilon\Sigma_1^Y(e^n)dx$
However, I do not think my last equality makes much sense. How would you proceed?
Comment: What's the $I$ here?
Comment: @jkm the indicator function
Comment: Draw graphs of some of the $X_n.$  The rest will be obvious.
Answer: Consider the following inequality for $\varepsilon \leq 1$
$$P( |X_n| > \varepsilon ) = P(e^n I_{\{Y>n\}}>\varepsilon) \leq P( I_{\{Y>n\}} > \varepsilon) = P(Y > n) = e^{-n} \to 0.$$
Actually we do not need to know the distribution of $Y$. We can use the cumulative function properties
$$ P( Y > n) = 1 - F_Y(n) \underset{n\to \infty}{\to} 1 -1 = 0$$
For $\varepsilon > 1$ we get
$$ P( |X_n| > \varepsilon) \leq P(|X_n| > 1) \to 0$$
Comment: If $n \geq 1$ then $ e^n I_{Y > n} > I_{Y > n}$. Also An indicator is $1$ or $0$ so the probability of being greater than $\varepsilon$ is equal to the probability ob being 1.
Comment: (+1) for the inequality in the first line. But note that you need to also consider the case $\epsilon\geq 1$ for completeness.
Comment: The last line "$X_n$ are positive with probability one": that's not correct, isn't it?
Comment: why the downvote?
Comment: I'll equal it out for you! I like this idea, but how can you justify the inequality and the equality thereafter?
Comment: I see. I like this line of reasoning, thank you!
Comment: True. I will add as you ask. Thanks
Answer: First, $\text{Prob}[Y>n]=e^{-n}$ since $Y$ has an exponential distribution. Therefore, $X_n$ can have only two values, namely $e^n$ with probability $e^{-n}$ and $0$ with probability $1-e^{-n}$. Let's fix some positive value $\epsilon$ and call $n'$ the smallest nonnegative integer bigger than $\ln \epsilon$. Then, for any $n \geqslant n'$, $\text{Prob}[X_n>\epsilon]=e^{-n}$, which clearly goes to 0 for large $n$.
Comment: (1) correct (2) $\epsilon$ can be small, but never equal to 0. Therefore $\text{Prob}[X_n>\epsilon]=\text{Prob}[X_n=e^n]=e^{-n}$.
Comment: I have several questions for you. (1) There are two values for $X_n$ due to the indicator function, correct? (2) Why isn't $P(X_n > \epsilon) = e^{-n} + (1 - e^{-n})$?
Answer: $X_n$ can have two values (i.e. either $0$ or $e^n$) with $p=P(X=e^n)=e^{-n}$. For $\epsilon\geq e^{n}$, $P(X_n>\epsilon)$ is always $0$. For $\epsilon<e^n$, $P(X_n>\epsilon)=p=e^{-n}$, and this goes to $0$ as $n$ goes to infinity.


How insert values from client page to database by use custom module

Question: Help me pls. I need update database table in Drupal 8. Exemple on image.

When user click on button "ADD PRODUCT TO CART", cart value must update on page (its not a problem), cart value and user ip-address must be write to database.
Im try use ajax for this logic, but values do not write to database.
Js code:
jQuery('.cart button').on('click', function () {
var price = jQuery('.price').text();
var priceValue = parseInt(price.substring(0, price.length - 1));
var cart = jQuery('#cart');
var cartValue = parseInt(cart.text().substring(0, cart.text().length - 1));
cart.text(cartValue + priceValue + '$');

jQuery.getJSON("http://jsonip.com/?callback=?", function (data) {
    var ip = data.ip.toString();

    jQuery.ajax({
        type: "POST",
        url: "../myupdate.php",
        data: ["ip="+ip, "cart="+jQuery('#cart').text()]
    }).done(function () {
        console.log('done');
    }).error(function () {
        console.log('error');
    });

});

});
php code:
<?php
$ip = $_POST[ip];
$cart = $_POST[cart];
db_insert('abweb')
    ->fields(array('ip' => $ip, 'product' => 'some product', 'price' => $cart))
    ->execute();

This logic is implemented in the custom module. HELP PLS!!
Answer: Here is way to do this. use line item type and create your filed.  Hook_commerce_cart_product_add() call in custom module.
  function mymodule_commerce_cart_product_add($order, $product, $quantity, $line_item) {

         $commerce_line_item = commerce_line_item_load($line_item->line_item_id);

         $field_main = $product->field_main['und']['0']['value'];

         $line_item->field_update[LANGUAGE_NONE][0]['value'] = $field_main;

         commerce_line_item_save($line_item);
     }

When user click on button "ADD PRODUCT TO CART", cart value add/update your commerce line item field.


Corporate Finance ok?

Question: I've seen the tag Corporate-Finance being used, but I've also seen a question closed...
I would like to post a question on the basic principle of Assets=Liabilities + Equity, and how to finance sales growth. Is this ok on this forum?
Answer: My standard advice in this situation is that you should post your question and see what happens. Especially during the beta, it is useful to see how a broad range of question types go down and nobody will be annoyed if it turns out your question isn't a great fit.

Corporate Finance ok?


Test Statistic of Transformation

Question: The two tests below seem to be the same to me. But when I tried an example and constructed the test statistics I couldn't see how they are the same.
$H_0: \lambda=1~~~~H_a: \lambda\not=1$
$H_0: \ln(\lambda)=0~~~~H_a: \ln(\lambda)\not=0$
Use exponential distribution with form $\lambda$=mean
Then the MLE of the mean is sample average $\bar X$, with asymptotic variance $\lambda^2$
Consider the transformation $\ln(\lambda)$, then we get the MLE is $\ln\bar X$ with asymptotic  variance 1.
So the test statistics should be:
$\sqrt{n}\frac{\bar X-1}{\bar X}\to Z$
$\sqrt{n}\ln(\bar X)\to Z$
So did I make an error (I asked somebody else and they said it looked fine) or do these tests just converge to the same thing while looking different the whole time?
Edit: I should be using  Wald test here instead I think.
Comment: "... variance $\bar{X}^2/n$" ??
Comment: @Dilip Ya that was a type sorry.
Answer: From Wooldridge's cross section book, there is a problem where a non-linear transformation changes the hypothesis outcome, so the two are not equivalent.

Test Statistic of Transformation


How to design a regular language that accept all binary string except for string that contains one specific occurrences?

Question: Good day, I come across this question on my textbook and would like to seek some help.
In this question, I was asked to design a regular language that accept all binary string except for string that contains "1001", and I have never come across this type of questions before.
I know that the regular language for all binary string is 1 * 0 * 1 * 0 *(probably not the best one, but at least it works), and my initial thought was that I can just make sure that were always at least 3 0s in the middle, like 1 * 000+ 1 * 0 *, but quickly I realized that it would not work as it eliminated the strings like "101" as well.
Please give me some insight on solving this problem such as what the pattern is, thank you
Comment: No $1^*0^*1^*0^*$ does not describe all binary strings. For example, $0101$ is not in that language.
Comment: All binary strings can be written as $(0|1)^*$. Or if you don't want to allow the empty string, $(0|1)(0|1)^*$. You will need  the $|$ operator for excluding $1001.$
Comment: @ThomasAndrews Thank you for the answer. Would you mind elaborate a little more bit on why (0|1)∗  can exclude 1001? I am afraid I fail to understand why adding a | is able to exclude it, thank you
Comment: It is usually easier to see that the complement of *any* regular language is a regular language using a discrete finite automata. Then you just reverse the "accept" and "reject" nodes in the machine. Writing it out as a regular expression is a bit painful, though.
Comment: $(0|1)^*$ doesn't exclude $1001.$ You said $1^*0^*1^*0^*$ was the regular language for all binary strings. It is not.
Comment: Sometimes regular expressions include a complement operator. This would make your task easy by just taking the complement of  $(0|1)^*1001(0|1)^*$. Eliminating a  single complement operator can cause a double exponential blow-up of the expression's length - so if you cannot use it, the answer of @ThomasAndrews starting from an automaton is better.
Answer: One approach would be to find a finite automata that passes your regular expression.
Let your states be $\{A,B,C,D,F\},$ with $A$ the initial state, and $F$ the failure state. All other states are pass states.
From $A,$ a $0$ keeps you in state $A.$ For $1,$ $A\mapsto B.$
From $B,$ a $1$ maps to $B,$ and a $0$ maps to $C.$
From $C,$ a $1$ maps to $B$ and a $0$ maps to $D.$
From $D,$ a $0$ maps to $A,$ and $1$ maps to $F.$
From $F,$ both $0$ and $1$ map to $F.$
You are in state $F$ if we ever found $1001.$ In any other state, you have not seen $1001.$
We are in state $A$ if the string has no $1$s or there are at least $3$ zeroes after the last $1.$
You are in state $B$ if the string ends in a $1.$
You are in state $C$ and $D$ if the string ends in $10$ and $100,$ respectively.
The simple loops from $D$ to $D$ look like:
$$1|01|0000^*1$$
The simple loops from $A$ to $A$ are $0$ and $(1(1|01)^*000)$
We can get to $C$ and $D$ by appending $0$ and $00$ after the state $B.$
So the routes ending in $A$ are:
$$(0|(1(1|01)^*)000)^*\tag1$$
The routes ending in $B$ are
$$(0|(1(1|01)^*)000)^*1(1|01)^*\tag2$$
The routes ending in $C$ and $D$ are, together, the map to $B$ followed by $(0|00)$:
$$(0|(1(1|01)^*)000)^*1(1|01)^*(0|00)\tag3$$
The final regular expression is quite messy, joining $(1),(2),$ and $(3).$
Another approach is to  note that if the string has any ones, the state must got through $D.$ From $D$ we can end in $A,B,C,D$ if we append $\epsilon|0|00|0000^*=0^*$, wher $\epsilon$ is the empty string. So you get:
$$0^*|(0|(1(1|01)^*)000)^*1(1|01)^*0^*\tag4$$
You can come up with a better regular expression with some cleverness if you think of the binary strings as blocks of $a_{0}\geq 0$ zeros, then $a_1\geq 1$ ones, then $a_2\geq1$ zeros, … followed by $a_{2k-1}\geq1$ ones and $a_{2k}\geq 0$ zeros. Then the condition, "there is no $1001$" is the same as $a_{2i}\neq 2$ for all $i=1,2,\dots,k-1.$
Then you get: $$0^*|0^*1(1|01|0000^*1)^*0^*$$
This means that after each $1$ we get either zero, one or more than two zeroes, except after the final $1,$ when we can get any number of $0$s.
You can simplify that one more time as:
$$0^*(1|01|0000^*1)^*0^*$$
But the most general way uses the automata. You get complicated results in general.
For example, excluding $10100,$ you get, after simplifying:
$$0^*\,\left((01)^*1|000^*1\right)^*\,\left((01)^*|(01)^*0|0^*\right)$$
Comment: Wow, thank you for the effort, I didn't expect to get such a well-explained answer


When I edit a display within a view, it changes other displays too

Question: I have a question about the way displays in views work:
I have a view that has 8 different displays. Each display shows content of the same content type, but with different taxonomy terms associated with each (so one display would be something like Research Topic 1, another would be Research Topic 2, and so on).
The issue is that with two of the displays, when I edit one, the other automatically takes the same changes. All other ones work fine, except two. What I'm editing is the type of research (which as I said I'm using taxonomy terms to decide what is what). When I select one research type on display X, it changes the research type in display Y to be the same.
I'm not the one who originally set up this view, so I'm not really sure what was differently, if anything, to the others to make them work fine. Any help is appreciated!
Edit: One additional thing I just noticed is that "FILTER CRITERIA" for the two displays that seem to be connected is not italicized, but that heading in Views for all of the other displays within this block are italicized. No idea what this means, but just adding this info in case it's helpful.
Comment: Never mind! I figured it out... Each display was still overriding the others, but I found where to change that and everything is good!
Answer: Never mind! I figured it out... Each display was still overriding the others, but I found where to change that and everything is good!
For anyone else with the same question:
When you click on a filter criteria, you can select options in the drop down at the top of this overlay which specifies whether you would like to apply the changes to all displays or just this page.


How to Create a Jar that uses a file to save app information to

Question: I have a Java project that creates a UI with a table, The table is editable and when you add entries, I have it being saved into a json in a data folder outside of src in the project folder. I'm using Intellij, and I know to create a JAR to make it in artefacts, and the data folder is selected as a resource. But when the JAR is created it cannot find the file data/items.json is this because JARS cannot edit files inside them?
Comment: Rewrite your code to use file system instead of classpath to operate the configuration files or use some library to persist data in the file system.
Answer: Jar files are read only, that means you can't save informations inside the jar. You can save files inside an read them but you can't edit them.
In your case there are more or less two options:

Save the <code>*.json</code> file in e.g the OS Temp path or somewhere else where you can read and write them.
Handle like zip - https://stackoverflow.com/a/4056713/8087490 -

Locate the JAR file and open as a ZIP archive reader.
Create a ZIP archive writer to write a new version of JAR file.
Write the application's files to the writer.
Write all resources from the ZIP reader to the writer, excluding old versions of the applications files.
Close the reader and writer.
Rename the new version of the JAR to replace the old one.


Why are my variables not updating in node.js?

Question: I have built a webapp with node.js to receive messages from an SNS topic via POST request and log them to the console then display them on the webpage, if I publish a message to my topic I get the message printed in the console but nothing updates on my webpage?
This is my code for app.js
const express = require("express");
const request = require('request')
const bodyParser = require('body-parser')
const app = express();
app.use(express.static('public'));
app.set("view engine", "ejs")

app.get('/', function (req, res) {
  let subject = "Your Subject Will Appear Here";
  let message = "Your Message Will Appear Here";
  res.render("index", {subject:subject, message:message});
})

app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())

app.post('/', (req, res) => {
  let body = ''
  req.on('data', (chunk) => {
    body += chunk.toString()
  })
  req.on('end', () => {
    let payload = JSON.parse(body)
    if (payload.Type === 'Notification') {
      console.log(payload.Message);
      let subject = payload.Subject;
      let message = payload.Message;
      res.render("index", {subject:subject, message:message});
    }
  });
});

app.listen(80, process.env.IP, function(request, response){
    console.log("## SERVER STARTED ##");
});

This is my code for index.js
<!DOCTYPE html>

    <meta charset="utf-8">
    <title>title</title>
    <link rel="stylesheet" type="text/css" href="/css/style.css">
    <link href='https://fonts.googleapis.com/css?family=Open+Sans:300' rel='stylesheet' type='text/css'>

      <fieldset>
        <% if(subject !== null){ %>

SNS Subject: <%= subject %>

          <% } %>
        <% if(message !== null){ %>

SNS Message: <%= message %>

          <% } %>
      </fieldset>

Answer: Because your variables are local variables within the get and post callback functions. If you want them to persist outside the scope of those callbacks, you need to move them to a scope that won't go away when the callback returns.
For instance, this makes them module globals BUT it seems like it would be odd for every POST, regardless of who it came from, to change the content of unrelated GETs (see *** comments):
// *** Declared at module scope
let subject = "Your Subject Will Appear Here";
let message = "Your Message Will Appear Here";

app.get('/', function (req, res) {
  // *** Not declared here
  res.render("index", {subject:subject, message:message});
})

// ...

app.post('/', (req, res) => {
  let body = ''
  req.on('data', (chunk) => {
    body += chunk.toString()
  })
  req.on('end', () => {
    let payload = JSON.parse(body)
    if (payload.Type === 'Notification') {
      console.log(payload.Message);
      subject = payload.Subject; // <======= *** no `let`, we're not declaring here
      message = payload.Message; // <======= *** same
      res.render("index", {subject:subject, message:message});
    }
  });
});

But again, that probably wouldn't make sense. You'll probably need some kind of per-user or per-session storage mechanism (such as a database, or expiring in-memory cache, etc.).
Comment: Thanks! This makes sense I've only recently started using nodejs coming from Python, I wasn't even thinking about scope!


HTML SelectBox view Salesforce Object data

Question: I am new to SalesForce and am using HTML for VisualForce pages, since it's more flexible to design pages than Apex.
Now I have been stuck at inputting SalesForce Object Data into HTML SelectBox with id "Lop".
This is my VisualForce code, this is where I am using HTML and CSS for design:
<apex:page docType="html-5.0" controller="ThemMoiController">

                Quay lại

            Họ

                <input id="Ho" type="text" value="{!hoVal}"/>

            Tên

                <input id="Ten" type="text" value="{!tenVal}"/>

            Giới tính

                <input id="GioiTinh" type="checkbox" value="{!gioitinhVal}"/>

            Ngày sinh

                <input id="NgaySinh" type="date" value="{!ngaysinhVal}"/>

            Điểm 1

                <input id="Diem1" type="text" value="{!diem1Val}"/>

            Điểm 2

                <input id="Diem2" type="text" value="{!diem2Val}"/>

            Điểm 3

                <input id="Diem3" type="text" value="{!diem3Val}"/>

            Lớp

                <select id="Lop"/>

                <button id="ThemMoi" type="button">Thêm mới</button>

</apex:page>

and this is my Controller Apex code:
public class ThemMoiController {
    public string hoVal { get; set; }
    public string tenVal { get; set; }
    public boolean gioitinhVal { get; set; }
    public date ngaysinhVal { get; set; }
    public double diem1Val { get; set; }
    public double diem2Val { get; set; }
    public double diem3Val { get; set; }

    public void doInsert(){
        HOCSINH__c hs = new HOCSINH__c();
        hs.HO__c = hoVal;
    }
}

Really appreciate for your help.
Answer: -> create new sobject variable with getter setter and initialize this variable in the constructor.
-> then use this sobject variable in vf to input the field value   from user.
-> update the doInsert method to create the new record.
Apex Controller:
public class ThemMoiController {
    public string hoVal { get; set; }
    public string tenVal { get; set; }
    public boolean gioitinhVal { get; set; }
    public date ngaysinhVal { get; set; }
    public double diem1Val { get; set; }
    public double diem2Val { get; set; }
    public double diem3Val { get; set; }

    public HOCSINH__c hs { get; set; }

    public ThemMoiController(){
        hs = new HOCSINH__c();
    }

    public void doInsert(){
        insert hs;
        system.debug('---->'+hs);
    }
}

VF Page:
    <apex:page docType="html-5.0" controller="ThemMoiController">

        <apex:form>

                    Quay lại

                name

                    <apex:inputField value="{!hs.name}"/>

                Họ

                    <apex:inputField id="Ho" value="{!hs.HO__c}"/>

                    <apex:commandButton action="{!doInsert}" value="Thêm mới" />

        </apex:form>
    </apex:page>

-> when we click on 'Thêm mới' button a new record will get created in the HOCSINH__c object.
Comment: Thank you, but I need is to display the LOP object to my selectbox. Cuz as you can see, there is a select tag before the Themmoi button. That is selectbox and I want it to display value from the LOP object.
Comment: this comment box is not allowing characters more than 400 so i have posted new answer.
Answer: you can create a variable which will be a list of lop sobject.
then in constructor fetch all the records of lop object that you want to show in select dropdown list using soql query and assign the result of soql query to above mentioned variable which is list of lop sobject.
then create getter method as mentioned below that will return list of selectOption class. put the dropdown values in the list of selectOption variable from the list of lop variable.
public List<SelectOption> getItems() {
     List<SelectOption> options = new List<SelectOption>();
     options.add(new SelectOption('US','US'));
     options.add(new SelectOption('CANADA','Canada'));
     options.add(new SelectOption('MEXICO','Mexico'));
     return options;
 }

use the below code to implement select functionality on VF Page.
<apex:form>
        <apex:selectList value="{!variableToStoreSelectedValue}" multiselect="true">
            <apex:selectOptions value="{!items}"/>
        </apex:selectList>

</apex:form>

go to the doc link for more details : https://developer.salesforce.com/docs/atlas.en-us.234.0.pages.meta/pages/pages_compref_selectList.htm


Natural order of rational trees?

Question: What would be a natural order of rational trees? Rational trees
arise naturally from free algebras if we view a term as a finite
tree. For example the term f(a,g(b,c)) could be viewed as the
following finite tree:
<code>        f
      /  \
     a    g
         / \
        b   c
</code>
Rational trees are now the terms that emerge when we look at
a diagram as above and drop the restriction that it must be
a finite tree. So if we allow graphs. Here is an example of
a rational tree:
<code>       _
      / \
     /   s
    /   / \
   /   s   0
   \_/  \
         1
</code>
Since its possible to lexically order finite trees, can we also
put rational trees into a strict total order? What is a natural
one that is closest to the lexical order?
Best Regards
P.S.: The above rational tree is a counter example that shows
that a lexical order is not always possible, communicated by
Mats Carlsson. Take A=s(B,0), B=s(A,1), then the assumption
A<B leads to B>A, and vice versa.
Comment: Are the roots of these trees distinguished? Say I write a tree with root r and children c1, c2… as [r, c1, c2, …].  Then your finite tree above is [f, a, [g, b, c]], and your infinite tree is T = [s, [s, T, 1], 0].  Is your infinite tree the same as the tree T = [s, [s, T ,0], 1], or is it a different infinite tree?
Comment: Yes, roots are important. T = s(s(T,1),0) is not the same as S = s(s(S,0),1).
Answer: Since lexicographic order is looking for a difference with a depth-first search (which can look arbitrarily far down one branch without going into any of the others), why not look for a difference with a breadth-first search ?
More precisely, define $A_n$, the $n$th truncation of a term $A$ as only the first $(n+1)$ levels in the tree. On your counterexample with $A = s(s(A,1),0)$, the $0$th truncation of $A$ is $s$, the $1$st truncation is $s(s,0)$, then $s(s(s,1),0)$ etc. Put any orders $<_n$ you want on the sets of $n$ levelled trees (for example, lexicographical), then define $A' = (A_0,A_1,A_2,\ldots)$ and define the order by $A < B \iff A' <_{lex} B' \iff \exists n. (\forall m. m<n \implies A_m = B_m) \land (A_n <_n B_n)$
If $A$ and $B$ are the same at every level, I think you would agree that they are the same trees, so this order is total, and transitivity follows from that of $<_{lex}$ and $<_n$
Comment: If two trees are distincts then there is a level that distinguishes them, and so you don't need to look infinitely far down the tree to decide how they are ordered. You just need to check level after level until you find the difference. So it boils down to deciding equality among rational trees
Comment: Is it decidable for rational trees?

Natural order of rational trees?


Issue with date and inferSchema option in spark 3.1

Question: I have a CSV file with a date column as shown below,
datecol
----------
2021-01-11
2021-02-15
2021-02-10
2021-04-22

If I read this file by enabling inferSchema in spark version 2.4.5 I'm getting below schema,
root
 |-- datecol: timestamp (nullable = true)

But in spark 3.1 below is the ouput.
root
 |-- datecol: string (nullable = true)

I have checked migration guide from spark documentation but didn't get any information about this.
Could anyone please confirm if it's a bug or do I need to use some other configurations?
Answer: This is an effect of the Spark migration to Java 8 new Date API since Spark 3+. You can read from the migration guide:

Parsing/formatting of timestamp/date strings. This effects on CSV/JSON
datasources [...]. New implementation performs strict checking of its input. For example,
the 2015-07-22 10:00:00 timestamp cannot be parse if pattern is
yyyy-MM-dd because the parser does not consume whole input. Another
example is the 31/01/2015 00:00 input cannot be parsed by the
dd/MM/yyyy hh:mm pattern because hh supposes hours in the range 1-12.
In Spark version 2.4 and below, java.text.SimpleDateFormat is used for
timestamp/date string conversions [...].

In fact, inferSchema does not detect DateType but only TimestampType. And since by default in CSV Data Source, the parameter timestampFormat is yyyy-MM-dd'T'HH:mm:ss[.SSS][XXX] then it is not converted  into timestamp for the reason cited above.
You can try to add the option when loading csv:
val df = spark.read.option("inferSchema", "true").option("timestampFormat", "yyyy-MM-dd").csv("/path/csv")

Comment: @MohanaBC did you specify `inferSchema = true` in options? I can get column inferred as timestamp using the above code.
Comment: Thanks for the answer. I tried with the option `timestampFormat` but it's inferring type as string only.
Comment: Oh! I missed that. It's working after adding `inferSchema`!. Thanks.


Mockito - mocking POST request - ParameterizedTypeReference.forType(String.class

Question: I am attempting to create a unit test where a post request is being made.
WebClient.post(URI, request, ParameterizedTypeReference.forType(String.class));
I am trying to use Mockit.when(WebClient.post(Mockito.anyString(), null, Mockito.eq(new ParameterizedTypeReference<String> {})).thenReturn(nothing);
I am getting
java.lang.IllegalArgumentException:ParameterizedTypeReference must not be null
How can I correctly mock this ParameterizedTypeReference.forType(String.class)?
Comment: Where/how do you create the mock object for `WebClient`?
Comment: the WebClient is created as static. it is an import from an external jar
Comment: if it is not a mock instance (created with `@Mock` or `Mockito.mock`), then you cannot set up stubbings on it (with `Mockito.when`). It's not a mock, thus cannot be used as one.
Answer: The error is due to passing a null value to the ParameterizedTypeReference.forType method.
@Test
void test() {
//Initialise uri, request
ParameterizedTypeReference < String > response = ParameterizedTypeReference.forType(String.class);
String returnValue = "response";
when(webClient.post().uri(uri).bodyValue(request).retrieve().bodyToMono(any(ParameterizedTypeReference.class)))
.thenReturn(Mono.just(returnValue));
 String result = mockClass.postMethod(uri, request, response);
 assertEquals(returnValue, result);
 }

Comment: no this does not work


Creating a complex filter for a dataset in R

Question: I have a sample dataset which is as follows:
father<- c(1, 1, 1, 0, 0)
mother<- c(1, 1, 1, 0, 0)
children <- c(0, 0, 2, 5, 2)
cousins   <- c(0, 5, 1, 1, 4)

dataset <- data.frame(father, mother, children, cousins)
dataset

father  mother  children cousins
1      1       0      0
1      1       0       5
1      1        2       1
0     0        5       1
0     0        2       4

I would like to apply a condition-based filter so I can get all fathers marked with a '1' and mothers would be 0 OR children would be 0 OR cousins would be 0. In addition, I would also like this filter to select for all fathers with a 0 AND mothers with a 0 AND children with a 0 AND cousins with a 0. Any ideas on how I can go about creating such a filter for my dataset.
Thank you!
Comment: In your sample dataset, there are no rows in which `father` is `1` and `mother` is `0`.  Similarly, there are now rows with `father` = `0` and `children` = 0.  So do you want a data frame with zero rows as your output? Please clarify.
Comment: Do you just want two `filter`?
Answer: Is this what you're looking for:
library(dplyr)
father<- c(1, 1, 1, 0, 0)
mother<- c(1, 1, 1, 0, 0)
children <- c(0, 0, 2, 5, 2)
cousins   <- c(0, 5, 1, 1, 4)

dataset <- data.frame(father, mother, children, cousins)

dataset %>%
  filter(father == 1 & (mother == 0 | children == 0 | cousins == 0))
#>   father mother children cousins
#> 1      1      1        0       0
#> 2      1      1        0       5

dataset %>%
  filter(father == 0 & mother == 0 & children == 0 & cousins == 0)
#> [1] father   mother   children cousins
#> <0 rows> (or 0-length row.names)

Created on 2022-06-04 by the reprex package (v2.0.1)
Answer: A variation on @Dave's answer using if_any and if_all:
library(tidyverse)

father<- c(1, 1, 1, 0, 0)
mother<- c(1, 1, 1, 0, 0)
children <- c(0, 0, 2, 5, 2)
cousins   <- c(0, 5, 1, 1, 4)

dataset <- data.frame(father, mother, children, cousins)

dataset |>
  filter(father == 1, if_any(-father, ~ . == 0))
#>   father mother children cousins
#> 1      1      1        0       0
#> 2      1      1        0       5

dataset |>
  filter(if_all(, ~ . == 0))
#> [1] father   mother   children cousins
#> <0 rows> (or 0-length row.names)

Created on 2022-06-04 by the reprex package (v2.0.1)
Comment: Thanks! Looks like these are being run in tidyverse. However, I am receiving the following error while running this package:

── Conflicts ────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::ident()  masks dbplyr::ident()
✖ dplyr::lag()    masks stats::lag()
✖ dplyr::sql()    masks dbplyr::sql()

Any suggestions on how I can resolve this error in order to run the scripts you have shared above?
Comment: It's informational (when the tidyverse loads) rather than an error, i.e. telling you `dplyr::filter` will now be used instead of the base `stats::filter`. Presume the code is running okay?
Comment: Note, also, you can use the `%>%` pipe instead of `|>` if you're not on a recent release of R. `|>` is the relatively new native pipe.


Permutations in $S_5$ commuting with (123)

Question: I know that $\tau \circ \sigma \circ \tau^{-1} = (\tau(1) \tau(2) \tau(3))$ for $\sigma = (123)$ but I don't know how to deduce everything that commutes with $(123)$ in $S_5$. Any hints/tips/solutions would be appreciated.
Thanks
Comment: Please include what you've tried using an edit.
Answer: Hint: $$\tau\circ\sigma=(\tau(1)\tau(2)\tau(3))\circ\tau\tag{1}$$ so it's enough to find all $\tau$ such that $(\tau(1)\tau(2)\tau(3))=(123)$. This gives the values of $\tau$ on the domain $\{1,2,3\}$, leaving $4,5$ free.
Comment: So are the six solutions the three above with either $(45)$ or $(4)(5)$ at the end?
Comment: I've tried to work through the problem but I'm not sure if I'm getting it right could you provide the solution?
Comment: Either $\tau(1)=1, \tau(2)=2, \tau(3)=3$; $\tau(1)=2, \tau(2)=3, \tau(3)=1$; or $\tau(1)=3, \tau(2)=1, \tau(3)=2$. Do you see why?
Comment: Don't forget to upvote and accept this answer once you understand it, @math.
Comment: Actually, I think I understand it now but I have six commuting permutations, namely $\{e, (23), (12), (123), (132), (13)\}$ is this correct?
Comment: No, it's not correct.
Comment: See my comment that starts with "Either". Don't forget $4,5$.
Comment: So how do we prove there are only these three solutions?
Comment: There are six solutions. Don't forget $4$ & $5$.
Comment: They are the only solutions because of equation $(1)$.
Comment: Yes, that's it.


Strengthening an inequality

Question: I'm reading a book and there's an example problem that goes like this:
Prove that
$$
\left(\frac{1}{2}\right) \left(\frac{3}{4}\right) ... \left(\frac{2n-1}{2n}\right) \le \left(\frac{1}{\sqrt {3n}}\right).
$$
At first the book tries to solve it right away using induction, but gets a false inequality. Then it says that the original inequality is too weak and that we need to strengthen it. The book strengthens the inequality by replacing 3n with 3n+1 so that the inequality turns to
$$
\left(\frac{1}{2}\right) \left(\frac{3}{4}\right) ... \left(\frac{2n-1}{2n}\right) \le \left(\frac{1}{\sqrt {3n+1}}\right).
$$
Then, by using induction, you get a solid proof.
Why is this correct? I mean, why can you strengthen an inequality like that? If you had
$$
x \lt 2
$$
and you strengthen the inequality so that it's
$$
x \lt 3,
$$
clearly you can't prove that x is smaller than 2 by proving that it's smaller than 3 (e.g. if x=2,5).
Or is this something that works only when using induction? Or am I just completely misunderstanding this?
Comment: If the book proves that your product is less than $\frac{1}{\sqrt{3n+1}}$, then it also must be less than $\frac{1}{\sqrt{3n}}$.
Comment: A stronger inequality than $x<2$ would be $x<1$. Your eg is that of a weaker ineuality.
Comment: Oh yeah, stupid me :D Thanks!
Answer: Maybe I got it wrong, but I think that the reason for proving
$$ \prod_{k=1}^{n}\frac{2k-1}{2k}\leq \frac{1}{\sqrt{3n}} $$
by proving a stronger inequality, is just that the stronger inequality
$$ \prod_{k=1}^{n}\frac{2k-1}{2k}\leq \frac{1}{\sqrt{3n+1}} $$
is way easier to prove by induction. So it is just a technical reason.
Anyway, we may also notice that, for any $n\geq 2$:
$$\begin{align*} \left(\prod_{k=1}^{n}\frac{2k-1}{2k}\right)^2 &= \frac{1}{4}\prod_{k=2}^{n}\left(1-\frac{1}{k}\right)\prod_{k=2}^{n}\left(1+\frac{1}{4k(k-1)}\right)\\&\leq \frac{1}{4n}\prod_{k=2}^{n}\exp\left(\frac{1}{4k(k-1)}\right)\\&\leq \frac{1}{4n}\cdot\exp\left(\sum_{k\geq 2}\frac{1}{4k(k-1)}\right)=\frac{e^{1/4}}{4n}\end{align*} $$
hence:

$$ \forall n\geq 2,\qquad \prod_{k=1}^{n}\frac{2k-1}{2k}\leq \sqrt{\frac{1}{(4e^{-1/4})\,n}}\leq \sqrt{\frac{1}{\left(3+\frac{1}{9}\right) n}}$$

that is even stronger. You may also replace that $3+\frac{1}{9}$ with a $\pi$, but such a result requires some form of the Wallis product to be proved.

Strengthening an inequality


Find circles containing only zeros in an arbitrary binary matrix

Question: Given a binary matrix of arbitrary size, I need to find circles which fit into this matrix and only cover "0"-fields and no "-1"-fields.
For this example
1 0 0 0 0 1 1
1 0 0 0 0 0 1
1 0 0 0 0 0 1
1 0 0 0 0 0 1
1 0 0 0 0 1 1

I want to find the following maximal circle (denoted in +)
1 0 0 + 0 1 1
1 0 + + + 0 1
1 + + + + + 1
1 0 + + + 0 1
1 0 0 + 0 1 1

Actually this is no circle, only an approximation of it. Finding rectangles in such a binary matrix can be done with the help of histograms, see here. Additionally, I try to find smaller circles like this one:
1 0 + 0 0 1 1                1 0 0 0 0 1 1
1 + + + 0 0 1                1 0 0 0 0 0 1
1 0 + 0 0 0 1   or this one  1 0 0 + 0 0 1
1 0 0 0 0 0 1                1 0 + + + 0 1
1 0 0 0 0 1 1                1 0 0 + 0 0 1

Does anybody have a smart idea (like the histogram-approach for rectangles) of how to identify these circles (or its discrete approximations, respectively)?
Comment: Should your circles be approximations of "real" (round) circles or just diamonds as in your examples?
Comment: Actually, they should be approximations of round "real" circles.
Comment: @samgak Indeed.
Answer: Perform a nearest neighbour search, for each 0 element to find the closest 1 element, using Euclidean distance and create another matrix storing the distance from each to the nearest 1 (or just keep track of the largest).
The space that can fit the largest circle will be given by the element with the largest distance to the nearest neighbour, and the distance for each element will specify the size of the circle that can fit in the space centered on that element (distance of 1, means a single +, distance of two means a simple cross, etc).
Note this only works for circles that are centered on a given element. It won't handle circles that are centered part-way between one element and another.
Answer: Step1:
    start iteration through one by one elements

1.1 : If the element is 0
     1.1.1 : check if it in the border , ie,  row = 0 or row = max_row or col = 0 or col = max_col
     1.1.2 : if not so, check if bottom , top, left and right elements are 0
     1.1.3 : if all are 0, increment radius
       1.1.4 : else break;
       1.1.5: now check bottom-1, top+1, left-1; right+1 to see if they are 0
       1.1.6: if they are also 0, again increment radius

       1.1.7: repeat the above steps till you break
 1.2 : check all elements of matrix

Comment: But using this algorithm will lead to a shape looking like a cross, and not like a circle (since you only check for bottom-1, top+1, left-1, right+1 and not e.g. top-left).


Install Ubuntu 10.04 (Paravirtualization) on CentOS 5 using Xen

Question: Anyone got instructions on how to install Ubutnu 10.04 (Paravirtualization) on CentOS 5 using Xen?
I was able to install using netboot: http://archive.ubuntu.com/ubuntu/dists/lucid/main/installer-amd64/current/images/netboot/xen/
Finished the installation but failed to bootup, anyone can help?
Detail:
1. Create a new disk with LVM
2. Downlaod vmlinuz & initrd.gz from
http://archive.ubuntu.com/ubuntu/dists/lucid/main/installer-amd64/current/images/netboot/xen/
3. /usr/sbin/xm create -c /etc/xen/ubuntu.cfg install=true
4. Format the whole disk as EXT3, mount as /
5. The installer show some warnings (randomly, since they are not repeatable)

Deboostrap warning
Warnings: Couldn't download package libnewt0.52

6. The clicked continue, and retry to install the base system again
7. When prompted to choose the kernel, I cannot find xen related, so I
choose linux-generic
8. Everything alright, the installer quit and system halt.
9. I launch the VM using  /usr/sbin/xm create -c /etc/xen/ubuntu.cfg, it show
Error: Bootloader isn't executable

p.s. ubuntu.cfg
==============
name = "Ubuntu"
kernel = "/root/ubuntu/vmlinuz"
ramdisk = "/root/ubuntu/initrd.gz"
vif = [ "mac=00:13:1e:6e:23:e2,bridge=xenbr0" ]
disk = [ "phy:/dev/vg0/ubuntu,xvda,w" ]
==============

Answer: I found some howtos, try to check it out, maybe it will help you http://drup.org/how-install-ubuntu-vm-under-centosred-hat-xen and the second one http://dev.osso.nl/herman/blog/2006/12/24/ubuntu-bootmenu-too-shiny-for-xen-with-hvm/ (it;s a little bit old but maybe will tell you more).
Executing this command
ls -l /usr/lib/xen-tool

should give you a list of all distributions that you can install. I hope some of these informations will point you closer to solve your problem. I am always installing only debian as guest so I never had these problems.


Search form directs to main site from sub-domain

Question: I have a sub-domain setup with a default main domain (or dual sub-domain if you prefer):  www.example.com  and  m.example.com  (mobile).
I am using Drupal 7.27. I am using Themekey 7.x-3.0 for theme switching which is working, but I'm not sure if that is what's causing the problem.
If I enter a search parameter under a form created on m.example.com, the form submit is redirected to www.example.com to handle the search page, meaning the theme is switched back.
I checked the output, and the form search action = /, and when I use that (as a test) in a link or other reference, it lands on m.example.com as it should.
Does anyone know what might be causing this?, is this default functionality?, and what I could do to prevent it or override it?
I don't have any other redirect modules, or Rules setup that should be redirecting.
Any help or direction would be greatly appreciated.
Thanks.
Comment: are using two settings.php file for drupal ?
Comment: No, themekey is handling all of the switching so I didn't add a separate sites folder for the sub-domain.
Comment: maybe something like base_url needs to be set in the settings.php
Comment: thanks! - that fixed it when I created a sub-domain folder and adjusted settings.php - however it also creates a new problem: under m.example.com it is trying to locate images under a different folder from default/files.
Comment: I just moved the files to a communally accessible area and all is good - thanks a bunch!
Comment: Please add that as an answer which might help other
Comment: will do - once the time limit expires for <10 rep.
Answer: Thanks to harshal:
The problem ended up being a lack of a distinct settings.php for the sub-domain.
- I created the folder under sites for the sub-domain
- copied and modified the default settings.php with the correct base_url
This created a new problem whereby the sub-domain was looking for files under the wrong folder (they were under the default folder)
- I adjusted the settings in admin/config/media/file-system for Public to sites/files (universally accessible)
- copied all files over


Does $Aut(G)$ finite $\rightarrow$ $G$ is finite?

Question: Does $Aut(G)$ finite $\rightarrow$ $G$ is finite?
I think this is not true, because if we look at the integers under addition, there are 2 generators, $1,-1$, and the only non-trivial automorphism permutes these generators, so $Aut(\mathbb{Z}) \cong \mathbb{Z}_2$
Is this correct?
Comment: @reuns: The kernel is the entire group (since $\mathbb Z$ is abelian and so has no nontrivial inner automorphisms), but what does that have to do with anything?
Comment: See [this question](https://math.stackexchange.com/questions/1300715/groups-with-finite-automorphism-groups?rq=1), or [this question](https://math.stackexchange.com/questions/1592550/examples-of-torsion-free-abelian-groups-with-finite-automorphism-group?noredirect=1&lq=1).
Comment: Yes. Also you can look at the natural homomorphism $G \to Aut(G), g \mapsto (h \mapsto ghg^{-1})$, what is its kernel ?
Answer: Yes, this is correct. $ { } { } { } $


How to prove that $X^{T}X$ is singular if $X$ is singular.

Question: Let's suppose $X$ is $n \times m$ not full rank matrix (ie any column vector in $X$ can be expressed as linear combination of others column vectors). If $$B = X^TX$$
why $B$ is singular?
The question is inspired from the statistics, wherein, if vector of independent variable is linearly dependent, then it is assumed that $X^TX$ is also dependent (or singular). It is difficult to grasp if $X$ is dependent then how $X^TX$ is dependent?
Comment: You should state that X is not full column rank
Comment: How do you define "singular matrix" for non-square matrices?
Comment: @gimusi thanks for suggestion
Comment: @Neeraj "not full column rank" suffices, the point is that $Null(X)\neq \emptyset$ thus exists v such that $Xv=0$ then $X^TXv=X^T0=0$ and thus $X^TX$ is singular.
Answer: Exists $v\neq0$ such that
$$Xv=0$$
then
$$X^TXv=0$$
Comment: @Neeraj  The point is that $Null(X)\neq \emptyset$ thus exists v such that $Xv=0$ then $X^TXv=X^T0=0$ and thus $X^TX$ is singular. Recall that A is not singular if $Av=0 \iff v=0$
Comment: can you please elaborate your first statement? what does it mean and from where it comes from?
Comment: @gimsuin Thanks a lot
Comment: You are welcome! Bye


Is there a way to find what processes are in nice CPU top list?

Question: I have a top output that shows a high amount in nic CPU percentage.

Is there a way to see what those processes are? And is it possible to do it from top?
Comment: That's busybox top, which indicates: (1) you're only going to get minimal information, (2) you're on Android, and (3) this doesn't appear to be on topic here.
Comment: The concern is still one of cpu usage tracking and management on a unix platform. In that sense, is it relevant?
Answer: You can also use a really nice tool called htop.
Not sure which package manager you're working with but htop can be downloaded from the general ones.
When you run htop there's a menu list in the bottom screen, if you press 'F6' it will show you different display options which one of them is to sort by NICE.
Example:

Edit:
htop also allows you to interactively increase or decrease nice value of processes.
Comment: how does that output add up to the nice number at the top? i.e. the number at the first line of my screenshot. can we see how they aggregate to a cpu consumed figure?
Comment: Since -11 is the lowest nice value at the moment of the screen shot then I guess that this is what it shows, the process with the highest priority. For more information check https://www.systutorials.com/docs/linux/man/1-htop/  and check the information I added to my answer.
Comment: Yeah, just need to understood how that becomes a cpu consumed figure. The first line in my top output is a breakdown of cpu consumption. It seems like some nice'd processes are consuming cpu.
Answer: I would suggest displaying all processes with ps and simply sort by niceness, for example:
ps ax -o pid,ni,cmd | sort -k 2

Comment: how does that output add up to the nice number at the top? i.e. the number at the first line of my screenshot. can we see how they aggregate to a cpu consumed figure?


arp --delete is not deleting an entry; merely marks entry as "incomplete"

Question: $ sudo arp -avn
? (10.10.7.30) at 00:cc:cc:bb:dd:86 [ether] on eth0
...

$ sudo arp --delete 10.10.7.30

$ sudo arp -avn
? (10.10.7.30) at <incomplete> [ether] on eth0

After --delete I expected no entry for 10.10.7.30.  Unexpectedly, the entry remains and is marked <incomplete>.
Using Ubuntu 10.04.
Answer: The entry will be removed, just be patient.
(If you want the shortest possible answer: incomplete == deleted)
Let's say "delete" is the wrong word for the action. What's really happening here is that the entry is manually set to the state "request sent, no answer" (thus "incomplete" ARP process) as if the machine would be really unreachable.
Now, the entry will be completely removed soon unless it gets a new valid ARP response in the meantime. In that case the entry would be re-added anyways even if it was removed instead of being marked as incomplete. So there's no actual pro or con to this behaviour.
But keep in mind that we're talking of a cache. Deleting things from caches is hard and expensive. It's way more efficient to invalidate an entry and wait if it gets replaced before it is finally removed. But for the system it's totally no difference if the entry is gone from the list or just marked incomplete.
Comment: But exactly that's the case here. If anything looks up an entry that is marked "incomplete" a new ARP request is send. If it is answered, the entry is updated and no longer "incomplete". If no answer returns, the entry is removed from the list after some time. (That's why I think it's pretty useless to "--delete" an entry manually. If the machine lives, the entry will be refreshed. If it is really gone, then why "--delete" the entry manually anyways?)
Comment: And there is no way to really delete it so that a subsequent need to know the destination would trigger a "new" ARP query?
Comment: ... it comes to my mind that you might want to "change" the ARP address for an IP address. In this special case you can "--delete" manually and have the next ARP request return the new address. But for this scenario, it doesn't matter at all if the entry is visibly removed from the list or marked "incomplete". For the tech involved, it's the same.
Comment: @KarmaFusebox do you know where this cache exists?  Is it in process memory?  If so, which process? or is it maintained by the kernel?  In a file on-disk?  elsewhere?
Comment: @KarmaFusebox Deleting an entry makes sense if the entries were also added manually (and therefore have the permanent flag set)
Answer: Just to complement all other answers, I found this link very useful.
In some cases using ip is more appropriate, like the command:
# ip -s -s neigh flush all

Results may depend on your linux kernel.
Comment: This worked perfectly for me. Anyone who is having trouble with other responses should try it.
Answer: Apart from the other fine answers it worths mentioning that it is possible to completely eredicate the arp cache by removing it. One of the less painful ways is:
ip link set arp off dev eth0 ; ip link set arp on dev eth0

This should remove all entries, be they in whatever state.
Alternative methods include downing and upping the interface and similar ways to make the arp cache completely removed and recreated.
Comment: This is the correct answer. I needed exactly this. Not just marking some entry for "incomplete" or "has not been reached for a while". Thank you.
Answer: I know this could sound obvious, but, for those not used to work with the shell, this is my suggestion:
Starting from the answer of @KarmaFusebox, why not simply grep the results...? :
arp -a | grep "incomplet" -i -v

It will show you only the existing/active cache entries.
I think it keeps being a POSIX compliant command. And it is a scriptable method, if that is what you need.
Comment: I'm looking for a reasonable explanation of why `arp` behaves this way.  It seems peculiar.  Tangentially, could `arp` be forced to remove the `` entries?
Comment: Well, @JamesThomasMoon1979, I would rather say that, on systems that seem not to behave this way (i.e: Windows console), they really are. The only difference is they don't show `incomplete` entries on the arp cache list command.


Custom fields always empty

Question: I followed the steps on skvare to index Civicrm contacts via Search API, then to have Views list the contacts with the search box as an exposed filter. This seems to work overall, but custom Civicrm fields don't seem to be showing up via Views. For example, I've got records showing up with display name, bio, and contact ID. But they're also supposed to have custom fields called "designation" and "role". I've checked that the corresponding contact created in Civi has values (strings "test" and "pencil sharpener" respectively) for these fields. I've made sure that these fields are indexed in the search API and I've rebuilt the search index, to no avail. I'm able to select the fields in the Views UI but when I look at my View, I don't see them and when I do a dump of the fields variable in the View template, those fields are there but they just have an empty value.
Why are my custom Civi field values empty when I try to display them on contacts indexed with the search API via Views? This is all in Drupal 7.
Comment: Hi Beth, just to rule this out ... you do have the custom data showing in standard, ie non Search API, views don't you?
Comment: Yes, a view of contacts (rather than a view of indexed contacts) shows these fields.
Answer: Hard to confirm why yours are not showing. If I am comparing like with like then I have built some Views and used Search API to give the custom data in the View columns and to provide blocks showing the count for each of the sub-types of that custom data.
Thing is there is so many steps involved I don't really know how to provide a clear recipe for you to follow assuming of course that would be better thank Mark's since I know I used that as part of my own journey on this.
Answer: I solved this problem by making sure each field I wanted to use was an indexed field in the Search API settings for my Contacts index, and then re-adding the fields to Views.
Comment: ah. your Q seemed to imply you had done that: "i''ve made sure that these fields are indexed in the search API"

Custom fields always empty


A line through the midpoint of the sides $AB$ and $AD$ of a rhombus $ABCD,$whose one diagonal is $3x-4y+5=0$ and one vertex is $A(3,1)$ is

Question: The equation of a line through the midpoint of the sides $AB$ and $AD$ of a rhombus $ABCD,$whose one diagonal is $3x-4y+5=0$ and one vertex is $A(3,1)$ is $ax+by+c=0$.Find the absolute value of $a+b+c$,where $a,b,c$ are integers expressed in lowest form.

My try:
As $A(3,1)$ does not satisfy $3x-4y+5=0$,so the diagonal $BD$ has equation $3x-4y+5=0$.As $AC$ and $BD$ are perpendicular,so equation of $AC$,i found is $4x+3y-15=0$,then i found point of intersection of the two diagonals is $O(\frac{9}{5},\frac{13}{5})$.I want to find the coordinates of $B$ and $D$,so that i can find out the equation of the required line.But for that i need $BO$.I am stuck here.Please help me.Thanks.
Comment: Can you find C?
Answer: As you wrote, the equation of the line $BD$ is $3x-4y+5=0$, and the intersection point of the two diagonals is $(9/5,13/5)$.
Note that the line we want is parallel to $BD$, so the equation of the line can be written as $3x-4y+k=0$.
Now, since this line passes through the midpoint of the line segment $AO$, i.e. $(12/5,9/5)$, we have $$3\cdot\frac{12}{5}-4\cdot\frac 95+k=0\Rightarrow k=0.$$
So, the line we want is $3x-4y+0=0$.
Answer: Notice, the line through the mid-points of the sides $AB$ & $AD$ should be parallel to the diagonal $BD$ (having slope $3/4$) hence the equation of the desired line:  $$3x-4y+c=0\tag 1$$
Now, in right $\triangle AOB$, the above line is parallel to the base $OB$ & passes through the mid-point of side $AB$ hence by using A-A-A similarity of triangles, the line should pass through the mid-point of line $AO$ joining the points $A(3, 1)$ & $O\left(\frac{9}{5}, \frac{13}{5}\right)$ whose mid-point is given as
$$\left(\frac{3+\frac{9}{5}}{2}, \frac{1+\frac{13}{5}}{2}\right)\equiv \left(\frac{12}{5}, \frac{9}{5}\right)$$
now, substituting the coordinates of the point $\left(\frac{12}{5}, \frac{9}{5}\right)$ in (1), we get
$$3\left(\frac{12}{5}\right)-4\left(\frac{9}{5}\right)+c=0$$ $$c=0$$ now, substituting value of $c=0$ in (1), we get equation of the required line:$$3x-4y=0$$ now, comparing the above equation with $ax+by+c=0$, we get $a=3, b=-4, c=0$ hence, $$\color{red}{|a+b+c|}=|3-4+0|=\color{blue}{1}$$


XML parser selector skips mixed content in migration

Question: I am trying to migrate and map the source fields of an external rss/xml file into Drupal.
I have built a custom module:
In config/install folder I have my yml file modules/custom/import_rss/config/install/migrate_plus.migration.xml_articles.yml:
id: xml_articles
label: 'Import articles'
status: true
source:
  plugin: url
  data_fetcher_plugin: http
  urls: 'https://myrss.xml'
  data_parser_plugin: simple_xml
  item_selector: /rss/channel/item
  fields:
    -
      name: guid
      label: GUID
      selector: guid
    -
      name: title
      label: Title
      selector: title
    -
      name: pub_date
      label: 'Publication date'
      selector: pubDate
    -
      name: link
      label: 'Origin link'
      selector: link
    -
      name: summary
      label: Summary
      selector: description
  ids:
    guid:
      type: string
destination:
  plugin: 'entity:node'
process:
  title:
    plugin: get
    source: title
  field_remote_url: link
  body: summary
  created:
    plugin: format_date
    from_format: 'D, d M Y H:i:s O'
    to_format: 'U'
    source: pub_date
  status:
    plugin: default_value
    default_value: 1
  type:
    plugin: default_value
    default_value: my_article

After running drush mim xml_articles all the items are imported except the body field in Drupal nodes are empty?
If I look at the source xml file the <description> tag looks like this:
<description>

You might have noticed some changes on Drupalize.Me lately. We've just wrapped up a huge content archiving project and I'd like to share what we did and how this will help us move forward with a library of tutorials for Drupal learners that we're committed to keeping up-to-date.

</description>

In the XML source the <description> has 3
 tags <description>

Text

</description>
I tried as selector description/*, but that's not working either.
How do I get this working?
Comment: description/div/div/div?
Comment: @Kevin thanks I got it working by `description/div/div/div/*`
Answer: By adding description/div/div/div/* to the selector.


Is it o.k. to bootstrap sample of a table from its proportions - and how to do so (in R)?

Question: I wish to create a confidence interval for a statistic calculated on a table (let's say the p.value of chisqr.test).
For this, I might sample a bootstrap sample with the same size, from a table with the proportions of the table I have.  And my questions are:

Is this a correct procedure to use, or is there some catch here that I might be missing?
How can a table be "sampled" in R?  (my first instinct is to "open up" the table using melt, then expend the table to rows, sample them, and fold the results back to a table.  Is there a better way?)

p.s: I am aware of the simulate.p.value switch in chisqr.test - my question is more general - does it make sense to use bootstrap in such a way.
Sample table if someone wishes to show something:
(x <- matrix(c(12, 5, 7, 7), ncol = 2))
prop.table(x)

# undoing a table
as.data.frame(as.table(x)) # but how do we open up the rows now? -

update: this was answered in the comments thanks to chl.
Comment: @whuber - I was wondering why did you ask this question. I am curious, can you pls tell. The only reason I can think of is bootstrapping might not produce the same marginals. Am I thinking correctly.
Comment: An answer to the last comment in your code: `library(reshape); untable(data.frame(X1=gl(2,1),X2=gl(2,2)), c(12, 5, 7, 7))`.
Comment: Do you need to reproduce any of the marginals in the table?
Comment: Whuber - That was a question I didn't think about.  Let's say we wish to create a CI for the chi square statistic using bootstrapping, should we then preserve the marginals of the bootstrapped table to be the same as the ones we had?
Answer: Let me see if I understand you correctly:
You have a contingency table say $M$ by $N$ and you calculate a statistic based on this table. You want a CI for this statistic and you don't have a theoretical CI (or may be you don't want to use it, due to some reason).
Generally, if you have a contingency table, the columns represent the possible levels of the response variable (here: $Y$ will have $N$ levels) and the rows represent the levels of the explanatory variables (here: $X$ will have $M$ levels).
Now if you want to bootstrap, you need to make sure that you couple $Y$s and $X$s together. That is, shuffle $(X_i, Y_i)$ together for $i$ from $1$ to total number of observations (note this will be greater than $MN$ unless all the entries in the table are $1$).
Basically, what you can do is translate your table to a "hypothetical" data set set by using row numbers as levels for your $X$s and the column number as the value for the corresponding $Y$s.
To make it mathematically clear:
Lets say your $(i, j)^{th}$ entry in the table is $a$, then you need to create, $a$ data points for which the value of $Y$ is $j$ (remember column number for $Y$ value) and $X$ value is $i$. Do the same for $i = 1 \ldots m$ and $j = 1 \ldots n$. Now you have a "hypothetical" data set. Bootstrap from this data set as you would do in linear regression, say $B$ number of times. Each of the newly generated data set is a bootstrap sample. For each of the bootstrapped sample calculate the corresponding contingency table (say for $Y=1$ how many $X$s=1, how may $X$s=2, ... , this will give you the (1,1), (2,1),... entry for your table). Calculate the statistic for each of the $B$ tables and you have the desired bootstrap distribution.
I don't know if this is what you meant by bootstrapping the table. Please let me know if you thought otherwise.
HTH
S.
Comment: Hi Suncoolsu - you got it right, that's what I was wondering on how to do.  And my question was - is it a reasonable solution for that question.  From your answer, I deduce that you believe it is valid to do it this way (Also chl directed me on where to look for implementing this within R, which is great).  Thanks for you input.  Tal
Answer: I once asked similar question on stackoverflow. Basically you sample from table the same way you sample vector.
Comment: thanks mpiktas. chl suggestion is also a good direction on how to handle such data.
Answer: how to bootstrap a table ought to depend on how it was obtained. there are a few ways this can be done.
for example, one can sample n individuals and classify all of them into an $r\times c$ table. in this scenario, neither row nor column totals are fixed. it would then seem plausible that a bootstrap sample would take a sample of size n  with replacement from the original n respondents and classify them into an $r\times c$ table.
in another scenario, the row totals for the table could have been fixed in advance. in that case, one would draw with replacement a sample of  $n_{i+}$ individuals from the $n_{i+}$ individuals in row $i$, separately for each row.
does this make sense? [apologies if this restates what has already been said. i don't quite grasp all of it.]
i would like to reiterate @whuber's query on why you are asking this question. one usually does not speak of obtaining a CI for a statistic [such as a p-value]. can you clarify this a bit?
Comment: Hi Ronaf.  This came up when I learned about Cramer's V statistic for contingency tables.  I was wondering what the CI for it are, and found a way to get them - but hoped to validate my results using bootstrapping.  Cheers, Tal


How do I rewrite Solr search results to show related entity reference field?

Question: 1: I have two node types: "Plants", and "Pests". related using a field in the plants node type called "affecting_pests".
How do I modify the solr Search result view; in the case of a Pests result, to display " name of pest==>affected plant " in the results title, and instead of link to pest page, redirect to the parent plant that the pest affects?
Answer: Assuming you are using the apachesolr module, implement hook_apachesolr_search_result_alter() in a custom module. The function you create will receive the $document object that you may modify. You will have to add logic in there to locate and load the appropriate related entity.


Proving $\lim_\limits{x \to 0} \frac{f(x)}{g(x)} = \frac{\lim_{x \to 0} f(x)}{\lim_{x \to 0} g(x)}$

Question: I know that $$\lim_\limits{x \to 0} \frac{f(x)}{g(x)} = \frac{\lim_\limits{x \to 0} f(x)}{\lim\limits_{x \to 0} g(x)}$$ but only if $\lim_\limits{x \to 0} g(x) \neq 0$, $\lim_\limits{x \to 0} f(x) = A \in \mathbb{R},$ $\lim_\limits{x \to 0} g(x) = B \in \mathbb{R}$.
And my question is, can we omit the condition $\lim_\limits{x \to 0} g(x) \neq 0$? I mean, is it true that

If $\lim_\limits{x \to 0} \frac{f(x)}{x} = L \in \mathbb{R}$ then $\lim_\limits{x \to 0} f(x) = 0$.

because otherwise, if $\lim_\limits{x \to 0} f(x) = A \leq \infty, A \neq 0 $ then $\lim_\limits{x \to 0} \dfrac{f(x)}{x} = A \times \lim_\limits{x \to 0} \dfrac{1}{x} = A \times \infty$ cannot be equal to real number.
Comment: The marked claim is true indeed. Your arguments are not completely mathematical though.
Comment: @Yanko Why? Because I multiply by infinity?
Comment: As you mentioned at the beginning,you can only use the arithmetic properties of the $\lim$ (say $\lim(f\cdot g)  = \lim f \cdot \lim g$) only if both $\lim f$ and $\lim g$ exist.
Comment: Also another problem is that you're assuming that $f(x)$ convergence to something (while not converging to zero doesn't mean it is convergence to something else).
Comment: @Yanko yes, I agree. It's a very useful remark
Answer: Formal proof for the marked claim:
You're given that for every $\varepsilon>0$ there exists $\delta>0$ such that for every $x\in (-\delta,\delta), x\not = 0$ we have $$|\frac{f(x)}{x}-L|<\varepsilon$$
Fix $\varepsilon=1$, so for a fixed $\delta$ we have that $L-1<\frac{f(x)}{x}<L+1$ for all $x\in (-\delta,\delta),x\not= 0$.
In particular it follows that $|f(x)|<\max(|x|(L+1),|x|(L-1))$ for all $x\not=0$ in an interval around zero. Thus, $|f(x)|$ convergence to zero as $x$ goes to zero.
Now it is not quit trivial (not that hard as well) to show that $\lim_{x\rightarrow 0}|f(x)|=0$ implies that $\lim_{x\rightarrow 0} f(x)=0$ which is what you wanted to prove.
Edit: It turns out to be even simpler than that (See the comment of Theo Bendit) using the algebra of limits theorems since $\lim_{x\rightarrow 0}\frac{f(x)}{x} = L$ and $\lim_{x\rightarrow 0} x = 0$ we have that $\lim_{x\rightarrow 0} f(x) = \lim_{x\rightarrow 0}\frac{f(x)}{x} \cdot \lim_{x\rightarrow 0} x = L\cdot 0 = 0$
Comment: Using the algebra of limits theorem, $$\lim_{x \to 0} f(x) = \lim_{x \to 0} \frac{f(x)}{x} \cdot \lim_{x \to 0} x = L \cdot 0 = 0.$$
Comment: @TheoBendit That's actually very nice!


L1 vs L2 stability?

Question: See this paragraph here: http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/

The instability property of the method of least absolute deviations means that, for a small horizontal adjustment of a datum, the regression line may jump a large amount. The method has continuous solutions for some data configurations; however, by moving a datum a small amount, one could "jump past" a configuration which has multiple solutions that span a region. After passing this region of solutions, the least absolute deviations line has a slope that may differ greatly from that of the previous line. In contrast, the least squares solutions is stable in that, for any small adjustment of a data point, the regression line will always move only slightly; that is, the regression parameters are continuous functions of the data.

For some reason, I can't find anything online describing this 'stability' phenomenon. Is it known under a different name?
Stability seems to refer to something like, for (x,y) dataset, "slightly nudge a single input x_i. For the L1 objective function, the slope of the prediction line changes massively, so the L1 objective is unstable."
I would really like a theory explanation for this picture included in that post:
http://www.chioka.in/wp-content/uploads/2013/12/programmatic-L1-vs-L2-visualization.png
Comment: The concept seems closely related to the *breakdown point* of robust statistics.
Comment: The terms "robustness" and "sensitivity analysis" (between inputs and outputs) come to mind.
Comment: Regarding the picture you attached, you will find more examples here : https://www.thekerneltrip.com/statistics/linear-regression-l1-minimization/ (it mostly deals with the implementation, not the theoretical details though)
Answer: This is generally called "sensitivity analysis" or "stability". An excellent paper deriving bounds based on this is Stability and Generalization. The bounds of course aren't necessarily tight!
If you look at Definition 19 and the follow-up Theorems and Lemmas you can see that if something is $\sigma$-admissable then there is a bound that is linear in $\sigma$ generally. For $L_1$ it's fairly simple to show that it is $1$-admissable (indeed, they state this for the $\epsilon$-insensitive $L_1$ loss for SVM - example 1, bottom of page 17 in the pdf (515 is the printed page number)), while $L_2$ requires the space $\mathcal{Y}$ to be bound - if one does the math this is basically because one can derive $$\sigma \geq \frac{|y_1^2 - y_2^2 - 2y'(y_1-y_2)|}{|y_1 - y_2|} = |y_1+y_2 - 2y'|$$. Thus, generally, one should expect the $L_1$ to actually have a better bound here. I don't think this completely addresses your question but it hopefully does give you a starting point for a more formal approach to analyzing your specifics.
Comment: One thing I want to point out here is that bounds are given in terms of regression performance - this is generally what is of interest. This is different of course than "how much" the solution changes.
Answer: L1 norm is based on minimising Least Absolute Deviation, with absolute deviation being calculated:
$$ AD = \Sigma^n_{i=1} |y_i-f(x_i)| $$
L2 norm is based on least squared deviation, with squared deviation being calculated:
$$ LSD = \Sigma^n_{i=1} (y_i-f(x_i))^2 $$
So what is the difference for small vs large nudges?
From the perspective of $AD$, if we think about it in terms of its proportion of mean difference then the nudge in $x_i$ will account for less than 1 proportion of the mean difference. if it is smaller than the mean difference. It will be greater than 1 if the nudge is large. The amount it influences the sum is linearly dependent on the magnitude of the nudge.
From the perspective of $LSD$, if we think about it in terms of its proportion of mean variance then the nudge in $x_i$ will be <1 if it is smaller than the mean variance. The amount it influences the sum is dependent on the square magnitude of the nudge. It will be greater than 1 if the nudge is large. Note that if we square number less than 1 it becomes smaller, which means L2 de-emphasises any small nudges.
I don't think the figure covers enough range, if the variance of the nudge is larger than the mean variance then L2 norm errors will in fact grow quicker than L1 error.
There is a range of nudges over which L2 is expected to be more stable (where the variance of the nudge is less than the mean variance) and a region where it is less stable (where the nudge is large). In between there is a region where the two will be more comparable in terms of error, based on the interaction of $AD$ and $LSD$. This rapid escalation as nudge grows is covered by @MotiN in his answer, if the bounds are loose, L2 can weaken dramatically as the nudges grow.
Comment: Nice point on the fact that $L_1$ grows faster than $L_2$ for small changes - obvious in hindsight.

L1 vs L2 stability?