Proper lme4 equation with fixed and random effects for group differences

Question: I  have a dataset comprised of whole fish concentrations (LAB_RESULT.x), filet fish concentrations (LAB_RESULT.y), and species name (COMMON_NAME.x). I am attempting to run a linear mixed effects model to determine the relationship between filet to whole fish concentrations by species, and if there is a difference among species in this model. This is a step to eventually see if a single linear model works for all species or if there is significant differences between species.

I put together the model:
lmer(LAB_RESULT.x ~ COMMON_NAME.x + (1|LAB_RESULT.y),
     data = SHFvFilet)

and get an output, but am struggling to determine if this is the proper equation for this question, as well as how to properly determine what the output is representing. From my understanding this is showing that for individual species whole fish concentration, species such as Largemouth Bass are significantly different than the overall model, and should not be included in an overall model, while those that are not significant could be included in an overall model. I am also using the lmerTest package with Satterthwaite's method being used to produce p values.
Output:
Comment: @levane07 May I suggest some initial data exploration by plotting. This will help to understand if the nature of the relationship is linear or curvilinear and requiring data transformation or fitting on a link scale which will help with specifying the model formula. The first step here seems to be to step back and start simply, and build complexity into your modeling as is necessary to avoid breaking modeling assumptions. It seems maybe there is some confusion over what each specification actually does e.g including random slopes and/or intercepts.
Comment: Welcome to CV. It looks like you specify LAB_RESULT.y as the random intercept. Why? This doesn't make much sense.
Comment: Thanks! LAB_RESULT.y was placed as the random intercept from my interpretation of page 4 of the lmer vignette (https://cran.r-project.org/web/packages/lme4/vignettes/lmer.pdf). It created the output syntax (where COMMON_NAME.x was shown as the individual fixed effects), where as others that I tried (such as lmer(LAB_RESULT.x ~ (1|COMMON_NAME.x) + LAB_RESULT.y,
data = SHFvFilet) ) did not.
Comment: In the example on page 4, observations are nested within subjects (i.e., there are several observations for each subject). In your case, the nesting factor should be COMMON_NAME.y. Just because you get some output when you specify LAB_RESULT.y as the random intercept does not justify this modeling decision.
Comment: Maybe what you are looking for is a different model. Have you considered an interaction between species and filet concentration, say in linear regression, before moving to mixed effects modeling?
Comment: It seems to me that these are nested measurements since they are two measures of the same variable in two parts of the fish. Which would suggest that the syntax from pg. 4 would work `lmer(LAB_RESULT.x ~ (LAB_RESULT.y + (COMMON_NAME.x|LAB_RESULT.y), data = SHFvFilet)). Would it not?
Comment: I don't think so, but @levane07 should clarify further. The formula does not make sense for me in any case.
Comment: There is a linear relationship between filet and whole, without a data transformation step, that was found using linear regression. From that, I was looking to determine if a single model works for all, or if individual species may be different from than an all encompassing model. To answer dandrews, both the equation shown on page 4 of the lmer vignette and the equation dandrews constructed result in an error and a result, but I dont believe it to be the one I am looking for as it covers all species together, not individually (updated in the original post). Let me know if more info is needed!
Comment: If you have convergence issues, your data doesn't support the random effects structure. After trying different optimizers (see `help("convergence")`) the only solution is usually to simplify the random effects structure. However, if you need individual coefficients for the species, you should just model it as a fixed effect. Of course, you might run into issues of overfitting ...
Answer: I agree with the advice that you should plot your data.
However: I'm sorry, but the mixed model you propose doesn't make any sense. This is obvious, because a continuous variable cannot be the grouping variable of a random effect.

I have a dataset comprised of whole fish concentrations (LAB_RESULT.x), filet fish concentrations (LAB_RESULT.y), and species name (COMMON_NAME.x). I am attempting to run a linear mixed effects model to determine the relationship between filet to whole fish concentrations by species, and if there is a difference among species in this model. This is a step to eventually see if a single linear model works for all species or if there is significant differences between species.

OK, so if there is no impact of species, your model would look like this:
fit0 <- lm(LAB_RESULT.y ~ LAB_RESULT.x, data = DF)

I believe you can just use a fixed-effects model including the species:
fit1 <- lm(LAB_RESULT.y ~ LAB_RESULT.x * COMMON_NAME.x, data = DF)

You can then check if the species effect is significant using the anova function:
anova(fit1, fit0)

Obviously, you will need to do the usual checks regarding model assumptions.
If you have many species, a mixed-effects model might be an alternative. With a random intercept:
fit2 <- lmer(LAB_RESULT.y ~ LAB_RESULT.x + (1 | COMMON_NAME.x), data = DF)

or with a random intercept, slope and their covariance:
fit3 <- lmer(LAB_RESULT.y ~ LAB_RESULT.x + (LAB_RESULT.x | COMMON_NAME.x), data = DF)

Again, you can check significance of the species effect using anova:
anova(fit3, fit2, fit0) #test random slope and test random intercept
anova(fit3, fit0) #test the entire species effect

Again, assumptions need to be checked.
Comment: I don't think you understand what a random effect is. I encourage you to read this book: https://link.springer.com/book/10.1007/978-0-387-87458-6
Comment: Hi Roland thank you so much, this makes a lot(!!!) more sense than what I was attempting, and was extremely thorough and easy to follow. After following this and confirming all assumptions this seems to me what I am looking for. Thank you!
Comment: One more quick question @Roland , for fit 3 the COMMON_NAME.x is being shown as a random effect, instead of a fixed effect. to show as a fixed effect would that just be: fit3 <- lmer(LAB_RESULT.y ~ LAB_RESULT.x * COMMON_NAME.x, data = DF), then any further random effects could be added with "(1|random effect)" ?


How do you hand over files to the user?

Question: in a #WASM / #UNO-platform project, how do you hand over files to the user?
In my case I'm generation locally a PDF and had to download it or display it in the browser.
Any clue?
Regards,
Michael
Answer: There's no API to do that directly, yet. But you can create a data: url on an anchor (a) HTML element.
For this you'll need to create some JavaScript. Here's how you can do it:

IMPORTANT: following code will only work with very recent version of Uno.UI. Version starting with v3.0.0-dev.949+

Create a ContentControl for the  tag
[HtmlElement("a")]
public partial class WasmDownload : ContentControl
{
    public static readonly DependencyProperty MimeTypeProperty = DependencyProperty.Register(
        "MimeType", typeof(string), typeof(WasmDownload), new PropertyMetadata("application/octet-stream", OnChanged));

    public string MimeType
    {
        get => (string) GetValue(MimeTypeProperty);
        set => SetValue(MimeTypeProperty, value);
    }

    public static readonly DependencyProperty FileNameProperty = DependencyProperty.Register(
        "FileName", typeof(string), typeof(WasmDownload), new PropertyMetadata("filename.bin", OnChanged));

    public string FileName
    {
        get => (string) GetValue(FileNameProperty);
        set => SetValue(FileNameProperty, value);
    }

    private Memory<byte> _content;

    public void SetContent(Memory<byte> content)
    {
        _content = content;
        Update();
    }

    private static void OnChanged(DependencyObject dependencyobject, DependencyPropertyChangedEventArgs args)
    {
        if (dependencyobject is WasmDownload wd)
        {
            wd.Update();
        }
    }

    private void Update()
    {
        if (_content.Length == 0)
        {
            this.ClearHtmlAttribute("href");
        }
        else
        {
            var base64 = Convert.ToBase64String(_content.ToArray());
            var dataUrl = $"data:{MimeType};base64,{base64}";
            this.SetHtmlAttribute("href", dataUrl);
            this.SetHtmlAttribute("download", FileName);
        }
    }
}

Use it in Your XAML Page
<myControls:WasmDownload FileName="test.txt" x:Name="download">
    Click here to download
</myControls:WasmDownload>

Note you can put anything in the content of your control, as any other XAML ContentControl.
Set the File Content in Code Behind
Loaded += (sender, e) =>
{
    download.MimeType = "text/plain";

    var bytes = Encoding.UTF8.GetBytes("this is the content");
    download.SetContent(bytes);
};

Result

Direct support by Uno
There is a PR #3380 to add this feature to Uno natively for all platforms. You can also wait for it instead of doing custom way.
The PR for FileSavePicker has been merged and the feature is now available in package Uno.UI since version 3.0.0-dev.1353.

How do you hand over files to the user?


Must "first-time bars" and "second-time bars" appear together?

Question: From http://en.wikipedia.org/wiki/Repeat_sign

When a repeat calls for a different ending, number brackets above the bars indicate which to played the first time (1), which to play the second time (2), etc. These are called "first-time bars" and "second-time bars," or "first and second endings."

Is there a case when you can only use one of them?
Comment: A second ending implies that a first ending must exist, and a "first ending only" would just use a regular repeat sign.
Answer: By definition, there isn't. You only need the first-time and second-time bars when you have two different endings.
When you don't have different endings, you don't need to use either first-time or second-time bars. When you do have at least two different endings, you must use both bars. There may be more, too -- some pieces have a common repeated part and more than 2 different endings.
So, it's a "both or nothing" kind of deal.
Comment: You can, however, have the "first-time bar" be more than just the first time ("1–3."); the "second-time bar" will still be the last ("4.").
Comment: Or you can even do weird things like ("1, 3.") and ("2, 4.").
Answer: if the music in the last bar/s of the phrase is identical, you'd use a repeat barline at the end of the phrase, but not put the 1st-time line over the last bar/s.  The performer would play the whole phrase identically twice (in the absence of further performance instructions such as 'first time f, second time p',) including the final bar/s.


Difference between Pooled variance and Combined variance

Question: Could someone explain for me what the difference between Combined variance and pooled variance is?
I have couple of groups (more than 2) with different sample size, I want to calculate the overall variance, Std, SE and CI... May I know which method is more appropriate for overall variance?
I found below formula:

Combined variance

$$S_c^2 = \frac{n_1[S_1^2+(\bar X_1 - \bar X_C)^2] +n_2[S_2^2+(\bar X_2 - \bar X_C)^2] }{n_1 + n_2}\,,$$
where $$\bar X_c=\frac{n_1\bar X_1+n_2\bar X_2}{n_1+n_2}$$

Pooled variance

$$ S_p^2 = \frac{(n-1)S_x^2 + (m-1)S_y^2}{n + m -2}$$
Comment: Here is a MathJax tutorial for typesetting math: https://math.meta.stackexchange.com/q/5020/321264.
Comment: Thank you. Should I edit my post?
Comment: Editing has been done.
Comment: There isn't any difference between "pooled" and "combined". Your $S_c^2$ is a valid expression in general when you just have some data. The $S_p^2$ is usually used in the context of inference on the unknown population variance. Both are for two groups of observations.
Comment: Thank you for your reply. I thought the same that combined and pooled variance are the same, but when I applied these two formula on my samples, got completely different numbers.
Answer: The above commenter's statement is incorrect. There are two differences between the formula you are calling "combined" variance and the formula you are calling "pooled" variance. One difference is relatively minor, while another difference is more substantive. To homogenize notation, let me index my groups as 1 and 2 in the pooled variance case as well, so
$$S_p^2 = \frac{(n_1-1)S_1^2 + (n_2 - 1) S_2^2}{n_1 + n_2 - 2}$$
A first difference between the pooled and combined variance formulas you have displayed is that the pooled variance formula given here contains what is known as a "degrees of freedom" correction. The point is that the usual sample variance formula divides by $n-1$ instead of $n$, as doing so produces an unbiased estimate for the population variance. When $n$ is large, this distinction hardly matters, so in discussing the second distinction, I will largely ignore it and consider the following version of the pooled variance formula
$$S_p^2 = \frac{n_1 S_1^2 + n_2 S_2^2}{n_1 + n_2}$$
After making this change, it is pretty easy to see that this pooled variance formula is still not the same as the combined variance formula. To understand the distinction, it is useful to think about populations rather than samples. In population, a well known result in probability is the so-called law of total variance, which states that for random variables $Y,X$,
$$\mathrm{Var}(X) = \mathbb E[\mathrm{Var}(X|Y)] + \mathrm{Var}(\mathbb E[X | Y])$$
In the context of pooled/combined variances, we can take $X$ to be the outcome which you are measuring and $Y$ to be which group somebody belongs to, so $Y = 1$ or $Y= 2$. The law of total variance would then read
$$\begin{aligned}\mathrm{Var}(X) &= \mathrm{Var}(X | Y = 1) \mathrm{Pr}(Y = 1) + \mathrm{Var}(X | Y = 2) \mathrm{Pr}(Y = 2)\\
&= \left(\mathbb E[X | Y = 1] - \mathbb E[X]\right)^2\mathrm{Pr}(Y = 1) + \left(\mathbb E[X | Y = 2] - \mathbb E[X]\right)^2\mathrm{Pr}(Y = 2)\end{aligned}$$
Now, with this formula in mind, we can better understand the formula for $S_c^2$. Note the following:

$n_1 / (n_1 + n_2)$ is the sample analogue of $\mathrm{Pr}(Y = 1)$ while $n_2 / (n_1 + n_2)$ is the sample analogue of $\mathrm{Pr}(Y = 1)$
$S_1^2$ is the sample analogue for $\mathrm{Var}(X | Y = 1)$ while $S_2^2$ is the sample analogue for $\mathrm{Var}(X | Y = 2)$.
$\bar X_1$ is the sample analogue of $\mathbb E[X | Y = 1]$ while $\bar X_2$ is the sample analogue of $\mathbb E[X | Y = 2]$.
$\bar X_c$ is the sample analogue of $\mathbb E[X]$.

From this, you can see that $S_c^2$ simply decomposes the variance of the outcome $X$ in terms of its "within group" and "between group" components. By contrast, $S_p^2$ only includes the variance within group. Thus, $S_c^2$ and $S_p^2$ correspond to subtly different questions. In particular, $S_c^2$ asks "if I drew a random observation from the entire population (i.e. where my draws could randomly come from group 1 or group 2), what is the variance of that draw" whereas $S_p^2$ asks the question "if I drew a random observation from the population but also recorded whether the observation was from group 1 or from group 2, how much variance would the prediction error of that draw be if I took the mean from the respective group as my prediction?"
One other thing to add. Because $S_p^2$ and $S_c^2$ are related via a variance decomposition and variances can only be non-negative, this implies that (unless $n_1, n_2$ are very small), we should expect that to the extent that the two numbers are different, $S_c^2$ should be larger.
Comment: What I wanted to say is that "combined variance" and "pooled variance" are used interchangeably. OP's "combined variance" is also someone's "pooled variance". I am sure the two expressions are not equal; they are not supposed to be equal since they are used in different contexts.
Comment: As explained in my answer, the pooled variance gives an answer to a very specific (and to your point, in many cases not necessarily conceptually useful!) question: what is the mean squared error of the best predictor of $Y$ if all you know is group identity. For instance, pooled variance divided by combined variance is nearly equivalent to $1-R^2$ of a regression of $Y$ on indicators for group identity.
Comment: One example where "average" variance reduction _is_ a useful statistic though, is the context of regression adjustment. In that context, if you control for group identity optimally when analyzing data from an experiment, the factor by which you shrink the variance of the resultant estimator is essentially pooled variance divided by combined variance. See, e.g., https://www.nber.org/papers/w30756
Comment: @stats_model, Thank you so much for your clear explanation.
However I am not still sure when use $S_c^2$ or $S_p^2$.
Let's say, I have 4 groups of independent samples with 5,4,3,6 size. If I want to make the overall mean and Variance then make a 95% CI for overall mean, should I consider $S_p^2$ ?
Thanks again
Comment: The answer is unfortunately "it depends". The world where what you call $S_p^2$ is correct is one where the size of the samples is fixed ex ante. The world where $S_c^2$ makes sense is one where your samples are drawn i.i.d. from some meta-population, and which group a given observation ends up in is by chance.
Comment: Thank you so much.
And also I found  "Weighted variance", May  I know what's the difference between weighted variance with Pooled and combined variance?
$$\text{Var}(X)=\frac{1}{\sum_i \omega_i}\sum_i\omega_i(x_i - \text{mean}(X))^2$$
where
$$\text{mean}(X):=\frac{1}{\sum_i \omega_i}\sum_i \omega_i x_i$$
And Where I use each of them?
Which one is more appropriate for overall variance of couple independent groups with different small sample size (given the equal population variance assumption)?
Comment: Hey, when you say "The law of total variance would then read ...", how do you get to the second equality? And what does it have to do with the law of total variance? I can't follow.
Comment: Hmm, you misstated the law of total variance, this leads to some confusion, I think.
Comment: Oh that's embarrassing. Let me fix that...
Comment: Just want to add this:
 "if I drew a random observation from the population but also recorded whether the observation was from group 1 or from group 2, how much variance would the prediction error of that draw be if I took the mean from the respective group as my prediction ***assuming that the variance of the two groups are equal***?"
Comment: Actually, the pooled variance formula above makes no such heteroskdasticity assumption. This is because in the formula, we compute a weighted average of $S_1^2$ and $S_2^2$. The former is the variance conditional on being in group 1 and the latter is variance conditional on group 2, so this formula already includes adjustments for different variances.
Comment: @stats_model correct me if I am wrong, but assuming equal variance gives more conceptual meaning when using pooled variance or else we could just use individual groups variance. We need not combine the groups,right? What motivation would one require then to calculate pooled variance?
Comment: @stats_model thanks a lot for the info.


Hide overflow of div on edges

Question: I got an main div called outerdiv with a div-element inside called innerdiv.
Because the outerdiv got the "border-radius" attribute and so the innerdiv conceals the nice edges of the outerdiv.
<> I hope you could get what I mean. But have a look at the code:

#outerdiv {
  position: fixed;
  width: 80%;
  left: 50%;
  margin-left: -40%;
  height: 30%;
  background: red;
  margin-top: 5%;
  border-radius: 20px 20px 0px 0px
}

#innerdiv {
  position: absolute;
  width: 100%;
  background: green;
  height: 40%;
  left: 0%;
  top: 0%;
}

>MOVE YOUR MOUSE POINTER OVER THE RED AREA TO GET WHAT I MEAN.

To my question: How can I make the overflow of the innerdiv hidden to don't let it cover up the nice edges of the outerdiv.
Note:  (The simplest solution) - Just adding the same border-radius attribute to the innerdiv is NO SOLUTION IN THIS CASE.
What I've tried to add is the attribute overflow but it doesn't seems working this way:
#innerdiv {
    overflow: hidden;
    overflow-x: hidden;
    overflow-y: hidden;
}

Edit: I'd like the green element to not display above the red corners.
Comment: why are the answers downvoted?
Answer: overflow: hidden does work, you need to apply it on your #outerdiv, not the #innerdiv.

#outerdiv {
  position: fixed;
  width: 80%;
  left: 50%;
  margin-left: -40%;
  height: 30%;
  background: red;
  margin-top: 5%;
  border-radius: 20px 20px 0px 0px;
  overflow: hidden;
}

#innerdiv {
  position: absolute;
  width: 100%;
  background: green;
  height: 40%;
  left: 0%;
  top: 0%;
}

Comment: Thanks for the solution :)

Hide overflow of div on edges


WebView and GridView into ScrollView, View too large to fit into drawing cache

Question: I have a layout memory issue. When I have a large webview it doesn't shows anything and the logcat shows "View too large to fit into drawing cache".
The layout is:
<code><ScrollView
    android:id="@+id/scrollNoticia"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:background="@drawable/ficha_curva"
    android:layout_below="@+id/linea"
    android:scrollbars="none"
    >

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:paddingBottom="12dp"

         >

        <WebView
            android:id="@+id/webViewNoticia"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:scrollbars="none"

             />

        <GridView
            android:id="@+id/gridGaleria"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_below="@id/webViewNoticia"
            android:horizontalSpacing="4dp"
            android:verticalSpacing="4dp"
            android:numColumns="4" >
        </GridView>
    </RelativeLayout>
</ScrollView>
</code>
Comment: Did u get any solution for this??
Comment: Yes I got a solution which is more of a workaround, i calculate the webview height and if it is greater than a particular height i just change the height of webview to that particular height.
I am using this in my current app and it works fine
Answer: Neither WebView or GridView need to be embedded in a scrollview.
The WebView can scroll by itself when the content size over the screen size, and the GridView too.
In the normal way, the GridView just create so many child views as are visible. Once a view goes out of screen, it will be reused. So if you embed a GridView in a ScrollView, maybe you will wrong the reuse pattern.
Comment: This is not helpful. He (and I) needs it in a scroll view from various reasons.
Answer: Try disabling the hardware acceleration : http://developer.android.com/guide/topics/graphics/hardware-accel.html
See WebView in ScrollView: "View too large to fit into drawing cache" - how to rework layout? for more informations.


No. of ways to shuffle a card

Question: How many ways are there to shuffle N cards such that exactly one card is in the same position?(Assuming that initially the card no. 1 is in the first position,card no.2 is in
the second position and so -on).
My attempt:Initially I counted no. of ways for small values of N by brute force.But I'm thinking that it can be solved using permutation and combination(along with inclusion-exclusion principle)but don't know how to apply it.
Answer: There are $N$ possibilities for the card that stays in the same position. The other $N-1$ cards are shuffled such that they all appear in a different place. This is called a derangement. Using the formula in the link, we get the answer $N\left[\frac{(N-1)!}{e}\right]$.

No. of ways to shuffle a card


Is $\liminf f(x)$ always unique?

Question: Searching for the question doesn't seem to give any relevant answers on stack exchange or on google. I know that $\liminf_{n \rightarrow \infty} f(x_n)$ is always unique, but I haven't seen a proof for it. Here is my attempt:
Assume $l, l'$ are two valid values of $\liminf_{n \rightarrow \infty} f(x_n)$ for some real valued function $f$ and a sequence $x_n$ that converges to $x_0$ as $n\rightarrow \infty$. Without loss of generality, assume that $l'=l+\eta$ for some $\eta > 0$. From the definition of $\liminf$, we have that for any fixed $\epsilon> 0$, there exists a $\delta > 0$ such that for all $x \in N_\delta(x_0)$: $f(x) > l - \epsilon$ and $f(x) > l' - \epsilon$. Adding these inequalities together and substituting the value of $l'$:
$$f(x) > l - \epsilon + \eta / 2$$
Let $\epsilon = \eta / 2$ so that the above inequality is: $f(x) > l$. But this is a contradiction since the $\liminf$ is the greatest lower bound: for some $x \in N_\delta (x_0)$ we have that $f(x) < l + \eta / 2$ and $\eta$ can be made arbitrarily small.
Is this last point correct? Is the result truly a contradiction or am I missing a step?
Comment: What's your definition? Suprema and infima are unique, and the limits superior and inferior are usually defined in terms of those.
Comment: For *existence* of $\liminf$, we need to allow possible values $+\infty$ and $-\infty$.  Then your reduction $l'=l+\eta$ is a problem.
Comment: Note $(x_n)$ needn't be convergent for $\liminf f(x_n)$ to exist, so this assumption is stronger than necessary.
Comment: Since I'm still familiarizing myself with the concept, please excuse my potential ignorance, but I believe it follows directly from the fact that limits of sequences are unique, hence why there wouldn't be a proof for it. Define the sequence $\left(\inf\limits_{i\geq n} x_i\right)_{n=m}^{\infty}$. Assuming this sequence converges, the limit of the sequence is unique, hence $\lim\limits_{n\rightarrow\infty}\inf x_n$ is unique.
Comment: That's a good point. I don't think we need the assumption on whether it converges. Taking your idea: $\liminf x_n = \sup_m \inf{n\geq m} x_n$. The infimum is unique for each $m$ and so is the supremum.
Comment: If you consider a sequence (assumed real sequence) being actually _a function_ $x:\mahtbb N\to\mathbb R$, then your $f(x_n)$ is simply a composition $(f\circ x)(n).$ Name the composed function: $g = f\circ x$ and your question becomes: is $\liminf$ of an arbitrary sequence $g_n = g(n)$ always unique?
Answer: I am giving you a demonstration , I hope it helps you.
Suppose $f(x_n)$ has at least two limits $M,L$ such that $L<M$. Considering $\epsilon = \frac{M-L}{2}$, It is well defined because $\epsilon > 0 $.As L is a Limit of $f(x_n)$ exists $N_1$ such that:
$$\forall n \geq N_1 , L - \epsilon < f(x_n) < L +\epsilon.$$
And As M is a Limit of $f(x_n)$ exists $N_2$ such that:
$$\forall n \geq N_2 , M - \epsilon < f(x_n) < M +\epsilon.$$
Defining $N = \max{(N_1,N_2)}$ therefore $f(x_N)$ verify both inequalities, in particular :
$$M - \epsilon < f(x_N) < L +\epsilon.$$
As $\epsilon = \frac{M-L}{2} \Rightarrow M-\epsilon = \frac {M+L}{2}$ and $L+\epsilon = \frac{M+L}{2}$, finally:
$$ \frac{M+L}{2}$ < f(x_N)< \frac{M+L}{2}$$.
In conclusion as it is a contradicction $f(x_n) $could only have one limit.

Is $\liminf f(x)$ always unique?


Better HTML-annotation practice for an element that is a member of multiple classes

Question: This isn't quite the same question as many others that it might resemble.
Consider a table full of sortable columns. The ascending/descending sort order that the visitor wants is kept in a  in the column's . JavaScript code refers to that  when the user calls for a sort; and the code changes the contents of the  as the user toggles the sort order.
Considering both the code and the HTML, is it "better" (cleaner, JS-friendlier, more concise, nicer-looking) practice to annotate the  part of the  this way:
content

or this way:
content

The first way makes it easier for code to track down the column's sort order. The second way rids the HTML of a lot of cruft. As it seems to me.
Thanks!
Comment: How about HTML5 `data` attributes?
Answer: It seems like what you're trying to do is to add a key=value pair to the element that says "the sortorder of this is 'ascending'". For this I'd use HTML5 data attributes:
…

They should be backwards-compatible and jQuery should be able to find / read them on any browser they support using .data().


Making a calculator in android. The app shows up on the emulator with no errors but when I press the calculate button it crashes

Question: I'm new to this and I'm trying to follow some code from another answer. I have three text fields to enter the numbers that are used for the calculation and a button below that is supposed to change some text below it to the answer. There are no errors shown when I run the app but when I press the button the app crashes and I get error messages in the log. This is the code:
public class CalculatorActivity extends AppCompatActivity {

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_calculator);
    Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
    setSupportActionBar(toolbar);

    class NewActivity extends CalculatorActivity {
        //Declare textviews as fields, so they can be accessed throughout the activity.
        EditText editTextSubsPrevious;
        EditText editTextSubsNow;
        EditText editTextDays;
        TextView calcText;
        Button calcButton;

        @Override
        public void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.activity_calculator);

            //Bind the EditText views

            editTextSubsPrevious = (EditText) findViewById(R.id.editTextSubsPrevious);
            editTextSubsNow = (EditText) findViewById(R.id.editTextSubsNow);
            editTextDays = (EditText) findViewById(R.id.editTextDays);

            calcButton = (Button) findViewById(R.id.calcButton);{
                View.OnClickListener onClickListener = new Button.OnClickListener() {
                    @Override
                    public void onClick(View v) {
                        //When the button is clicked, call the calculate method.
                        calculate();
                    }
                };
            }
        }

        public void calculate(){
            //get entered texts from the edittexts,and convert to integers.
            Double value1 = Double.parseDouble(editTextSubsPrevious.getText().toString());
            Double value2 = Double.parseDouble(editTextSubsNow.getText().toString());
            Double value3 = Double.parseDouble(editTextDays.getText().toString());
            //do the calculation
            Double difference = (value2 - value1) / 7;
            Double percent = (difference / 325);
            Double amount;
            amount = value2 * Math.pow(1 + percent, value3);
            calcText.setText(amount.toString());

        }

    }
}

}
Here is the error message:
FATAL EXCEPTION: main
Process: com.pausetheapp.futurefollowerandsubscribercalculator, PID: 3420
java.lang.IllegalStateException: Could not find method sendMessage(View) in a parent or ancestor Context for android:onClick attribute defined on view class android.support.v7.widget.AppCompatButton with id 'calcButton'
     at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.resolveMethod(AppCompatViewInflater.java:325)
     at android.support.v7.app.AppCompatViewInflater$DeclaredOnClickListener.onClick(AppCompatViewInflater.java:284)
     at android.view.View.performClick(View.java:5198)
     at android.view.View$PerformClick.run(View.java:21147)
     at android.os.Handler.handleCallback(Handler.java:739)
     at android.os.Handler.dispatchMessage(Handler.java:95)
     at android.os.Looper.loop(Looper.java:148)
     at android.app.ActivityThread.main(ActivityThread.java:5417)
     at java.lang.reflect.Method.invoke(Native Method)
     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

Here is the xml file:
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:paddingBottom="@dimen/activity_vertical_margin"
android:paddingLeft="@dimen/activity_horizontal_margin"
android:paddingRight="@dimen/activity_horizontal_margin"
android:paddingTop="@dimen/activity_vertical_margin"
app:layout_behavior="@string/appbar_scrolling_view_behavior"
tools:context="com.pausetheapp.futurefollowerandsubscribercalculator.CalculatorActivity"
tools:showIn="@layout/activity_calculator">

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:textAppearance="?android:attr/textAppearanceLarge"
    android:text="@string/textViewTitle"
    android:id="@+id/textViewTitle"
    android:textAlignment="center"
    android:layout_alignParentTop="true"
    android:layout_alignParentLeft="true"
    android:layout_alignParentStart="true" />

<EditText
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:inputType="number"
    android:ems="10"
    android:id="@+id/editTextSubsPrevious"
    android:layout_marginTop="28dp"
    android:layout_below="@+id/textViewTitle"
    android:layout_centerHorizontal="true"
    android:hint="@string/previousSubsTextField"
    android:width="300dp" />

<EditText
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:inputType="number"
    android:ems="10"
    android:id="@+id/editTextSubsNow"
    android:layout_below="@+id/editTextSubsPrevious"
    android:layout_centerHorizontal="true"
    android:layout_marginTop="24dp"
    android:width="300dp"
    android:hint="@string/currentSubsTextField" />

<EditText
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:inputType="number"
    android:ems="10"
    android:id="@+id/editTextDays"
    android:hint="@string/daysTextField"
    android:width="300dp"
    android:layout_marginTop="30dp"
    android:layout_below="@+id/editTextSubsNow"
    android:layout_alignLeft="@+id/editTextSubsNow"
    android:layout_alignStart="@+id/editTextSubsNow" />

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:textAppearance="?android:attr/textAppearanceSmall"
    android:text="@string/accuracyText"
    android:id="@+id/textViewAccuracy"
    android:layout_below="@+id/editTextDays"
    android:layout_alignLeft="@+id/editTextDays"
    android:layout_alignStart="@+id/editTextDays"
    android:textSize="10dp" />

<Button
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="@string/calculateButton"
    android:id="@+id/calcButton"
    android:layout_marginTop="42dp"
    android:layout_below="@+id/textViewAccuracy"
    android:layout_centerHorizontal="true" />

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:textAppearance="?android:attr/textAppearanceLarge"
    android:text="@string/calcDays"
    android:id="@+id/calcText"
    android:layout_marginTop="22dp"
    android:textSize="25dp"
    android:layout_below="@+id/calcButton"
    android:layout_centerHorizontal="true"
    android:gravity="center" />

I just added this line to the java file.
calcText = (TextView) findViewById(R.id.calcText);

Comment: Actually, I just corrected the error but the button still isn't working.
Comment: It looks like that you are not creating the calcText in your code.. so if you will try to call setText on calcText (which is null because i don't see where you created it) the app will crash..
Comment: Could you please share your layout file as well?
Comment: Your class's blocks are messed up ! Please notice that you are defining a sub-class inside the method onCreate. I think you miss typed the code into the SO question field. If this is really your code, try to declare your NewActivity class outside of a method implementation.
Answer: There are two possibilites of error
1> You have not defined calcText
 calcText.setText(amount.toString());

2> Or May be
 Double value1 = Double.parseDouble(editTextSubsPrevious.getText().toString());
        Double value2 = Double.parseDouble(editTextSubsNow.getText().toString());
        Double value3 = Double.parseDouble(editTextDays.getText().toString());

editTextSubsPrevious,`editTextSubsNow`,`editTextDays` => does not contain numbers so that it can be parsed to `Doubles`

Answer: Probably, you have following attribute in your layout XML file:
 android:onClick="sendMessage"

Then, when you click, android search for method sendMessage(View view) and it does not find (because you did not implemented)

Could not find method sendMessage(View) in a parent or ancestor Context for android:onClick attribute defined on view class android.support.v7.widget.AppCompatButton with id 'calcButton'

So, since your are setting a onClickListener at runtime, I think you can delete android:onClick="sendMessage" from layout file.
Also, line below should be fixed as follows:
calcButton = (Button) findViewById(R.id.calcButton);
calcButton.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        //When the button is clicked, call the calculate method.
        calculate();
    }
});

Comment: I added the calcText to the java and I got rid of the "sendMessage" in the xml file. Unfortunately, it's still not working when I press the calculate button. Nothing changes.
Comment: Ok.. I fould several other errors.. For example, you created one activity inside another one.. I'll simulate your code here and return with updates.
Comment: @PauseTheCat Could you please share also activity_calculator to prepare proper fix to your code?
Answer: In your layout most probably you have assigned sendMessage() method to handle onClick event and have not created any such method in activity.
check for android:onClick="sendMessage" in your layout.
Either create a method sendMessage(View view) and write your code there or remove it and use your calcBtn.setOnClickListner(View.OnClickListner Object).
Answer: If your app crashes in response to a button click it could be due to the following:

The function you registered for the onClick attribute of the button
in the layout .xml file does not exist in the activity. You can make sure the function name in the onClick attribute matches the function name you mean to call in your activity.
Something in the onClick function causes an Exception. If this is the case you can use the debugger by setting a breakpoint inside
that function and inspecting your code and values while you step
through the program. Debug Your App
Answer: if u r using this
    calcButton = (Button) findViewById(R.id.calcButton);{
                    View.OnClickListener onClickListener =         new Button.OnClickListener() {
                       @Override
                        public void onClick(View v) {
                           //When the button is clicked, call the calculate method.
                            calculate();
                        }
                    };
                }

u should use
    android:onClick="onClick"

else u need to  use
                    calcButton.setOnClickListener(new View.OnClickListener(){
                    @Override
                     public void onClick(View v) {
                        //When the button is clicked, call the      calculate method.
                                 calculate();
                                }

      });


How to provide a custom option for an exposed filter in Views?

Question: I have a site in which members can upload pictures of themselves (imagine a personals site).  Users can upload up to five pictures, and each time they upload a picture, a rule increases the value of a custom field, Number of pictures.
Members can search for other members using a view.  This view has a number of exposed filters.
What I would like to do is add another option, Show members with pictures only.  Ideally, this would be a checkbox.  However, I have no idea how to add such a thing to a view.  I could add the Number of pictures field as an exposed filter, but since this is an integer, users would have to type "1", which is ugly and unintuitive.  How might I start to go about this?
Answer: if you are not able to achieve what you need with views default options.
You can create your custom views handler. In this case, it would be a filter handler.
https://api.drupal.org/api/views/handlers!views_handler_filter.inc/group/views_filter_handlers/7
https://drupal.org/node/1239866
Extending views functionallity to provide an approach to your problem
Answer: You are looking for Sexy Exposed module. I think you can do it with this module, It makes any views exposed filter's select list as checkbox list.

This module makes any views exposed filter's select list sexy using
  the jQuery Dropdown Check List plugin.
After installing this module, you will get a new checkbox on your
  view's exposed filter settings form called 'Make it sexy'. By ticking
  it your exposed filter will be pimped up by some hot javascript magic
  thanks to jQuery Dropdown Check List.
Comment: This looks like a useful module, but in this case, the field is not a text value list; it's an integer value.
Comment: Sexy Exposed is not sexy UX.


Is it possible to find the mode of the borrowed chord?

Question: I came across a piece which uses a dorian progression:
i - IV - VII - III - ii7 - v - V

V is a borrowed chord, but is it possible to determine from which mode this chord was borrowed? (lydian, locrian, ionian)
Answer: If it's just that isolated V chord that is borrowed, then no, I don't see any way to determine whether that chord is borrowed from Ionian or Lydian.
(It won't be borrowed from Locrian, because the Locrian V is built a half step lower. In D Locrian, V is built on A♭, whereas it is built on A♮ in D Ionian and D Lydian.)
Now, if there are other elements in the work that also use borrowed material---let's say, for instance, that another section of the work clearly borrows from Lydian---that could be a good reason to suggest it's borrowed from Lydian, if only for the analytic consistency.
I'll also say that, normally, "mode mixture" and "borrowed chords" deal with switches between major and minor; I'm not used to hearing these terms in conjunction with modes, but maybe I'm just not in the right circles. In your cited instance, I think most would simply say this progression "uses a major V."
Answer: Dorian is a minor mode, and the v will be minor too. This is similar to the natural minor. The melodic and harmonic minors usually take the dominant as V, making the leading note a semitone under the tonic - far more convincing. That's what's happened here. No particular borrowing, just making things sound more tuneful.


Give an example of an open cell in a cell complex for which the image of the characteristic map is not a closed cell.

Question: Let $(X, \Gamma)$ be a cell complex, and let $e \in \Gamma$ be a open cell of dimension $n \geq 1$.
I want to try and construct an example of a characteristic map $\Phi : D \to X$ (where $D$ is some closed $n$-cell) for which $\Phi[D] = \operatorname{Cl}_X(e)$ is not a closed cell. To do that I need to find a $\Phi$ that is not injective on the boundary of $D$.
I am not sure how to construct such an example (and in this example, I'd need to choose, $X$, $\Gamma$, $e$ and $\Phi$ appropriately so that $\Phi$ is not injective on the boundary)
The biggest problem is having that $\Phi$ must be continuous, for example we could take $X = [0, 1]$ and $\Gamma = \{(0, 1) \{0\}, \{1\}\}$ and define $\Phi : [0, 1] \to [0, 1]$ by $\Phi(x) = x$ for $x \in (0, 1)$ and $\Phi(x) = 0$ on $x \in \{0, 1\}$, and $\Phi$ satisfies all the criteria for being a characteristic map except for being continuous.

Note that this is not a homework problem, but rather a problem that I came up with while reading a claim in a book and wanting to find an explicit example for myself
Comment: What's the usual cell structure on the circle?
Answer: Here's the simplest example.  Let $X$ be the circle $\{z\in\mathbb{C}:|z|=1\}$ and put a cell complex structure on it as follows.  We have one $0$-cell, which is just $\{1\}$.  We have one $1$-cell, whose characteristic map $\Phi:[0,1]\to X$ is defined by $\Phi(t)=e^{2\pi i t}$.  This is not injective, since $\Phi(0)=\Phi(1)$.
Comment: Thanks for the answer! Just a quick question. don't you mean $z \in \mathbb{C}$ in your answer?
Comment: Oops, yes, I've fixed that.


Dúvida preg_match PHP

Question: Preciso fazer um preg_match para verificar se uma string atende às seguintes condições:

Iniciar com uma letra: a-z;
Conter apenas letras, número e os caracteres ,, ., - e _;

Código atual:
preg_match('/^[0-9]{1,}[a-z0-9.]*[a-z0-9]/', $_POST['login']);

Comment: Se deve iniciar com uma letra, por que colocou `[0-9]{1,}` no início?
Answer: Entendimento:

Iniciar com uma letra: a-z;

Deve iniciar com uma letra de a a z minúscula. OK!

Conter apenas letras, número e os caracteres ,, ., - e _;

Como não informa um número mínimo de caracteres após o primeiro caractere que é obrigatório ("Deve iniciar com uma letra..."), considera-se (pelo que entendi) que do segundo caractere em diante é opcional; mas se tiver, deve obedecer os critérios acima.
Logo:
Dentro dos critérios expostos na pergunta, você poderia usar esta expressão:
^[a-z][a-z\d_,\-.]*$

Explicação:
^             Delimita o início da string.
[a-z]         O primeiro caractere deve ser de "a" a "z" (minúsculas).
[a-z\d_,\-.]  Segundo caractere em diante.
              A partir do segundo caractere, pode ou não ter:
                 [a-z]  letras de "a" a "z" (minúsculas)
                 \d     números
                 _      underline
                 ,      vírgula
                 \-     hífen (escapado)
                 .      ponto
*             Quantificador que irá encontrar nada ou quantas ocorrências tiver
              a partir do segundo caractere.
$             Delimita o fim da string.

Exemplo usando uma função:
<?php
function checa($str){

   return preg_match('/^[a-z][a-z\d_,\-.]*$/', $str);

}

// 0 = false, 1 = true
echo checa("a"); // 1
echo checa("1"); // 0
echo checa("h1-11,11,"); // 1
echo checa("Aa"); // 0
echo checa("1abc_"); // 0
echo checa("aaaaaaaa"); // 1
echo checa("a-----,"); // 1
echo checa("a-12,_#,"); // 0
echo checa(""); // 0
?>

Para verificar, você pode usar um if:
<?php
if(checa("a")){
   // com o valor "a" irá entrar aqui.
   // significa que a string começa com uma letra minúscula
   // e atende ao critério da regex
   echo "passou";
}else{
   echo "não passou";
}
?>

Lembrando que letras maiúsculas e acentuadas serão consideradas
  inválidas.
Comment: Se não me engano, dentro de colchetes, o ponto não precisa de escape (`[.]` é o mesmo que `[\.]`, [veja](https://regex101.com/r/FTKVXP/1/)), mas o hífen sim. Ou seja, `[,-\.]` é "o intervalo entre os caracteres `,` e `.`", que **coincidentemente** só possui o caractere `-` (além dos próprios `,` e `.`), [veja](https://regex101.com/r/DqJQLh/2/). Mas se trocar a ordem dos caracteres, o intervalo fica inválido e só vai funcionar se escapar o hífen: veja [aqui](https://regex101.com/r/DqJQLh/3) a regex inválida e [aqui](https://regex101.com/r/DqJQLh/4) a versão corrigida, com `\-`
Comment: Verdade. Só precisaria escapar o hífen mesmo. Valeu!
Answer: Seguindo seus parâmetros, fiz esse código:
$strings = array(
    'uma string inválida',
    'uma string valida',
    '0uma string invalida',
    'uma string invalida !',
    'uma-string_valida.',
    '@ uma-string_invalida.'
);

foreach ($strings as $str) {
    if (preg_match('/^[^a-z]+|[^a-zA-Z\s\,\.\-\_]+/', $str))
        echo 'String COM caracteres inválidos: '.$str.'
';
    else
        echo 'String SEM caracteres inválidos: '.$str.'
';
}

Retorna:

String COM caracteres inválidos: uma string inválida
String SEM caracteres inválidos: uma string valida
String COM caracteres inválidos: 0uma string invalida
String COM caracteres inválidos: uma string invalida !
String SEM caracteres inválidos: uma-string_valida.
String COM caracteres inválidos: @ uma-string_invalida.

Repare na expressão regular (RegEx):
^[^a-z]+|[^a-zA-Z\s\,\.\-\_]+

Busco qualquer coisa diferente de a-z no início da string:
^[^a-z]+

Ou (|)
Qualquer coisa diferente dos caracteres: a-z, A-Z (letras maiúsculas), \s (espaço), ,, ., - ou _.
[^a-zA-Z\s\,\.\-\_]+

Fora dos seus parâmetros, adicionei as letras maiúsculas e o espaço. Caso não queira, só remover da expressão regular: ^[^a-z]+|[^a-z\,\.\-\_]+.

Dúvida preg_match PHP


How GetCurrentProcessorNumber() works? CPU core of a thread at runtime?

Question: Is there any smooth way to find out the CPU core id of a thread running in a multithreading code during runtime? I tried to use GetCurrentProcessorNumber(), but it seems it is not giving the CPU core id where the individual threads are running. The code I have been using is:
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Runtime.InteropServices;

class S
{
    [DllImport("kernel32.dll")]
    static extern int GetCurrentProcessorNumber();

    static void Main()
    {
        Task t1 = new Task(A, 1);
        Task t2 = new Task(A, 2);
        Task t3 = new Task(A, 3);
        Task t4 = new Task(A, 4);

        int myProcessorNum = GetCurrentProcessorNumber();
        Console.WriteLine("processNo: " + myProcessorNum.ToString());

        Console.WriteLine("Starting t1 " + t1.Id.ToString());
        t1.Start();
        Console.WriteLine("Starting t2 " + t2.Id.ToString());
        t2.Start();
        Console.WriteLine("Starting t3 " + t3.Id.ToString());
        t3.Start();
        Console.WriteLine("Starting t4 " + t4.Id.ToString());
        t4.Start();

        Console.ReadLine();
    }

    static void A(object o)
    {
        int temp = (int)o;
        int myProcessorNum = GetCurrentProcessorNumber();
        Console.WriteLine("Method A &" + "Thread Id: " + temp.ToString() + " and " + "processNo: " + myProcessorNum.ToString());
        B(o);
    }

    static void B(object o)
    {
        int temp = (int)o;
        int myProcessorNum = GetCurrentProcessorNumber();
        Console.WriteLine("Method B &" + "Thread Id: " + temp.ToString() + " and " + "processNo: " + myProcessorNum.ToString());
    }
}

if I recheck the core id, it gives a different number:
static void A(object o)
{
    int temp = (int)o;
    int myProcessorNum = GetCurrentProcessorNumber();
    Console.WriteLine("Method A &" + "Thread Id: " + temp.ToString() + " and " + "processNo: " + myProcessorNum.ToString());
    myProcessorNum = GetCurrentProcessorNumber();
    Console.WriteLine("Method A &" + "Thread Id: " + temp.ToString() + " and " + "processNo: " + myProcessorNum.ToString());
    B(o);
}

Comment: Do you realise that threads can be moved between processors?
Comment: @user2371160 That would depend on a lot of factors, Console.WriteLine for example may temporarily block and cause a thread switch.
Comment: What is it returning that makes you think it's not giving the number of the logical processor back?
Comment: @Joachim: for example, if I call GetCurrentProcessorNUmber in the method A once again and print out the second return it gives a different number. Briefly, if I just the write lines in A twice, the prints of core id's are different.
Comment: @Matthew, Yes but would an individual thread move between processors while just passing from line to line in a method?
Comment: Ok, Thanks for the comments and explanation.
Answer: if you want the same cpu core, you need to affect one (or more) !
http://omegacoder.com/?p=94
[DllImport("kernel32.dll")]

static extern IntPtr GetCurrentThread();

[DllImport("kernel32.dll")]

static extern IntPtr SetThreadAffinityMask(IntPtr hThread, IntPtr dwThreadAffinityMask);

in your prog :
SetThreadAffinityMask(GetCurrentThread(), new IntPtr([nb of your cpu you want]));

it work i was do that on my app


Bayesian estimation of the parameter of a distribution with censored data

Question: I am trying to understand the effect of censored data on parameter estimation, in particular with respect to Bayesian modeling. Here is an illustrating example with some code:
Assume you have $N = n+m$ i.i.d observations from an exponential distribution with parameter $\lambda=3$. We will also define a value $L$ such that any observation less than $L$ is  censored. With this in mind, the following is the likelihood for the data
\begin{align}
\ell(x|\lambda) &= \prod_{i=1}^mF(x)\prod_{i=1}^nf(x)\\
&=\prod_{i=1}^m(1-e^{-\lambda L})\prod_{i=1}^n\lambda e^{-\lambda x_i}
\end{align}
where $m$ and $n$ are the number of censored and noncensored observations.
We also place a prior such that $\lambda\sim\text{Gamma}(0.01,0.01)$. Now, no matter what the level of censoring is, as long as I have a single uncensored data point my models seems to do well recovering the true $\lambda$. However, as soon as I make the entire data set censored, I cannot estimate $\lambda$ well. This doesn't seem too shocking to me given that you don't have any information other than the censoring point, however, I would have thought that the estimate of $\lambda$ would be whatever the prior mean is, which is not the case. Should you be recovering the prior if you have all censored data?
Here is come code that estimates $\lambda$ in an MCMC. To change the percent of censoring you just need to change the value for prob in the quantile() function.
likelihood <- function(x, censor, lambda){

  llik1 <- log(1 - exp(-lambda * x[censor == 1]))
  llik2 <- log(lambda) - lambda * x[censor == 0]
  lik  <- sum(llik1) + sum(llik2)

  return(lik)
}

prior <- function(lambda, sigma){

  prior <- dgamma(lambda, .01, .01, log = TRUE)

  return(prior)
}

posterior <- function(x, censor, lambda){

  post <- likelihood(x, censor, lambda) + prior(lambda)

  return(post)
}

set.seed(4)
N = 100
x = rexp(N, 3)
L = quantile(x, prob = 1) # Censoring point
censor = ifelse(x <= L, 1, 0) # Censoring indicator

x[censor == 1] <- L

S <- 10000
lambda <- rep(NA, S)
lambda[1] <- 1

for(i in 2:S){

  # MCMC for lambda
  lambda.star = lambda[i-1] + rnorm(1,0)

  if(lambda.star < 0){
    alpha = 0
  }else{
    ratio <- exp(posterior(x, censor, lambda.star) - posterior(x, censor, lambda[i-1]))
    alpha <- min(1, ratio)
  }

  if(runif(1) < alpha){
    lambda[i] <- lambda.star
  }else{
    lambda[i] <- lambda[i - 1]
  }

}

lambda <- lambda[5000:S]
mean(lambda)

Answer: It is not the case that you should recover the prior if all your data is censored; there is still information in the data.
Consider the following situation.  The data is i.i.d Gaussian with unknown mean $\mu$ (whether or not we know the variance is irrelevant to this example.)  My prior on the mean is that $\mu \sim N(0,10)$.   My censoring point is $\gamma = 10$.  I draw 100 observations; every one is above the censoring point.  Without getting into the details of what my posterior should look like, why, given that I have observed 100 values $> 10$ and none $< 10$, and the underlying distribution is Normal, would I hold to the belief that there's a 50-50 chance that the true mean $< 0$ (which is a straightforward implication of the prior)?  For any proposed value of $\mu \leq 10$, there is only a $2^{-100}$ probability that all 100 observations would lie above $\mu$ (thanks to the symmetry of the Gaussian); clearly our posterior should be substantially different than our prior if we want to capture this information.
Comment: Thanks for that very useful answer. I think that makes sense to me. An additional question then would be is the parameter estimate under 100% censoring reliable? In the sense that will the MCMC converge to a stationary distribution if left to run long enough?
Comment: If you have proper priors on all your parameters, writing informally, almost certainly, otherwise, possibly not.  For the latter, consider the standard improper prior on $\mu \sim U(-\infty, \infty)$; if all we know is that we have 100 observations $>10$, the posterior will still be improper because there's no information to cause the right tail to start dropping off at a sufficiently great rate to be integrable.   Also note that the Gamma(0.01,0.01) distribution is no longer considered a good choice of uninformative prior for scale parameters.
Comment: I strongly suspect that you can construct counterexamples even with proper priors, but I also strongly suspect you have to work to do it, so your chances of doing so by accident are pretty low.
Comment: Thanks for the useful information. Can you recommend a useful noninformative prior for scale parameters?
Comment: http://www.stat.columbia.edu/~gelman/research/published/taumain.pdf , although not directly applicable to your problem, contains suggestions that I have found helpful in the past (early days when I still used the Gamma dist'n.)  I'd use the half-t if I were in your shoes.
Comment: Thank you so so much! Have a good weekend.
Answer: If all the data points are censored then this is just like the case where you have an observable set of binary data (i.e., all data points zero or one) and you observe all the values are zeroes (or equivalently, all the values are ones).  This is the famous "sunrise problem" examined by Laplace.  Under the sunrise problem, you can still get an inference for the probability of the outcome that has occurred in the data.
Transferring over to your model, this is the probability of a value being censored, which is a function of $\lambda$.  Consequently, you still get an inference for $\lambda$ in this case, and the posterior is not equivalent to the prior --- i.e., the data is still informative about the parameter.  Of course, the inferences is not great (compared to the case where you have non-censored data), but that is hardly surprising, since your data essentially reduces to one outcome from a binary.


Correlation between two Decks of cards?

Question: I've written a program to simulate an overhand card shuffle.
Each card is numbered, with suit going from CLUBS, DIAMONDS, HEARTS, SPADES and the rank from Two up to Ten then Jack, Queen, King and Ace. Thus the Two of Clubs has a Number of 1, the Three of Clubs a 2 ....Ace of Clubs is 13... Ace of Spades is 52.
One of the methods to determine how shuffled the cards are is to compare it to an unshuffled card and see if the order of cards is correlated.
That is, I might have these of cards, with the unshuffled card for comparison:

Unshuffled
Shuffled
Unshuffled number
Shuffled number

Two of Clubs
Three of Clubs
1
2

Three of Clubs
Two of Clubs
2
1

Four of Clubs
Five of Clubs
3
4

Five of Clubs
Four of Clubs
4
3

The correlation by the Pearson method would be: 0.6
With a large set of cards (all 52) you might see patterns emerge. My Hypothesis is that after more shuffling you'll get less correlation.
However, there are lots of ways to measure correlation.
I've tried my hand at Pearson's correlation but I'm not sure if this is the right correlation to use in this situation.
Is this a suitable correlation measure? Is there a more suited measure?
Bonus Points I sometimes see this sort of data in my results:

Clearly there is some correlation, because there are multiple separate lines of with different gradients, but I don't know how you measure the separate 'trendlines'?
Comment: @gung the wikipedia page you linked to has entries for both "riffle shuffle" and the "overhand shuffle" the OP was talking about. Its good to read the links you link to :)
Comment: @Pureferret In that case, I will rephrase. You *should be* computing rank correlation measures.
Comment: To help us better understand what you want, perhaps you could be a little more precise about what you mean by "the order of cards is correlated."
Comment: @whuber, I think the OP means the position of a given card before shuffling & after. Eg, the ace of hearts might have been 3rd from the top beforehand & 8th afterwards.
Comment: I wonder if by "overhand shuffle", you mean what [Wikipedia calls](http://en.wikipedia.org/wiki/Shuffling) a "riffle shuffle"?
Comment: @gung, Nope. Riffling results in tightly interleaved cards, overhand is a different technique. I'll try to add details tomorrow.
Comment: @gung http://cardshuffles.com/overhand-shuffle.htm
Comment: Shouldn't you be computing [rank correlation measures](http://en.wikipedia.org/wiki/Correlation_and_dependence#Rank_correlation_coefficients)?
Comment: @fgnu: I honestly don't know.
Comment: Could you please specify what you mean by "how shuffled the cards are"? What are the characteristics you expect from "well/bad shuffled cards"?
Comment: @QuantIbex they should not resemble the previous arrangement of cards before any shuffling took place.
Answer: You can measure the relative level of correlation (or more precisely, the increasing level of randomness) using the Shannon entropy of the difference in face value between all pairs of adjacent cards.
Here's how to compute it, for a randomly shuffled deck of 52 cards. You start by looping once through the entire deck, and building a sort of histogram. For each card position $i=1,2,...,52$, calculate the difference in face value $\Delta F_{i} = F_{i+1} - F_{i}$.  To make this more concrete, let's say that the card in the $(i+1)$th position is the king of spades, and the card in the $i$th position is the four of clubs.  Then we have $F_{i+1} = 51$ and $F_{i} = 3$ and $\Delta F_{i} = 51-3 = 48$.  When you get to $i=52$, it's a special case; you loop around back to the beginning of the deck again and take $\Delta F_{52} = F_{1} - F_{52}$.  If you end up with negative numbers for any of the $\Delta F$'s, add 52 to bring the face value difference back into the range 1-52.
You will end up with a set of face value differences for 52 pairs of adjacent cards, each one falling into an allowed range from 1-52; count the relative frequency of these using a histogram (i.e., a one-dimensional array) with 52 elements.  The histogram records a sort of "observed probability distribution" for the deck; you can normalize this distribution by dividing the counts in each bin by 52.  You will thus end up with a series of variables $p_{1}, p_{2}, ... p_{52}$ where each one may take on a discrete range of possible values: {0, 1/52, 2/52, 3/52, etc.} depending upon how many pairwise face value differences ended up randomly in a particular bin of the histogram.
Once you have the histogram, you can calculate the Shannon entropy for a particular shuffle iteration as $$E = \sum_{k=1}^{52} -p_{k} ln(p_{k})$$  I have written a small simulation in R to demonstrate the result.  The first plot shows how the entropy evolves over the course of 20 shuffle iterations.  A value of 0 is associated with a perfectly ordered deck; larger values signify a deck which is progressively more disordered or decorrelated.  The second plot shows a series of 20 facets, each containing a plot similar to the one that was originally included with the question, showing shuffled card order vs. initial card order.  The 20 facets in the 2nd plot are the same as the 20 iterations in the first plot, and they are also color coded the same as well, so that you can get a visual feel for what level of Shannon entropy corresponds to how much randomness in the sort order.  The simulation code that generated the plots is appended at the end.

<code>library(ggplot2)

# Number of cards
ncard <- 52
# Number of shuffles to plot
nshuffle <- 20
# Parameter between 0 and 1 to control randomness of the shuffle
# Setting this closer to 1 makes the initial correlations fade away
# more slowly, setting it closer to 0 makes them fade away faster
mixprob <- 0.985
# Make data frame to keep track of progress
shuffleorder <- NULL
startorder <- NULL
iteration <- NULL
shuffletracker <- data.frame(shuffleorder, startorder, iteration)

# Initialize cards in sequential order
startorder <- seq(1,ncard)
shuffleorder <- startorder

entropy <- rep(0, nshuffle)
# Loop over each new shuffle
for (ii in 1:nshuffle) {
    # Append previous results to data frame
    iteration <- rep(ii, ncard)
    shuffletracker <- rbind(shuffletracker, data.frame(shuffleorder,
                            startorder, iteration))
    # Calculate pairwise value difference histogram
    freq <- rep(0, ncard)
    for (ij in 1:ncard) {
        if (ij == 1) {
            idx <- shuffleorder[1] - shuffleorder[ncard]
        } else {
            idx <- shuffleorder[ij] - shuffleorder[ij-1]
        }
        # Impose periodic boundary condition
        if (idx < 1) {
            idx <- idx + ncard
        }
        freq[idx] <- freq[idx] + 1
    }
    # Sum over frequency histogram to compute entropy
    for (ij in 1:ncard) {
        if (freq[ij] == 0) {
            x <- 0
        } else {
            p <- freq[ij] / ncard
            x <- -p * log(p, base=exp(1))
        }
        entropy[ii] <- entropy[ii] + x
    }
    # Shuffle the cards to prepare for the next iteration
    lefthand <- shuffleorder[floor((ncard/2)+1):ncard]
    righthand <- shuffleorder[1:floor(ncard/2)]
    ij <- 0
    ik <- 0
    while ((ij+ik) < ncard) {
        if ((runif(1) < mixprob) & (ij < length(lefthand))) {
            ij <- ij + 1
            shuffleorder[ij+ik] <- lefthand[ij]
        }
        if ((runif(1) < mixprob) & (ik < length(righthand))) {
            ik <- ik + 1
            shuffleorder[ij+ik] <- righthand[ik]
        }
    }
}
# Plot entropy vs. shuffle iteration
iteration <- seq(1, nshuffle)
output <- data.frame(iteration, entropy)
print(qplot(iteration, entropy, data=output, xlab="Shuffle Iteration",
            ylab="Information Entropy", geom=c("point", "line"),
            color=iteration) + scale_color_gradient(low="#ffb000",
            high="red"))

# Plot gradually de-correlating sort order
dev.new()
print(qplot(startorder, shuffleorder, data=shuffletracker, color=iteration,
            xlab="Start Order", ylab="Shuffle Order") + facet_wrap(~ iteration,
            ncol=4) + scale_color_gradient(low="#ffb000", high="red"))
</code>
Comment: I know it's been a long time since this answer was posted but I'm interested in calculating the maximum entropy for a given deck. Is there a formula to calculate the maximum entropy a 52 card deck or a 100 card deck can take? Or how can one find an upper bound for the sum?
Answer: I know that this post is almost 4 years old, but I am a hobbyist cryptanalyst, and have been studying playing card ciphers. As a result, I have come back to this post over and over to explain deck shuffling as a source of entropy for randomly keying the deck. Finally, I decided to verify the answer by stachyra by shuffling the deck by hand, and estimating the deck entropy after each shuffle.
TL;DR, to maximize deck entropy:

For only riffle shuffling, you need 11-12 shuffles.
For cutting the deck first then riffle shuffling, you only need 6-7 cut-and-shuffles.

First off, everything that stachyra mentioned for calculating Shannon entropy is correct. It can be boiled down this way:

Numerically assign a unique value to each of the 52 cards in the deck.
Shuffle the deck.
For n=0 to n=51, record each value of (n - (n+1) mod 52) mod 52
Count the number of occurrences of 0, 1, 2, ..., 49, 50, 51
Normalize those records by dividing each by 52
For i=1 to i=52, calculate -p_i * log(p_i)/log(2)
Sum the values

Where stachyra makes one subtle assumption, is that implementing a human shuffle in a computer program is going to come with some baggage. With paper-based playing cards, as they get used, oil from your hands transfers to the cards. Over an extended time, due to oil buildup, cards will begin sticking together, and this will end up in your shuffle. The more heavily used the deck, the more likely two or more adjacent cards will stick together, and the more frequently it will happen.
Further, supposed the two of clubs and jack of hearts stick together. They may end up stuck together for the duration of your shuffling, never separating. This could be imitated in a computer program, but this isn't the case with stachyra's R routine.
Also, stachyra has a manipulation variable "mixprob". Without fully understanding this variable, it is a little bit of a black box. You could incorrectly set it, affecting the results. So, I wanted to make sure his intuition was correct. So I verified it by hand.
I shuffled the deck 20 times by hand, in two different instances (40 total shuffles). In the first instance, I just riffle shuffled, keeping the right and left cuts close to even. In the second instance, I cut the deck deliberately away from the middle of the deck (1/3, 2/5, 1/4, etc.) before doing an even cut for the riffle shuffle. My gut feeling in the second instance was that by cutting the deck before shuffling, and staying away from the middle, I could introduce diffusion into the deck more quickly than stock riffle shuffling.
Here are the results. First, straight riffle shuffling:

And here is cutting the deck combined with riffle shuffling:

It seems that entropy is maximized in about 1/2 the time of the claim by stachyra. Further, my intuition was correct that cutting the deck deliberately away from the middle first, before riffle shuffling did introduce more diffusion into the deck. However, after about 5 shuffles, it didn't really matter much anymore. You can see that after about 6-7 shuffles, entropy is maximized, versus the 10-12 as the claim made my stachyra. Could it be possible that 7 shuffles is sufficient, or am I being blinded?
You can see my data at Google Sheets. It is possible that I recorded a playing card or two incorrectly, so I can't guarantee 100% accuracy with the data.
It's important that your findings are also independently verified. Brad Mann, from the Department of Mathematics at Harvard University, studied how many times it would take to shuffle a deck of cards before the predictability of any card in the deck is completely unpredictable (Shannon entropy is maximized). His results can be found in this 33-page PDF.
What's interesting with his findings, is that he is actually independently verifying a 1990 New York Times article by Persi Diaconis, who claims that 7 shuffles are sufficient for thoroughly mixing a deck of playing cards via the riffle shuffle.
Brad Mann walks through a few different mathematical models in shuffling, including Markov chains, and comes to the following conclusion:

This is approximately 11.7 for n=52, which means that, according to
  this viewpoint, we expect on average 11 or 12 shuffles to be necessary
  for randomizing a real deck of cards. Note that this is substantially larger than 7.

Brad Mann just independently verified stachyra's result, and not mine. So, I looked closer at my data, and I discovered why 7 shuffles is not sufficient. First off, the theoretical maximum Shannon entropy in bits for any card in the deck is log(52)/log(2) ~= 5.7 bits. But my data never really breaks much above 5 bits. Curious, I created an array of 52 elements in Python, shuffled that array:
<code>>>> import random
>>> r = random.SystemRandom()
>>> d = [x for x in xrange(1,52)]
>>> r.shuffle(d)
>>> print d
[20, 51, 42, 44, 16, 5, 18, 27, 8, 24, 23, 13, 6, 22, 19, 45, 40, 30, 10, 15, 25, 37, 52, 34, 12, 46, 48, 3, 26, 4, 1, 38, 32, 14, 43, 7, 31, 50, 47, 41, 29, 36, 39, 49, 28, 21, 2, 33, 35, 9, 17, 11]
</code>
Calculating its entropy-per-card yields about 4.8 bits. Doing this a dozen times or so shows similar results varying between 5.2 bits and 4.6 bits, with 4.8 to 4.9 as the average. So looking at the raw entropy value of my data isn't enough, otherwise I could call it good at 5 shuffles.
When I look closer at my data, I noticed the number of "zero buckets". These are buckets where there is no data for deltas between card faces for that number. For example, when subtracting the value of two adjacent cards, there is no "15" result after all 52 deltas have been calculated.
I see that it eventually settles around 17-18 "zero buckets" around 11-12 shuffles. Sure enough, my shuffled deck via Python averages 17-18 "zero buckets", with a high of 21 and a low of 14. Why 17-18 is the settled result, I can't explain ... yet. But, it appears that I want both ~4.8 bits of entropy AND 17 "zero buckets".
With my stock riffle shuffling, that's 11-12 shuffles. With my cut-and-shuffle, that's 6-7. So, when it comes to games, I would recommend cut-and-shuffles. Not only does this guarantee that the top and bottom cards are getting mixed into the deck on each shuffle, it's also just plain quicker than 11-12 shuffles. I don't know about you, but when I'm playing card games with my family and friends, they're not patient enough for me to perform 12 riffle shuffles.

Correlation between two Decks of cards?


DIV redimensionar após ser clicada

Question: olá
tenho uma pagina com varias DIV.  que são varias Notícias diferentes.
essas DIV tão num tamanho de 300px por 250px,
 gostaria de saber como faço para ao clicar em alguma dessas div ela redimencionar na tela e ficar maior ,fazendo com que apareça todo o conteúdo que ta dentro dela
Comment: Mas a div aumentada vai ficar por cima das outras, vai empurrar as outras, como é?
Comment: Bom tb colocar o HTML do que já fez porque dependendo de como está fazendo por influenciar no resultado.
Comment: @Sam  na verdade quando essa DIV abrir não vai aparecer as outras.
Comment: Posta o código na pergunta.
Comment: Então vc quer que ao clicar abra um popup com a noticia? Tipo clica na div e abre uma "janela" por cima da pagina mostrando apenas a noticia clicada?
Comment: Modal Boostrap ficaria bom no seu caso
Comment: @hugocsl  e isso  ai
Answer: Veja se o código abaixo te ajuda.
Ele não esta utilizando jQuery, bootstrap que ferramentas que vale apena você dar uma olhada

function onClick(){
    var self=this, // ; elemento clicada
        isOpen = self.classList.contains('open');

    Array.from(document.getElementsByClassName('noticia'))
       .forEach(function(el,idx){
            if( isOpen )
            {
                el.className='noticia'; // ; mostra a list noticia
            }
            else if( self != el )
            {
             el.className += ' close'; // ; esconde a lista
            }
            else
            {
             el.className += ' open'; // ; espande a noticia clicada
            }
       });
    }

/// Adicionar evento de onClick no elementos com a class noticia
Array.from(document.getElementsByClassName('noticia')).forEach(function(el,idx){
    el.onclick = onClick;
});
.noticias-container{
    text-align:center;
}
.noticias-container .noticia{
    display: inline-block;
    border:1px solid red;
    width: calc( 32.222% - 5px ); /* Coloquei tamanho flexivel para exibir aqui */
    margin: 3px 3px;
    height: 250px;
}
.noticias-container .noticia.close{
    display:none;
}

.noticias-container .noticia.open{
    width: 600px;
    height: 550px;
}

ELEMENTO 1

ELEMENTO 2

ELEMENTO 3

ELEMENTO 4

ELEMENTO 5

ELEMENTO 6

DIV redimensionar após ser clicada


How can I prevent Postfix to send messages waiting in the incoming / active queue for more than 1 day?

Question: I have a first line / graveyard postfix servers structure, the first line servers send messages to the graveyard when it received a soft fail.
But sometimes I guess the queues are too busy and messages are locked in the incoming queue and cannot be sent but days after. I prefer to discard mails than sending very old ones.
Setting the maximal_queue_lifetime won't work because it needs to send the message at least once, and if the message is too old, I don't want it be be sent not even once.
Answer:
But sometimes I guess the queues are too busy and messages are locked
  in the incoming queue and cannot be sent but days after

As documented, postfix picks one message in turn from the deferred queue and the incoming queue, as long as there is room in the active queue.
So if you see messages remaining in the incoming queue for days, this means your mail server cannot handle the traffic (the default active queue limit is 20000 messages).
This can be network-related or a server resource issue.


Can't import from XML using Feeds importer and XPath parser

Question: I'm trying to import book objects from the XML file you can see XML file Here. I will tell step by step what I did to achieve this, so it would be easier to help me.

Created a Book content type with all the corresponding fields from the XML file
Installed Feeds importer and XPath parser modules with their dependant modules
Created a Feed importer called bookImport
In the Basic settings I attached it to Book content type
The fetcher is a default one - HTTP
For parser I chose XPath XML parser
Processor is Node. As I want to create new Nodes for every single book
In the Node processor settings I chose Book bundle and everything else is left as it was by default.
In the mapping I mapped all the xpathparser:* source values to target ones from Book content type.

When trying to import, in the XPath Parser Settings for context I put book and leave everything (all the other book fields empty), I turn on my error messages and after clicking Import I get these two lines:
context :
There are no new nodes.

I've been trying to solve this for 2 days now without any luck, any help would be highly appreciated and I hope my post is informative enough, if there is something missing, please let me know and I will update!
Answer: Solved this on my own, the problem was that in context I had to write //book, instead of book. Stupid mistake which caused me headache for 2 days. I hope this will be helpful for people who will face the same problem.


Normality test of non "normally distributed" values prior to Anova test

Question: I am trying to perform a normality test to multiple continuous values before doing an anova test. The p-value I am getting for the data does not make much sense and I want to make sure I am not missing something.

My data consists of 40k rows, I cannot use scipy.stats.shapiro so I am using kstest
When doing a shapiro test I believe the W value has to be close to 1. Does the same apply to D statistic value?
most p-values are 0.0 which makes me think I am missing something.
What values from the kstest will render the anova results valid?
Should I be using Anderson-Darling test given that the data is not normally distributed? if so would it still count as a normality test?
I tried converting some columns from lognorm to norm by doing df['income'] = df['income'].apply(lambda x: math.log10(x)) that seem to result in p-values that approach zero. but I am not sure if that's the right method. if it is, should the anova analyze the log(income) as well or it does not matter?

here is the code I used to do the test:
from scipy.stats import norm, kstest

for var in numerical_features:
    loc, scale = norm.fit(df[var].to_numpy())
    n = norm(loc=loc, scale=scale)
    d, p = kstest(df[var].to_numpy(), cdf=n.cdf)
    print("{0} {1} {2}".format(var, d, p))

Here is the data itself:
age:
D=0.054 p=9.488e-84

income:
D=0.142 p=0.0

vehicles owned:
D=0.409 p=0.0

years of experience:
D=0.175 p=0.0
Answer: Well you hardly need a test to tell you that your data are not normally distributed. If you take a quick look at your histograms you can directly see that. Moreover if you have a large sample size as you do you will almost by definition reject the $H_0$ of normality even if your data is very close to being normally distributed (see e.g. here on CV). Thus your small p-values are simply a result of 1) the fact that your data is indeed not normal distributed and 2) you have a very large sample size.
More to the point however is if you need normality of your data (see e.g. this related question).  ANOVA is pretty robust, so violation of normality is not always a problem, especially for large sample sizes where we can rely on the central limit theorem (see among others again here and here).
If you do decide to transform your data to have it conform to the normality, then indeed you need to do the ANOVA on the transformed data. Keep in mind though, that comparing the means of income is not exactly the same as comparing the log of income, so this will affect your results (see here)


Test for markov-property in a time-series

Question: Given an (observed) time-series $X_t$ with $X_t\in\{1,...,n\}$, is there a statistical test for testing the null-hypothesis that $P(X_t|X_{t-1},X_{t-2},...,X_1)=P(X_t|X_{t-1})$ (i.e. the markov-property)?
Comment: I think the paper, "[Testing for the Markov Property in Time Series](http://www.wise.xmu.edu.cn/Activity/XMU-HUB2008/paper/HONG.pdf)" contains useful insight and literature review.
Comment: if you want to test the Markovian assumption in isolation, you will have to do something like the paper @Pardis linked. If you want to check this assumption in the context of some kind of model fitting my inclination would be to do something informal like: write down the joint likelihood under the Markovian assumption and fit the model. Next, write down the joint likelihood without the Markovian assumption and re-fit the model. If the estimates are about the same, then nothing is lost by using the Markovian assumption. (I'm making this a comment since it doesn't explicitly answer the question)
Comment: Great reference from Pardis!  Along the lines of what Macro is saying if you fit an AR(1) model to the data and it fits well then in a way that tests the Markov property because AR(1) processes are Markovian.
Comment: Yes @MichaelCherknick, but there are surely other Markovian models. The AR(1) fitting poorly doesn't tell you the model is **not** Markovian.
Comment: @Pardis, 404 on link to "Testing for the Markov Property..."
Answer: Great question!! On the top of my head, a consequence of the Markov property, is that conditionally on $X_{t-1}$, $X_t$ is independent of $X_{t-2}$, $X_{t-3}$, ... (this is used in Bayesian networkmodeling).
So you can prove the Markov property if you can prove $P(X_t, X_{t-2}, X_{t-3}, ...|X_{t-1}) = P(X_t | X_{t-1}) P(X_{t-2} X_{t-3}, ....| X_{t-1})$ for every index.
The only case that this will be (relatively easy) is if the variables are multivariate Gaussian. Otherwise it can be quite hard to to implement, especially if you observations are continuous. Still, you can use tests for independence such as $\chi^2$, or more advanced techniques based on Kullback-Leibler divergence as shows in this article for instance.
Comment: I'm afraid I don't quite understand how I would do that. Can you elaborate on how to proceed in practice? Note that I have univariate observations from a discrete set $X_t\in\{1,...,n\}$ for all $t$. Exactly which distribution has to be multivariate gaussian?


How to change size of remote ssh session in my terminal?

Question: I use the "Terminal" app. for SSH'ing to my Ubuntu VPS. The problem I have is that the width/size of the remote terminal doesn't fill my Terminal app. window, only part.
How can I make it fill the entire Terminal area?
Answer: What is you terminal application ? running on what OS ?
Perhaps what you are looking for is:
<code>eval $(resize)
</code>
Comment: yes, OS X Terminal app, but i think it depends on remote machine, cause i use few vps's and only with this one i have this issue.
Comment: Your comment helps me a lot, i found that i need xterm package, after installing it everything works fine. Thanks.


Perfmon counters for monitoring general IIS/MSSQL server activity

Question: I have a virtual server with a hosting company so CPU and RAM are limited. I need to watch load/performance over time to know when to upgrade the server.
The site runs multiple ASP.NET apps as well as PHP apps, all with MSSQL backend.
What are some vital perfmon counters to use to monitor a standard IIS/MSSQL web server: I.e. incoming IIS request volume, response times, MSSQL load, etc? A short description on why the counter matters and what to watch for (if not obvious) would also be helpful.
Answer: You should start with the basics;

Processor Information\% Processor Time
Memory\Pages/Sec
Memory: Available Bytes
Logical Disk*\Average Disk Queue Length
SQL Server:Buffer Manager\Buffer cache hit ratio

If you went with the default install of non-express SQL then it will take all the memory it can and starve IIS/Windows. On the other hand, if SQL doesn't have enough memory it will have a low Buffer cache hit ratio and you will probably see disk queueing.  Most other counters will be a side-effect of a constraint on disk/memory/processor for a single server installation.  Once you get to the point where you need to determine if it is: IIS/SQL, php vs asp.net, identify intensive WPs, etc. then the monitoring becomes more complex and you will need to do a lot of research or hire a contractor.
User performance monitor or advisor tool linked at the bottom to setup a data collection schedule to log the key performance counters. Generally a 5-15 minute sample is sufficient for overall system monitoring/analysis.  If you are trying to diagnose a specific issue you may go down to 15s or less for short periods of time.
Pages/Sec: Average of extended periods > 150 can indicate a problem, but it also depends on disk.  On a cloud server you might want to avg < 150 as the disk is likely shared or remote.  Also look at Available Bytes, if pages/sec is high and Available Bytes is high, then the counter can be misleading as there are other causes than paging that are beyond the scope of this question
% Processor Time: If it averages or you see extended periods > 75-80% you are CPU bound.
Buffer cache hit ratio: Generally should be 96-99%, lower indicates that SQL is not effectively caching and needs more memory
Average Disk Queue Length: Log for each individual disk,  > 2 indicate that the disk is saturated.  You can also log PhysicalDisk\% Idle Time and if it falls below 20% it indicates disk saturation.
You can also use Microsoft's performance advisor to jumpstart your monitoring and have more thorough monitoring based on Microsoft recommendations;
Microsoft Performance Advisor
Answer: Your question is difficult to answer objectively. However, I can tell you that if your DB is getting regular action and the DBs aren't small, MSSQL will grab as much memory as it can figure out how to use. (Though SQL Server Express version won't grab more than 1 GB.) So don't be suprised if SQL Server is using all of your available memory. And if it is, that doesn't necessarily mean you are in need of a memory upgrade, it could still perform fine under this scenario. My point is, memory usage may not be as meaningful of a metric as say, cpu usage or website latency.


Is this an overflow?

Question: Perform the indicated operation of the numbers, which are in two's-complement representation.  Check your results by converting all numbers to decimal.  Indicate overflow and carryout.
(a) 10010111-00111100
1001 0111 (151)
- 0011 1100 (60) ---->+ 1100 0100 (-60)

I used two's complement to change (60) to (-60).

10010111 (151) + 11000100 (-60) = 1 0101 1011

The answer is in 9 bits, while the rest are in 8 bits.
The range is -128 to 127 : -(2^(8-1)) to (2^(8-1)-1)

Is this a carry out? Is this an overflow? I know that it's a carry out because we are adding a bit into the sign. But is this considered an overflow? Discarding the 1 on the left-most side, we convert the answer to (91) in decimal, which matches with (151) - (60) = (91).

I at first considered this NOT an overflow, because (91) is in range. However, I am not sure if I am supposed to take into account the carry out of 1. The 1 also confuses me because the answer is positive and not negative. Am I supposed to discard the 1? Is this signed, meaning that the answer is negative? But the answer is supposed to be positive?? I'm just getting a little more confused the more I think about it.
(I know I could have just subtracted normally, but I wanted to do it this way)
Comment: In two's complement it's really easy to tell whether overflow happens during addition: if the carry-out is different from the carry-in to the sign bit, then overflow has occurred. Otherwise it hasn't.
Answer: Since we have 2's complement and 8 bits, the leftmost bit is the sign bit. This means 1001 0111 is NOT 151, but rather -105.
To continue the math:
1001 0111 (-105)
- 0011 1100 (60) ---->+ 1100 0100 (-60)
Then as you did before,
10010111 (-105)
+ 11000100 (-60)
= 1 0101 1011
Since we are working in 8bits,0101 1011 is 91 in decimal.
Note that overflow applies when we work with signed numbers and occurs when we add to the sign bit, while carry out applies when we work with unsigned number and occurs when the bits "fall" off the left.
Therefore, in this case, we do have overflow since the sign bit has been altered.
In general, the term carry out doesn't really concern signed number, but in the interest of your problem, you could say there is carry out since you do have one bit that couldn't be represented on the left.
Comment: But the answer's decimal conversion doesn't match. So (-105) + (-60) = (-91)?
Comment: Yes and No. I am claiming if you do the 2's complement computation,
 (-105) + (-60) = 91 (note the positive). Clearly it doesn't match real arithmetic, and therefore we have an overflow.

Is this an overflow?


PHP and MySQL find all children and grandchildren of parents in database

Question: I'm trying to achieve something like the following:
Get all child, grandchild etc nodes under parent using php with mysql query results
But I'm looking to do it without putting everything in an array and without nesting if statments - just go through the first level one by one and:
 - print the item
 - if the item has a child find it and print it
 - if the child has a child find it and print it
 - if the child has no child move up one level and look for the next child or item and it's childs
 - and so on and so on ... preferably into a UL LI set of lists and sublists
My database and the output also needs to be ordered so it looks more like this:
id     name       parent_id     orderby
1  Electronics          0         0
2  Televisions          1         10
3  Portable Electronics 1         20
4  Tube                 2         20
5  LCD                  2         10
6  Plasma               2         30
7  Mp3 Players          3         30
8  CD Players           3         20
9  2 Way Radios         3         10
10 Flash                7         10

I can do it using if statements but to do this I need to know how many levels the deepest child is and if there are, say, 7 levels that gets messy. I've seen it (at the link above) put into a multidimensional array but then it seems that you need nested for each statements to get the data out again which is pretty much the same as the nested if solution.
I'm pretty sure the answer is under my nose but haven't been able to find it here or elsewhere....
WordPress seems to have a way of doing it but I haven't been able to uncover the code there.
Any help would be much appreciated!
Comment: I'm not sure I understand what you are expecting your output to look like when finished.  Can you give an example of that as well?
Comment: What is `orderBy` for?
Comment: And you can achieve any level of nested solution using recursion.
Comment: Right now I want to create a page which lists all the pages on a website (like File Manager in windows). I need the orderby because the order in which the pages (and their children) are shown is important. Is that a bit clearer?
Answer: Use the following code to get the data :
function getChildren($parent) {
    $query = "SELECT * FROM tableName WHERE parent_id = $parent";
    $result = mysql_query($query);
    $children = array();
    $i = 0;
    while($row = mysql_fetch_assoc($result)) {
        $children[$i] = array();
        $children[$i]['name'] = $row['name'];
        $children[$i]['children'] = getChildren($row['id']);
    $i++;
    }
return $children;
}

Call this function using
$finalResult = getChildren('*');

EDIT by James
Just to finish this answer, to print out the results into the list:
<?php
    function printList($array = null) {
        if (count($array)) {
            echo "
";

            foreach ($array as $item) {
                echo "
";
                echo $item['name'];
                if (count($item['children'])) {
                    printList($item['children']);
                }
                echo "
";
            }

            echo "
";
        }
    }

    printList($finalResult);
?>

Comment: Maybe I don't follow. But this just gets the immediate children of a parent? What about if the children have children and so on up to any number of levels?
Comment: @Doodled That is why they call `getChildren` again within the `while` loop.
Comment: Also, maybe you should be calling `$finalResult = getChildren('0');` instead of `*` to only get the first level from the initial call.
Comment: That's beautiful!!! And thanks James for adding the output function - first time I've seen a multi-dimensional array output in such a clean way.


gcm server response value

Question: Step1
my server send message to app device
response is success
<code>{
    "multicast_id":7728707543791363238,
    "success":4,
    "failure":0,
    "canonical_ids":0,
    "results":[
        {"message_id":"0:1470905757075944%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470905757075946%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470905757075383%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470905757075948%4d9f3641f9fd7ecd"} ]
}
</code>
Step2
delete add form device(android)
Step3
my server send message to same device
response is success too
<code>{
    "multicast_id":5618205143815724662,
    "success":4,
    "failure":0,
    "canonical_ids":0,
    "results":[
        {"message_id":"0:1470906000290483%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470906000289946%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470906000291651%4d9f3641f9fd7ecd"},
        {"message_id":"0:1470906000289948%4d9f3641f9fd7ecd"} ]
}
</code>
i think success->0 , failure->4 and
<code>"result":[
    {"error":"NotRegistered"}
    {"error":"NotRegistered"}
    {"error":"NotRegistered"}
    {"error":"NotRegistered"} ]
</code>
because since there is no App on device.
Deive Receive success.
Server Language : php
Device : android
I reading Table 5. Downstream HTTP message response body (JSON) in document (Ref : https://developers.google.com/cloud-messaging/http-server-ref), but i don't know problem...
Why did I receive the wrong response?
Can I get a "NotRegistered"?
I am careful where I look?
Comment: Have you tried following the [Try Cloud Messaging for Android](https://developers.google.com/cloud-messaging/android/start)? This will help you understand the flow for Google Cloud Messaging: Register to enable GCM, Send and receive downstream messages, Send and receive upstream messages. For each device, first get a registration token for Google Cloud Messaging from the InstanceID API. This token is used to identify the instance of the quickstart running on your device.
Answer: What you are seeing is working as intended. When you uninstall the app the token will eventually be invalidated however for some time invalid tokens will still be usable. If the app is not installed then even if the device receives the message it will never be passed to the app. Then eventually on the server side it will start to fail.
Comment: I knew immediately that judgment.
Thanks for your help.

gcm server response value


iOS Google map SDK: Setting maxZoom level issue

Question: I notice when I set the max Zoom level of example to 19, the zoom go up to 20. I don't know why.
It's always 1 more zoom level than the one I set.
GMSCameraPosition *camera = [GMSCameraPosition cameraWithLatitude:23.589571946369546
                                                        longitude:58.14204730042655
                                                             zoom:16];

self.mapView_.camera=camera;
self.mapView_.myLocationEnabled = YES;

self.mapView_.mapType = kGMSTypeHybrid;
self.mapView_.settings.compassButton = YES;
[self.mapView_ setMinZoom:5 maxZoom:19];

// Creates a marker in the center of the map.
GMSMarker *marker = [[GMSMarker alloc] init];
marker.position = CLLocationCoordinate2DMake(23.168520, 58.008163);

marker.map = self.mapView_;
// ------ add layer
// Implement GMSTileURLConstructor
// Returns a Tile based on the x,y,zoom coordinates, and the requested floor
GMSTileURLConstructor urls = ^(NSUInteger x, NSUInteger y, NSUInteger zoom) {
    NSString *url = [NSString stringWithFormat:@"http://www.example.com/%tu/%tu/%tu.png", zoom, x, y];
    NSLog(@"url=%@",url);
    return [NSURL URLWithString:url];
};

Answer: How do you find out the maxZoom is 20? Which version of iOS Maps SDK you use?
I tried the sample hello map application from Google Maps' Github page with iOS Maps SDK version 1.9.1 and put the below code in the videDidLoad() method. It prints the correct max zoom which is 19.
GMSCameraPosition *camera = [GMSCameraPosition cameraWithLatitude:23.589571946369546
                                                            longitude:58.14204730042655
                                                                 zoom:16];

    mapView_.camera=camera;
    mapView_.myLocationEnabled = YES;

    mapView_.mapType = kGMSTypeHybrid;
    mapView_.settings.compassButton = YES;

    // Create the GMSMapView with the camera position.
    mapView_ = [GMSMapView mapWithFrame:CGRectZero camera:camera];

    [mapView_ setMinZoom:5 maxZoom:19];

    NSLog(@"max zoom: %f", mapView_.maxZoom);

Comment: Thanks @ztan ... let me check it again. Maybe I did a mistake.
Comment: I posted the same issue again on think link.. Please check ... http://stackoverflow.com/questions/27606375/ios-google-map-sdk-setting-maxzoom-level-issue-when-using-tiles