Frequentist Justification for using Student's t-Distribution to Construct Confidence Intervals

Question: In my most recent statistics class, the general1 frequentist approach to constructing confidence intervals2 for an estimator (of any kind) was described as follows:

Specify the estimator and the data generating process (i.e., the likelihood $\mathcal{L}(x; \theta)$).
Apply the estimator to observed data to obtain the estimate.
Calculate the distribution of the estimator, assuming the estimate obtained in Step 2 is the true value of the estimand, and output the appropriate cutoffs to give a (central) $(1 - \alpha)$–confidence-interval.

For instance, in the case of MLE estimators, one would proceed as follows:

Construct the likelihood function $\mathcal{L}(x; \theta)$ for the data generating process.
Given observed data $X$, find the maximum likelihood estimate $\theta_{\text{MLE}} = \arg\min_{\theta} \mathcal{L}(X; \theta)$.
Calculate the distribution of the estimator $\min_{\theta} \mathcal{L}(\tilde X; \theta)$ assuming $\tilde X$ has the distribution implied by $\theta_{\text{MLE}}$—e.g., the density of $\tilde X$ is given by $\mathcal{L}(x; \theta_{\text{MLE}})$—and then find $b_{\text{lower}}$ and $b_{\text{upper}}$ such that $\Pr(\arg\min_{\theta} \mathcal{L}(\tilde X; \theta) < b_{\text{lower}}) = \Pr(\arg\min_{\theta} \mathcal{L}(\tilde X; \theta) > b_{\text{upper}}) = \tfrac \alpha 2$ and output the interval $[b_{\text{lower}}, \, b_{\text{upper}}]$.

This process makes a lot of sense to me as a "theory" of frequentist CIs, but the most basic form of frequentist CI—viz., a CI for a sample mean—doesn't fit with this procedure.
In particular, if we assume that $X = (X_1, \ldots, X_n)$ represents $n$ i.i.d. draws from a normal $\mathcal{N}(\mu, \sigma^2)$, then we have that
\begin{align}
  \mu_{\text{MLE}}    &= \tfrac 1 n \sum_{i = 1}^n X_i \\
  \sigma_{\text{MLE}} &= \sqrt{\tfrac 1 n \sum_{i = 1}^n (X_i - \mu_{\text{MLE}})^2}
\end{align}
Therefore, Step 3 suggests that we should look at the distribution of $\sum_{i = 1}^n Y_i$ where $Y_i \sim \mathcal{N}(\mu_{\text{MLE}}, \sigma_{\text{MLE}}^2)$; the distribution of this sample mean is, of course, $\mathcal{N}(\mu_{\text{MLE}}, \tfrac {\sigma_{\text{MLE}}^2} {n})$. However, the standard advice is to construct the confidence interval as if the sample mean were drawn from the $t$-distribution with $n-1$ degrees of freedom (appropriately scaled and recentered). This gives a slightly wider confidence interval which (based on a bunch of simulation I did and a century of received statistical wisdom) generally comes closer to nominal coverage.
Is there any way to make sense of this standard advice in light of the "general theory of frequentist confidence intervals" laid out above? Is this "general theory of frequentist confidence intervals" not actually the general theory at all?

1 I realize that this is not actually the most general formulation mathematically. I just mean that it was the "most general" in the sense of, "This is how we think about constructing confidence intervals in a frequentist setting."
2 "Frequentist confidence intervals" is, perhaps, tautological if you distinguish between "confidence intervals" and "credible intervals"—I just mean error bars that are not Bayesian (or Fiducial or ...).
Comment: You are overlooking the fact that $\sigma^2_{MLE}$ is an estimate too; if it were actually known, you would be correct, and the Normal distribution as you suggest would be appropriate.  The distribution of $(\mu_{MLE}-\mu)/\sigma_{MLE}$ on the other hand is $t_{n-1}$.
Comment: I think I'm maybe still missing something. Assuming, for the moment, that the data are drawn from a normal distribution, the sample mean and sample variance are independent. Therefore, when we get to Step 3, marginalizing over $\sigma_{\text{MLE}}^2$, we're still left with a normal distribution for $\mu_{\text{MLE}}$. So it still seems like the standard advice for constructing CIs for sample means and the standard story for constructing frequentist CIs laid out above aren't compatible.
Comment: You haven't marginalized over $\sigma^2_{MLE}$ in step 3.  Your MLE is two-dimensional, and the $t$ statistic is based on both elements of it.   You also seem unclear about what a likelihood function is - it is not the case that "the density of $\tilde X$ is given by $\mathcal{L}(x; \theta_{\text{MLE}})$", for example.  The density of $\tilde X$ is $N(\mu, \sigma^2/n)$ - but you don't know $\sigma^2/n$, hence the $t$ distribution when you substitute the (random) estimate $s^2$ for $\sigma^2$.
Comment: Thanks for continuing to help me work through this! Unfortunately, I'm still fairly confused. The likelihood function, after fixing a value of the parameter (i.e., $\theta_{\text{MLE}}$ here) _is_ a density over possible observed data $\tilde X$ ([Wikipedia](https://en.wikipedia.org/wiki/Likelihood_function#Continuous_probability_distribution)). My choice of notation probably wasn't optimal—$\tilde X$, in the example we're discussing, is meant to denote not a single draw from a univariate normal but a vector of $n$ i.i.d. draws from $\mathcal{N}(\mu_{\text{MLE}}, \sigma_{\text{MLE}})$, ...
Comment: ... where $\mu_{\text{MLE}} = \tfrac 1 n \sum_{i = 1}^n X_i$ and $\sigma_{\text{MLE}} = \sqrt{\tfrac 1 n \sum_{i = 1}^n (X_i - \mu_{\text{MLE}})^2}$, where $X = (X_1, \ldots, X_n)$ are the observed data. It's definitely true, as you point out, that the joint distribution of $(\tfrac 1 n \sum_{i = 1}^n \tilde X_i, \sqrt{\tfrac 1 n \sum_{i = 1}^n (\tilde X_i - \mu_{\text{MLE}})^2})$—where $\tilde X = (\tilde X_1, \ldots, \tilde X_n)$ denotes a vector of $n$ i.i.d. $\mathcal{N}(\mu_{\text{\MLE}}, \sigma_{\text{MLE}})$ variables—is two dimensional. ...
Comment: ... But it's also true that $\tfrac 1 n \sum_{i = 1}^n \tilde X_i$ and $\sqrt{\tfrac 1 n \sum_{i = 1}^n (\tilde X_i - \mu_{\text{MLE}})^2}$ are independent by [Basu's Theorem](https://en.wikipedia.org/wiki/Basu%27s_theorem), and so, by independence, we get that the marginal distribution of $\tfrac 1 n \sum_{i = 1}^n \tilde X_i$ is exactly $\mathcal{N}(\mu_{\text{MLE}}, \tfrac {\sigma_{\text{MLE}}^2} n)$. Then, it follows that (according to Step 3, above) we will calculate $b_{\text{lower}}$ and $b_{\text{upper}}$ according to $\mathcal{N}(\mu_{\text{MLE}}, \tfrac {\sigma_{\text{MLE}}^2} n)$.
Comment: To be clear, my confusion is not why the $t$-statistic $[x - \mu_{\text{MLE}}] / [\sigma_{\text{MLE}} / \sqrt{n}]$ has the student's $t$-distribution; rather, my question is _why, if Steps 1, 2, and 3 really represent the standard frequentist framework for constructing confidence intervals, do we proceed as if $\tfrac 1 n \sum_{i = 1}^n \tilde X_i$ has a $t$-distribution with $n-1$ degrees of freedom._ It seems that probably Steps 1, 2, and 3 _don't_ represent the frequentist approach to confidence intervals; ...
Comment: ... [this wikipedia page](https://en.wikipedia.org/wiki/Confidence_interval#Statistical_theory), for instance, gives a different (and seemingly more sensible) framework from which I understand why you would use the $t$-distribution.
Comment: 1. Your conclusion that "by independence we get that the marginal distribution of..." is incorrect.   If I divide a standard Normal variate $Y$ by another, independent, standard Normal variate $X$, I don't get a standard Normal variate just because the numerator $Y$ is a standard Normal variate, I get a Cauchy distribution.  Try it with a simulator!  Try it with the denominator being distributed $U(0,10)$ and see what happens!
Comment: I'm afraid I don't completely follow, but $\tfrac 1 n \sum_{i = 1}^n \tilde X_i$ and $\sqrt{\tfrac 1 n \sum_{i = 1}^n (\tilde X_i - [\tfrac 1 n \sum_{j = 1}^n \tilde X_j])^2}$ are independent here. You can see this from [Basu's Theorem](https://en.wikipedia.org/wiki/Basu%27s_theorem), [derive it from scratch](https://statisticaloddsandends.wordpress.com/2019/08/25/proof-that-sample-mean-is-independent-of-sample-variance-under-normality/), or [simulate it](https://gist.github.com/jgaeb/e35be387eb2af18e0045e4a42e5c5819). (That simulation also shows that the marginal distribution is correct.)
Comment: (To derive it, you do need that $\tilde X$ is a vector of i.i.d. normal draws—I don't know of a more rigorous general result, although intuitively it seems like it should be approximately true for any "reasonable" distribution.)
Comment: Yes, they are independent.  So?  It does not follow that you can treat the sample std. deviation as if it were the known true value, which it isn't.  If I have random $x,y$, then $f(x,y)$ is a function of *both* and is influenced by the randomness of *both*; I can't say $x$ and $y$ are independent, therefore, when calculating the distribution of $f$, I can ignore the randomness of $y$.
Answer: The issue, I've now realized, is that Steps 1, 2, and 3 are not actually the frequentist definition of a confidence interval. According to Wikipedia, the definition is actually a pair of functions $u$ and $v$ such that $\Pr(u(X) < \theta < v(X)) = 1 - \alpha$. Steps 1, 2, and 3 are just a general way of (approximately) constructing such $u$ and $v$. So, the point is, in this particular case,
$$
  \Pr(\mu_{\text{MLE}} - t^{n-1}_{\alpha/2} \sigma_{\text{MLE}} < \theta) = \tfrac \alpha 2
$$
while
$$
  \Pr(\mu_{\text{MLE}} - z_{\alpha/2} \sigma_{\text{MLE}} < \theta) > \tfrac \alpha 2
$$
where $t^{n-1}_{\alpha/2}$ is the $\tfrac \alpha 2$-th quantile for a $t$-distribution with $n-1$ degrees of freedom and $z_{\alpha/2}$ is the $\tfrac \alpha 2$-th quantile for a standard normal.


Why is the law of large numbers defined as the probability of a limit rather than the limit of probability?

Question: The law of large numbers states for $X_1,...,X_n$ random variables with finite expectation,
\begin{equation}
\mathbb{P}\bigg(\lim_{n\to\infty}\frac{S_n}{n}=\mu\bigg) = 1
\end{equation}
where $S_n = X_1+...+X_n$.
I'm struggling to understand the difference between this definition and
\begin{equation}
\lim_{n\to\infty} \mathbb{P}\bigg(\frac{S_n}{n}=\mu\bigg) = 1.
\end{equation}
Are they equivilant? Does one imply the other? I know in general that the probability of a limit and the limit of the probability are not equivilant but struggling to understand the reason in this case.
Comment: For a continuous distribution, the probability that the sample mean is exactly equal to the population mean is $0$.  Indeed, the probability of getting any one, exact value is $0$.
Comment: Maybe it will be helpful for you to read about [convergence in probability](https://en.wikipedia.org/wiki/Convergence_of_random_variables#Convergence_in_probability) and [almost sure convergence](https://en.wikipedia.org/wiki/Convergence_of_random_variables#Almost_sure_convergence).
Answer: Essentially you are asking why the probability measure is not continuous.  Note that given a sequence of events $A_n$,
$$\lim_{n \to \infty}\mathbb{P}(A_n)=\mathbb{P}\left(\lim_{n \to \infty }A_n\right)$$
holds if $A_n$ is monotonic, i.e. either $A_1\subseteq A_2\subseteq A_3...$ or $A_1\supseteq A_2\supseteq A_3...$
Without monotonicity, continuity need not hold.
Consider $X_1,X_2,X_3,...$ iid $Bern(1/\pi)$. SLLN tells us
$$P\left(\frac{1}{n}\sum_{i=1}^n X_i \overset{n\to \infty}{\to}1/\pi\right)=1.$$
However, since $\sum_{i=1}^n X_i\sim Bin(n,1/\pi)$ is integer-valued,
$$P\left(\frac{1}{n}\sum_{i=1}^n X_i =1/\pi\right)=P\left(\sum_{i=1}^n X_i =n/\pi\right)=0\overset{n\to \infty}{\not\to}1.$$
Comment: @homelessmathaddict My response has nothing to do with continuous distributions; rather, I was referring to continuity of the probability measure (which would allow swapping $\mathbb{P}$ with $\lim$).  The comment from lulu above already mentions how this swapping is not justified for continuous distributions, nor is it justified for discrete distributions (as my answer shows).  Likewise, why do you think we can write WLLN that way?  The events $\{|S_n/n-\mu|>\epsilon\}$ need not be monotonic in $n$.
Comment: If you move the limit inside the probability, then I believe that you get the strong law of large numbers.  So your question boils down to asking what's the difference between the two.
Comment: So the two statements are equivilant for continuous distributions, but not for discrete distributions. In that case, should we not define the WLLN as $\mathbb{P}(\lim_{n\to\infty} |S_n/n - \mu|>\epsilon) = 0$ rather than $\lim_{n\to\infty}\mathbb{P}( |S_n/n - \mu|>\epsilon) = 0$?
Comment: @Teepeemm Ok, so $\mathbb{P}(\lim_{n\to\infty}|S_n/n-\mu|>\epsilon),\forall \epsilon>0$ doesn't make sense as $\lim_{n\to\infty}|S_n/n-\mu|$ is always $0$, but I don't think you would immediately get the SLLN (though I see what you mean). I guess it would be $\mathbb{P}(\lim_{n\to\infty}|S_n/n-\mu|=\epsilon), \epsilon=0$ which is then equivilent to SLLN.


Single opperation to change the frequency of data

Question: I am looking for a mathematical operation that can help me express daily data as weekly and monthly. I mean that I have $100$ daily observations and I am trying to produce an equation of the form $$ V=C+x_{1}V_{d}+x_{2}V_{w}+x_{3}V_{m}, $$ where $C$ is a constant, $x_{i}$ are coefficients and
$$ \begin{align*}
V_{d} & = \,\, \text{daily realised volatility} \\
V_{w} & = \,\, \text{weekly realised volatility} \\
V_{m} & = \,\, \text{monthly realised volatility} \\
\end{align*}$$
However, my statistical package (EViews) allows me to enter only series of the same frequency, hence, I can't do it in the normal way. This means that I have to use the daily observations for all three terms. Does anyone know a way of changing daily data to weekly and monthly?
Comment: I think it would help if you defined the terms in your equation.

Also, see here to help you express it more clearly (it is not clear if some letters you have used are subscripts or factors in a product, for example): http://math.stackexchange.com/help/notation
Comment: If I understand you correctly, you want to know what RVd, RVw and RVm are? These are daily realized volatility-RVd (a single from my observations), weekly realized volatility RVw (the sum of 5 RVd divided by 5) and monthly realized volatility RVm (the sum of 22 RVd divided by 22). The x are just coefficients, and C is a constant.
Answer: To convert realised variance from a weekly rate to a daily rate, we divide by $5.$ To convert realised variance from a monthly rate to a daily rate, we divide by $22.$ Volatility is just the square root of the variance. So we get
$$ \begin{align*}
V_{w} & \rightarrow \frac{V_{w}}{\sqrt{5}} \\
V_{m} & \rightarrow \frac{V_{m}}{\sqrt{22}}
\end{align*} $$
so that your equation should become
$$ V = C + x_{1}V_{d} + x_{2}\frac{V_{w}}{\sqrt{5}} + x_{3}\frac{V_{m}}{\sqrt{22}} $$
The volatilities should now all be at a daily rate.


Using a homomorphism to draw conclusions about subgroup indices

Question: I've been thinking about this problem:
Let $H$ and $K$ be subgroups of $G$. Assume that $K$ has index two in $G$. prove that either $H \cap K$ has index two in $H$, or $H \subseteq K$.
The suggestion was given to use a homomorphism to prove this fact. I'm unclear on what homomorphism to use, the domain and codomain of the homomorphism, and most of all how a homomorphism would even help me prove this fact about the index of $H \cap K$.
So far, I see that $K$ must be a normal subgroup because it has an index of 2, and that $H \cap K$ is a subgroup since it is the intersection of two subgroups of G. Beyond that, I'm kind of stuck on the how to utilize a homomorphism here. Any help finding a path to a solution (not necessarily the solution itself) and understanding how homomorphisms are useful in these situations would be much appreciated.
Comment: Hint: What's the connection between normal subgroups and homomorphisms?
Answer: $\renewcommand{\phi}{\varphi}$Consider the quotient group $G/K$, and the homomorphism $\phi : H \to G/K$ given by $h \mapsto h K$.
Spoiler 1

Since $G/K$ has order $2$, there are two possibilities.

Spoiler 2

Either the image of $\phi$ has order $1$, which means $H \le K$.

Spoiler 3

Or the image of $\phi$ is the whole of $G/ K$. By the first isomorphism theorem, $G/\ker(\phi)$ is isomorphic to $G/K$, so that it has order $2$. But it is immediate to show that $\ker(\phi) = H \cap K$.
Answer: If $\;H\rlap{\;\,/}\subset K\;$ , then there exists $\;h\in H\;$ such that $\;h\notin K\;$, and thus $\;h(H\cap K)\neq H\cap K\;$ , so
$\;[H:H\cap K]\ge 2\;$ .
On the other hand,
$$H/(H\cap K)\cong HK/K\le G/K\implies [H:H\cap K]\le[G:K]=2$$
Answer: Consider the quotient map $\phi \colon G \to G/K$. If $H \not \subseteq K$, then $\phi_{|H}$ will be surjective. Then if we call $\psi := \phi_{|H}$, it is clear that $\ker(\psi) = H \cap K$. By the first isomorphism theorem, $H/(H\cap K) \cong G/K$, and we're done.
Answer: Use the Second Isomorphism theorem:
As $K$ is normal i,n $G$, $HK$ is a subgroup of $G$, and
$$HK/K\simeq H/H\cap K$$is a subgroup of $G/K$. As the latter  has order $2$, $HK/K$ is either $G/K$ or to K/K.
The first case implies $H/H\cap K$ has order two, which is exactly $[H:H\cap K]=2.
In the second case, $HK=K$, which implies $H\subset K$.


Null Coalescing tem prioridade sobre outros operadores?

Question: Ao usar o operador de Null Coalescing ?? em uma expressão com um valor não-nulo, este valor é retornado, como por exemplo:
"foobar" ?? "unbar"        // "foobar"
null ?? "unbar"            // "unbar"

E também sabemos que o literal true não é nulo, pois:
true ?? false              // true

nos dá true de volta. Mas, ao testar a seguinte expressão:
true ?? false == false     // true

É obtido true. Sabemos que true ?? false é true, logo a expressão seria true == false que seria, em teoria, false, mas o contrário é obtido.
É compreensível que o interpretador esteja interpretando false == false primeiro, e comparando true ?? true logo em seguida. Mas, não era para ele interpretar true ?? false primeiro? Qual operador tem prioridade nessa expressão?
Um exemplo prático da reprodução deste problema: https://ideone.com/UAELv0
Answer: É fácil verificar, basta olhar a documentação e ver a tabela de precedência. Ela tem uma prioridade baixa, mas não tanto assim. Especificamente comparando esses dois operadores podemos ver que o ?? com o == percebemos que o primeiro tem menos prioridade então será executado depois. Mais ainda a associatividade é da direita, então primeiro ele tenta resolver o que está mais à direta antes de aplicar a ele. Portanto ele faz assim:
true ?? (false == false)

Porém tem mais uma questão. Pode por o que quiser do lado direito do operador, qualquer valor mesmo, ele não será usado neste caso, porque o lado da esquerda tem um valor válido, então a função do operador é justamente pegar um valor válido e o da esquerda é, portanto:
"foobar" ?? false == false

dá a string. Já:
null ?? false == false

dá true.
Na verdade usar booleano nesses caso só cria confusão, deveria até evitar isso em código normal.
Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.


Moving Average (q) interpretation

Question: What does it mean that Moving Average Process is first-order, second order, third order, etc.
MA(1), MA(2), MA(3)?
How to simple understand it, without any complicated formulas, etc?
Kind regards, thank you for help!
Answer: A moving average process is a weighted "moving average" of a stationary white noise process. Usually it is not an actual average but rather a linear combination of previous noise, or error terms, originating from an unknown process we cannot measure. These noise points are assumed to be independent and identically distributed (and usually Normal).
The order of this process is the number of previous points that are taken into account. A MA(1) process will use only the last time point, MA(2) uses two, etc.
Even if you ask to avoid formulas it is usually easier to read this in a formula instead of in words. I will try to explain what I write in detail.

MA(1): $ x_{i} = \varepsilon_{i} + a_1 \cdot \varepsilon_{i-1}$
MA(2): $ x_{i} = \varepsilon_{i} + a_1 \cdot \varepsilon_{i-1} + a_2 \cdot \varepsilon_{i-2}$
...

This means that at a certain time point (here called $i$) the process will take the value $x_{i}$ which is modelled as the current point in the driving noise $\varepsilon_{i}$ and some factor $a_1$ times the previous noise value $\varepsilon_{i-1}$. In the MA(2) we also have a term for yet another point, two steps back in time.
When fitting a model of this type you generally try to infer the noise variance and regression parameters (i.e find the multipliers called $a$ in the formula) so that you can predict future values or in some other way understand your process better. For a higher order model you therefore need to find more multipliers than you would need to do for a lower order model.
Answer: Just take the ma polynomial (psi weights )and compute it's inverse. This will be the auto-projective polynomial ( pi weights) reflecting how previous values are weighted. In this way the model's memory in terms of the Y's will be better understood and accepted.

Moving Average (q) interpretation


case when exists in oracle update query

Question: I have update query like
update dedupctntest a set a.city = case when exists(
select b.shortname from DEDUPADDRESSDICT where lower(a.city) =lower(b.shortname) and rownum = 1) b  then
b.fullname else a.city end;

but it will give missing keyword error
can anybody tell what is wrong in that ?
Answer: You cannot reference b.fullname outside its scope, which is inside the exists() clause.
Maybe this does what you need:
update dedupctntest a
set a.city = coalesce
              ( ( select b.fullname
                  from DEDUPADDRESSDICT
                  where lower(a.city) = lower(b.shortname)
                  and rownum = 1
                )
              , a.city
              );

i.e. if the query from DEDUPADDRESSDICT returns a non-null fullname use that, else use a.city.  Note that if there is a row in DEDUPADDRESSDICT with a null fullname then a.city will be used.
Answer: Try something like below
update dedupctntest a set a.city = case when exists(
select * from DEDUPADDRESSDICT where lower(a.city) =lower(b.shortname) and rownum = 1) b  then
(select fullname form DEDUPADDRESSDICT where lower(a.city) =lower(b.shortname) and rownum = 1)
b.fullname else a.city end;

case when exists in oracle update query


How solve this differential equation?

Question: This is the differential equations:
$$ y'+ a[sin(x)]+ b[cos(x)]/y =0 $$
where $a,b  \in \mathbb{R}$$and$ $b>0$
How could i solve? Thank you in advance for any idea or proposed solution.
Comment: this equation looked similar to another question posted on here $$y'' + a\sin y y' + b\cos y =0$$ is there a reason for these DEs?
Comment: I posted this equation because it is formally less complex and more 'likely that someone has already seen or solved in the mathematical literature.
If it's a problem I can keep them both a levarne; I apologize if I have committed any breach of Regulation
Comment: The reason why I posted is a study of a structural problem; unfortunately for days looking for a solution to this differential equation, but I can not find
Answer: $y'+a\sin x+\dfrac{b\cos x}{y}=0$
$y\dfrac{dy}{dx}=-ay\sin x-b\cos x$
This belongs to an Abel equation of the second kind.
Let $t=a\cos x$ ,
Then $\dfrac{dy}{dx}=\dfrac{dy}{dt}\dfrac{dt}{dx}=-a(\sin x)\dfrac{dy}{dt}$
$\therefore-ay(\sin x)\dfrac{dy}{dt}=-ay\sin x-b\cos x$
$y\dfrac{dy}{dt}=y+\dfrac{b}{a\tan x}$
$y\dfrac{dy}{dt}-y=\dfrac{bt}{a\sqrt{a^2-t^2}}$
This belongs to an Abel equation of the second kind in the canonical form.
Please follow the method in https://arxiv.org/ftp/arxiv/papers/1503/1503.05929.pdf or in http://www.iaeng.org/IJAM/issues_v43/issue_3/IJAM_43_3_01.pdf
Comment: Thanks so much for your excellent suggestion!
I will follow the road that you have shown me .... thanks again!

How solve this differential equation?


How media queries are work when i define in meta tag content="max-width=1200px;"

Question: How media queries are work when i define in meta tag content="width=1200px;" And my media queries are like
<code>@media (max-width:480px){}
@media (max-width: 600px) {}
</code>
and etc. other media queries.
Comment: There's a lot of information about media queries on the net - https://developer.mozilla.org/en-US/docs/Web/Guide/CSS/Media_queries
Comment: http://www.allenpike.com/2010/choosing-a-viewport-for-ipad-sites/
Answer: content="width=1200px" means your layout needs at least 1200 pixels of width.
For pages that set an initial or maximum scale, this means the width property actually translates into a minimum viewport width.
For example, if your layout needs at least 500 pixels of width then you can use the following markup. When the screen is more than 500 pixels wide, the browser will expand the viewport (rather than zoom in) to fit the screen:
<code><meta name="viewport" content="width=500, initial-scale=1">
</code>
https://developer.mozilla.org/en/docs/Mozilla/Mobile/Viewport_meta_tag
http://www.allenpike.com/2010/choosing-a-viewport-for-ipad-sites/


Compute $\operatorname{cov}(X, \max(X,Y))$ and $\operatorname{cov}(X, \min(X,Y))$ when $X$ and $Y$ are standard normal

Question:
Compute $\operatorname{cov}(X, \max(X,Y))$ and $\operatorname{cov}(X, \min(X,Y))$ where $X,Y \sim N(0,1)$.

i think the way to calculate it is to get
$$\begin{align}
\operatorname{cov}(X, \max(X, Y) + \min(X,Y)) & = \operatorname{cov}(X, X+Y) \\
& = \operatorname{cov}(X, \max(X,Y)) + \operatorname{cov}(x, \min(X,Y)) \\
\end{align}$$
and
$$\begin{align}
\operatorname{cov}(X, \max(X,Y) - \min(X,Y)) & = \operatorname{cov}(X, \operatorname{abs}(X-Y)) \\
& = \operatorname{cov}(X, \max(X,Y)) - \operatorname{cov}(X, \min(X,Y)) \\
\end{align}$$
although this is pretty much as difficult to solve as $\operatorname{cov}(X, \max(X,Y))$  unless there is some particular trick. Anyone can help with this?
Comment: Welcome to MSE. In probability theory, there is a somewhat important distinction between $X$ and $x$ (using the usual notation). Please try to keep these straight and consistent in submitting a question.
Comment: Are $X,Y$ independent?
Answer: Assume that the random variables $X$ and $Y$ are i.i.d. square integrable with a symmetric distribution (not necessarily gaussian).
Let $Z=\max(X,Y)$, then the covariance of $X$ and $Z$ is $\mathbb E(XZ)-\mathbb E(X)\mathbb E(Z)=\mathbb E(XZ)$. Using $Z=X\mathbf 1_{Y\lt X}+Y\mathbf 1_{X\lt Y}$, one sees that
$$
\mathbb E(XZ)=\mathbb E(X^2;Y\lt X)+\mathbb E(XY;X\lt Y).
$$
What is the value of the last term on the RHS? By symmetry, $\mathbb E(XY;X\lt Y)=\mathbb E(XY;Y\lt X)$ and the sum of these is $\mathbb E(XY)=\mathbb E(X)\mathbb E(Y)=0$ hence $\mathbb E(XY;X\lt Y)=0$. Thus,
$$
\mathbb E(XZ)=\mathbb E(X^2F(X)),
$$
where $F$ denotes the common CDF of $X$ and $Y$. Since $X$ is distributed as $-X$, $F(-X)=1-F(X)$ and
$$
\mathbb E(X^2F(X))=\mathbb E((-X)^2F(-X))=\mathbb E(X^2(1-F(X))=\mathbb E(X^2)-\mathbb E(X^2F(X)).
$$
This yields
$$
\mathrm{cov}(X,\max(X,Y))=\tfrac12\mathrm{var}(X).
$$
On the other hand, $\min(-X,-Y)=-\max(X,Y)$ hence, once again by symmetry,
$$
\mathrm{cov}(X,\min(X,Y))=\tfrac12\mathrm{var}(X).
$$
Edit: A much simpler proof is to note from the onset that, since $\max(X,Y)+\min(X,Y)=X+Y$, $\mathrm{cov}(X,\max(X,Y))+\mathrm{cov}(X,\min(X,Y))=\mathrm{cov}(X,X+Y)=\mathrm{var}(X)$, and that, by the symmetry of the common distribution of $X$ and $Y$ and the identity $\min(-X,-Y)=-\max(X,Y)$, $\mathrm{cov}(X,\max(X,Y))=\mathrm{cov}(X,\min(X,Y))$. These two elementary remarks yield the result and allow to skip nearly every computation.
Answer: Assuming that $X$ and $Y$ are independent random variables,
the direct way to solve this problem would be to compute $E[X\max(X,Y)]$ and
$E[X\min(X,Y)]$ via integration, breaking the double integral into two
double integrals over the regions where the maximum is $y$ and where
the maximum is $x$, and then using a change to polar coordinates.
We have
$$\begin{align*}
E[X\max(X,Y)] &= \int_{y=-\infty}^\infty\int_{x=-\infty}^y
\frac{xy}{2\pi}e^{-(x^2+y^2)/2}\,\mathrm dx\,\mathrm dy
+ \int_{y=-\infty}^\infty\int_{x=y}^\infty
\frac{x^2}{2\pi}e^{-(x^2+y^2)/2}\,\mathrm dx\,\mathrm dy\\
&= \int_0^\infty\int_{\pi/4}^{5\pi/4}
\frac{r^2\cos\theta\sin\theta}{2\pi}e^{-r^2/2}\,r\,\mathrm d\theta\,\mathrm dr
+ \int_0^\infty\int_{-3\pi/4}^{\pi/4}
\frac{r^2\cos^2\theta}{2\pi}e^{-r^2/2}\,r\,\mathrm d\theta\,\mathrm dr
\end{align*}$$
and similarly for $E[X\min(X,Y)]$.
Answer: The OP states $X,Y$ ~ $N(0,1)$, but doesn't specify whether $X$ and $Y$ are independent or dependent.
Whereas the other posters assume independence, instead  ... consider here the more general problem that nests same, namely $(X,Y)$ ~ standardBivariateNormal, with joint pdf $f(x,y)$:

The general solution to Cov[$X$, max$(X,Y)$] ... obtained here using the mathStatica / Mathematica combo ... is simply:

The 'min' case is symmetrical, but here it is anyway:


Give a counter example

Question: I was considering the possibility of writing primes (except 2 and 3) in the form 2n + 3 for some positive integer n. Is it always possible ?
Comment: This just asks whether every prime $p>3$ is odd, i.e., of the form $2m+1$ with $m>1$ - which is clearly true.
Answer: Yes, it is always possible. Indeed a prime number $p>3$ is odd and therefore can be written $p=2m+1$ with $m > 1$ (since $p>3$). Then $p=2(m-1+1)+1=2(m-1)+3=2n+3$ with $n=m-1$ a positive integer.
Note that this result is very general and does not help to distinguish prime numbers since it is true for every odd number greater than 3.
Answer: Every odd number $p$ greater than 4 can be written in the form $2n + 3$; just choose $n = \frac{p-3}{2}$. So this statement is certainly true for all primes other than 2 and 3.

Give a counter example


How do I setup mail/mailx in linux to mail to outside domain?

Question: If I understand email correctly, I need to setup the smtp configuration before my client can send email to outside domain? If that is the case, where do I set this up in linux's mail/mailx command?
Answer: mail/mailx expects to be able to send mail via a sendmail command (which is traditionally the way to send mail on Unix).
The convention is that there is a command  sendmail that accepts a mail on stdin.
While this convention originated with the "sendmail" mailserver (duh), most other mail servers provide such a sendmail command. So you need to setup a locally running mail server, at least for sending mail.
There are many to choose from, most Linux distros set one up for you, usually postfix or Exim. Check your distro's docs for details.
Answer: If you are on Fedora/Redhat/Centos either Postfix or sendmail is available.  Postfix is probably the easier to configure.  You will also need an MAA such as Dovecot to retrieve your incoming mail.
Answer: You can setup postfix email relay through an external smtp server (which can be gmail or your own ISP).
see here:
https://easyengine.io/tutorials/linux/ubuntu-postfix-gmail-smtp/
and here:
https://www.howtoforge.com/tutorial/configure-postfix-to-use-gmail-as-a-mail-relay/
after that you can send email from command line, and example from Debian 9 stretch.
Note the -e for echo to allow newline characters, and -r for mailx to show a name along with an outgoing email address:
$ echo -e "testing email via yourisp.com from command line\n\nsent on: $(date)" | mailx -r "Foghorn Leghorn <firstname.lastname@example.com>" -s "test cli email $(date)" -- firstname.lastname@example.com

Hope this helps!


Hook to change display widget in facet api from template.php

Question: In my site i have search api with one block facet option "Display widget" set to "Links with checkboxes"

but now i want to change the value to "Links"

because when the website is displayed on a mobile device i need to change to this option value.
Is there a way to change this option through any hook or another way to get this accomplished?
Maybe put a hook in template.php is my closes approach.
Thanks :)
Comment: Just wondering, what about a JS solution?
Comment: @Lance yes that solution would be valid too :)
Answer: Create another index and configure facet display differently


Tube Amp to Headphones

Question: I have a B-52 AT-100 all-tube amplifier head. I currently have it connected to a LINE-6 Spider 412 four-speaker cabinet. In order to really drive the tubes, I have a vintage Scholz Power Soak, which is a bunch of resistors that allow the user to step down the volume output while still running the head at maximum volume.
The setup looks kind of like this:

The quietest setting on the Power Soak is -32Db, which allows for playing at a reasonable volume while still driving the tubes at max power.
While the -32Db setting on the Power Soak is labeled "PHONES," presumably indicating that it's possible to drive headphones instead of a cabinet.
However, it's still really a lot of power output, I still have to turn down the main volume to about 30% in order to play it in my house. Without the Power Soak, it's likely that I'd blow out the windows in a fifteen mile radius.
In interests of space, I'd like to get rid of the cabinet and simply take the output of the Power Soak and route to actual headphones or a recording hub. I'm not really ever playing stadiums, so the cab is a bit overkill. However, I really like the sound of the amp being driven through the Power Soak.
What are my options for getting rid of my cabinet in a way that would allow me to play with headphones? I could always buy another Power Soak and step things down even further, but as I'm trying to reduce physical space as much as possible, is there a better option?
Comment: Do you really need to use such a massive full-tube head just for headphones practising? I know the Spider is much-hated for its sound, but for practising it should be pretty well-suited. In fact it'll almost certainly sound _better_ with headphones than the tube head does (because that doesn't have any cabinet emulation). And for recording or live, where sound really matters but loudness isn't such a problem, use the tube amp into a better cabinet, with or without the power soak.
Answer: The main purpose of the power soak is not to reduce the signal level, but to offer a reliable low impedance like a cabinet, even if none is connected. A tube power amp requires such an impedance, else it'll destroy itself.
With the power soak in place and set to -32 dB, it doesn't really matter anymore what you connect. You can as well step it down yet a bit further with a simple voltage divider. This can be built from ordinary resistors†, which you can buy for less than 1$.

You can also use a passive headphones splitter with volume control, that's the same as a voltage divider but with a dial.
Just, as I already commented, this doesn't really make any sense. Even the best tube amp does not sound "right" if it's not connected to a proper guitar cabinet (either real or simulated). Many power-soak-like devices have simple cabinet emulation built in for recording or headphones use. In that case, the headphones output also has an extra volume control.
If you want good sound, you'll need the cabinet. The usually more sensible alternative is to just go all-digital for practising.

†Basically, the power soak also consists only of simple resistors, but these need to be able to dissipate lots of power, which is the real engineering challenge.

Tube Amp to Headphones


MIDI routing (?) in Ableton freezes after a few seconds... any ideas?

Question: Setup: M-Audio Keystation 49 -USB-> Windows 10 PC running Ableton Live hosting the AddictiveKeys VSTi. I've already stripped other stuff out of the equation to make things simpler.
[See updates below -- various tests that establish the problem is internal to Live, not a hardware issue.]
This has been working for several months with no issues (in fact, as part of a much more complicated setup), but I had to disassemble my studio last week and reassemble it a few days later and now I have a weird problem. In the intervening time the PC has been unplugged and since turning it back on I haven't updated any software (including Windows).
Here is what happens:

Fire up Ableton. Everything is fine. Can play AddictiveKeys with the keyboard.
5-10 seconds later, the last note(s) played on the keyboard sustain forever and further key presses result in no response from AddictiveKeys.
I go to Options -> Preferences -> Link MIDI and turn off the Track Input for the keyboard, then turn it back on again. Everything starts to work fine again for 5-10 seconds and the cycle repeats.

The "MIDI Track In Indicator" in the top right corner of the screen continues to flash when notes are pressed, which I think confirms that the keyboard is working correctly. But the Max4Live MIDI monitor on the track shows no new notes coming in, as does the little MIDI monitor on the track itself.
Furthermore, MIDI-Ox shows the keyboard sending MIDI fine and doesn't cut out or do anything else unexpected.
I think this must mean that some internal routing within Ableton is malfunctioning, but I can't see how to diagnose this any further.
What I've tried that didn't help:

Swapping USB cables and ports
Removing all unnecessary USB devices
Swapping AddictiveKeys with another VSTi
Turning off and on again (PC and Ableton itself)
Looking at Ableton's website for customer support options (I'm on a Lite license I got free with a piece of gear so I'm out of luck).
Googling. I see a couple of people reporting a similar issue on other forums but with different setups and no suggestions for how to fix it.
Setting the MIDI track in Ableton to listen only on the channel that the keyboard sends on (not All Channels).
Tested my external devices are working in MIDI-OX and LMMS (everything works fine in both, no sign of any weirdness).

If anyone has any ideas what to try next I'd be much obliged -- I'm relatively new to Ableton Live.
UPDATE: I rebooted the PC with no USB devices attached except the audio interface, keyboard and mouse. Created a new Live set with only one channel for AddictiveKeys, with MIDI input coming from the computer keyboard. Turned on the keyboard button in the top right and it plays fine -- for 5-10 seconds. Then freezes again in exactly the same way. So the issue is certainly internal to Ableton, not with any external device.
As an aside, I also tested that all my USB ports are powered, which they are (tested by plugging in a USB-powered LED lamp). It's quite possible that since tearing down & setting back up, devices got plugged into different ports but I've tried enough combinations now to be pretty sure that's not the issue.
UPDATE 2: Further tinkering has revealed more problems with Ableton between its tracks' MIDI commands and what goes in and out. If I plug in an external hardware synth and write a simple MIDI track in Ableton, it shows the MIDI note on/off messages being generated on the correct channel (the MIDI note indicator jumps at the appropriate moments), but no MIDI data is arriving at the other end. Again, this was working perfectly before and MIDI-OX can send notes to the external synth no problem at all.
UPDATE 3: I tried deleting both Ableton folders in my user directory; this appeared to restore Live to "factory settings", forgetting all my preferences and my activation code. But the problem was still present. I guess it's time to give up on Live and move to a different product.
Comment: when the last note gets stuck on, that normally means the MIDI Note Off message was missed somehow. What happens when you use one of Ableton's internal MIDI instruments?
Comment: look for a setting in the Ableton MIDI menu that always sends NOTE OFF at end of a clip, although that doesn't exactly match your issue it sounds related.
Comment: Good thoughts. I just tried with Electric and got exactly the same behaviour. There does seem to be an issue with the last note getting stuck but since these are polyphonic instruments I don't think that can be the *whole* problem...
Comment: I would just delete every control surface in Ableton, and then start from scratch. It sounds like something is muddled up somewhere. Odd problem though.
Comment: I think I tried that, in the sense that if you unplug everything and start Live it doesn't see any control surfaces, they're not present in the Preferences window anyway... then as you plug them in it sees them again. Maybe there's a way to "reset to factory settings" as it were without a complete reinstall (which is my next try, I think).
Comment: yes really delete them and then plug in and configure only the Keystation.
Comment: delete all control surfaces AND all MIDI ports. Then add them one by one and get them going.
Comment: Sadly no luck: no control surfaces or MIDI ports, no external hardware connected at all, just playing a VSTi with the computer keyboard and the same problem occurs. I think a full uninstall & reinstall is my next step; failing that I'll have to go back to Cubase I guess :-(
Comment: How has Ableton support responded?
Comment: @piiperi I'm on Lite, which means I can't access any support except the online FAQs. Their official forum used to have a space for users to provide mutual help but they shut it down.
Comment: Is the same problem with Ableton's bundled synth plugins? Have you tried playing/typing on the "Computer MIDI Keyboard" during the MIDI freezes? To activate it, select it from the Options menu or press Cmd-Shift-K on the Mac, maybe Ctrl-Shift-K on Windows. I guess Live Lite has it too?
Comment: @piiperi Yes, I tried both -- even just playing an internal instrument (e.g. Electric) with the computer keyboard reproduces the problem.
Answer: To satisfy my curiosity, I uninstalled and reinstalled Live this morning. Everything was working fine after that. So in case someone in the future encounters this problem and despite that wants to persist with Ableton, that's the only thing that worked for me.
Thanks to @danmcb and @piiperi for helping me through the debugging process.
Answer: Hey no need to uninstall live either I've found solution. We can reset the preference and it works as it was working before. Imma put down the link so check it out if you're having the Same issue. It was really pain in the ass so i had to share it:
https://help.ableton.com/hc/en-us/articles/209070609-How-to-reset-Live-
Comment: Please add the gist of how your solution works. Answers that are just links eventually get deleted.


Euler lagrange equation for the following functional

Question: I'm going through some video lectures that show how to apply variational methods in computer vision: here the Euler lagrange equations are derived.
I do wonder however how would this apply for a functional of the form
$$
E_{\lambda}(u) = \left[\int_{\Omega} (u(x)-f(x))^2 +\lambda \lVert \nabla u(x)\rVert^2 \right] dx
$$
Where $\Omega = [0,X]\times[0,Y]$, $f$ a given function and $u : \Omega \rightarrow \mathbb{R}^2$ a suitable function for differentiation (i.e. it does belong to an appropriate function space. $\lambda$ is a positive parameter.
I'm totally confused how to apply the euler lagrange formula to my case, where my function $\mathcal{L}$ is of the form $\mathcal{L}\left(x,u,\nabla u \right)$
I can easily find $\frac{\partial \mathcal{L}}{\partial u}$ but when it comes to something that should be $\frac{\partial L}{\partial \nabla u}$  I get totally confused.
Answer: $$E_\lambda(u)=\int_\Omega J_\lambda(u) dx=\int_\Omega [u(x)-f(x)]^2 + \lambda||\nabla u||^2\,dx$$
Take the first variation:
\begin{align}
\frac{\delta J_\lambda}{\delta u} &= \frac{\partial J_\lambda}{\partial u} - \sum_j \frac{\partial}{\partial x_j}\left(\frac{\partial J_\lambda}{\partial u_{j}}\right)\\
&= 2(u-f)-\sum_j \frac{\partial}{\partial x_j} [2\lambda u_j]\\
&= 2(u-f)-2\lambda\sum_j u_{jj} \\ &= 2(u-\lambda\,\Delta u-f)
\end{align}
where $u_j := \partial_j u = \partial u/\partial x_j,\, u_{jj} := \partial_{jj} u = \partial^2 u/\partial x_j^2$.
So the Euler-Lagrange equations are: $$ \lambda\,\Delta u = u-f $$ unless I've made a mistake.


How to find the sum of largest odd factor of a number?

Question: I've a problem to find sum of largest odd factor of a number ( F(x) ) and F(x) = f(1)+f(2)+...+f(x). As you know the largest factor of 1 is 1, 2 is 1, 3 is 3, 4 is 1, and so on...
e.g The sum of largest odd factor of a number 6 is f(1)+f(2)+f(3)+f(4)+f(5)+f(6), that is 1+1+3+1+5+3 or 14.
And I want to try to solve a number until 2*10^9
So this is my code for the f(x) that get 82/100 before timeout
unsigned long long int biggestOddFactor(unsigned long long int n){
    //!(n & (n - 1))
    while(n%2 == 0){
        n /= 2;
    }
    return n;
}

This is the another method by removing the zero on the last bit of a number, but it only make 77/100
#include <bitset>

unsigned long long int biggestOddFactorUsingBinary(unsigned long long int n){
    std::string bin = std::bitset<32>(n).to_string();
    int delet = 0;
    for(int i = bin.length()-1; i >= 0; i--){
        if(bin[i] == '0'){
            delet += 1;
        }else{
            break;
        }
    }
    bin = bin.substr(0, bin.length()-delet);
    return std::bitset<32>(bin).to_ulong();;
}

Is there any way to optimize my algorithm?
Comment: You mean "odd factor", not "even factor", or?
Comment: a lookup would probably be faster, the idea is this recursion `f(x)=x%2?f(x/2):x`
Comment: and you may also need `F(x)=f(x)+F(x-1)`
Comment: or maybe use something to count number of trailing zeros
Comment: Using recursion only make it slower.. I only get 36/100 using it
Comment: @KlausGütter ah right. Sorry, I've edit it
Comment: Why does it need to timeout?
Comment: "get 82/100 before timeout" what does this mean?
Comment: @n.m. It means his code processes 82 of the 100 test inputs before it uses up more time than allowed for the solution to run.
Answer: You're asking the wrong question. The problem is not that you are finding the biggest odd factor too slowly, the problem is that your algorithm for finding the sum is too slow, not that this one part of it is too slow.
For example, the largest odd factor of every odd number is the number itself, and there's a formula for the sum of the first n odd numbers. Why are you not using that to halve the number of times you call biggestOddFactor? That's just for starters.
The largest odd factor of any even number is the same as that for half that number. So the sum of the largest odd factors of, say, 16, 14, 12, and 10 is the same as that for 8, 7, 6, and 5. Yet you compute these two ranges separately? Why?
And so on. You need to optimize your algorithm, not your implementation of a bad algorithm. The concepts above suggest several possible recursive implementations that will be much faster.
I just very quickly whipped up a solution to this problem using a better algorithm and it's thousands of times faster than just calling biggestOddFactor on every number. Note that my solution is recursive.
You should always consider algorithmic optimizations before you try to micro-optimize an implementation. The payoff tends to be much greater and the result is much less fragile.
Comment: I misunderstood, by *sum of the largest odd factors of, say, 16, 14, 12, and 10 is the same as that for 8, 7, 6, and 5* you meant `f(16) + f(14) + f(12) + f(10) = f(8) + f(7) + f(6) + f(5)`?
Comment: Did you mean that the largest factor of 16 is the same as the largest factor of 8? The sum of largest factors for 16 is `f(1) + f(2) + ... + f(15) + f(16) = 1 + 1 + 3 + 1 + 5 + 3 + 7 + 1 + 9 + 5 + 11 + 3 +  13 + 7 + 15 + 1 = 86` but for 8 is `f(1) + f(2) + ... f(7) + f(8) = 1 + 1 + 3 + 1 + 5 + 3 + 7 + 1 = 22`
Comment: I'm not sure I understand what you're asking.
Comment: This is an excellent answer - for `2e9` it's not thousands of times faster but *millions* of times faster. Although I note that for `2e10` `naive()` and `smart()` return different values (`4 206 124 817 370 522 026` and `8 817 810 835 797 909 930` respectively). Is that perhaps due to overflow at `return (max * max - min * min)/4;`? I can't think what else would cause that disagreement...
Comment: @Ghost I suspect so. There are ways to modify that code to avoid an overflow if needed. `((max/2) * (max/2))-((min/2)*(min/2))` might help.


How to submit HTML form into a Sheet?

Question: I've checked around and most things I've found are from 2012 and workarounds for a then-existent bug. Unfortunately I'm not understanding Google's documentation on this very well.
I have a script project that is serving a web page to visitors with an HTML form:
<form id="gradingform">
<input type="text" name="name" placeholder="Name">
<input type="number" name="grade" placeholder"100">
<input type="submit" onclick="<this is where I'm having issues>">
</form>

I believe that this needs to be handled like any other time getting a script while serving a web page - by using the google.script.run. With a form specifically, I think that it's supposed to be using a success handler, so for example, something along the lines of
google.script.run.withSuccessHandler(gradeSubmitted).recordGrades()
gradeSubmitted() would be a function that just dispays a message, easy enough by doing some easy div changing. What my real issue is what recordGrades() would be like.
How do I pass the form to this function, and how do I collect the information from the form? From there I will be adding it to a Sheet, which is easy enough once the information is in an array because I can just append it. The documents say the form information should be passed as a Blob, but Google's example is kind of confusing.
Answer: This is what you have to do
onclick="google.script.run.withSuccessHandler(gradeSubmitted).recordGrades(this.form)"
And in the code.gs file you will be receiving a json as
[16-03-25 10:51:51:046 IST] {grade=10, name=Anees}
Comment: Every time I press the submit button the page goes completely blank/white - as if I were trying to redirect somewhere when I'm doing no such thing - I have the same code for the button as before, with simply the new onclick part. Any ideas?
Comment: It is because of sandbox mode set as HtmlService.SandboxMode.IFRAME. You can apply a quick fix by using HtmlService.SandboxMode.NATIVE. In your doGet HtmlService.createHtmlOutputFromFile('myfile').setSandboxMode(HtmlService.SandboxMode.NATIVE);
Comment: I invite you learn more javascript and jquery so that you can use full capabilities on HTML services in GAS. :)

How to submit HTML form into a Sheet?


UItextField data disappears when I scroll through the TableView -- Swift

Question: I'm having problems with my app. I have a table view where every cell consists of a textfield. When i write in it and scroll down, than scroll back up, the data i wrote in it disappears.
These are some of my functions in ViewController
class ViewController: UIViewController, UITableViewDataSource, UITableViewDelegate, UITextFieldDelegate, UIScrollViewDelegate {

    var arrayOfNames : [String] = [String]()
    var rowBeingEdited : Int? = nil

    public func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
        return initialNumberOfRows
    }

    var count: Int = 0;
    public func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        let cell: TableViewCell = tableView.dequeueReusableCell(withIdentifier: "Cell") as! TableViewCell

        if(arrayOfNames.count > 0 && count < arrayOfNames.count) {
            cell.TextField.text = self.arrayOfNames[indexPath.row]
        }else{
            cell.TextField.text = ""
        }

    count += 1

        cell.TextField.tag = indexPath.row
        cell.TextField.delegate = self
        return cell
    }

func textFieldDidEndEditing(_ textField: UITextField) {
    let row = textField.tag
    if row >= arrayOfNames.count {
        for _ in ((arrayOfNames.count)..<row+1) {
            arrayOfNames.append("") // this adds blank rows in case the user skips rows
        }
    }

    arrayOfNames[row] = textField.text!
    rowBeingEdited = nil

}

func textFieldDidBeginEditing(_ textField: UITextField) {
    rowBeingEdited = textField.tag
}
}

As you can see, I'm saving all of my written text in the textfield into an array. Any help would be appreciated.
Thanks in advance
Answer: When you scroll back up tableView(tableView: cellForRowAt:) gets called again. Inside that method you increment count every time that is called, thus instead of using the first condition it goes to the second conditional statement that sets cell.TextField.text = "" as count is probably greater than arrayOfNames.count. What are you using count for? Maybe rethink how you could code that part a little better.
Comment: Instead maybe do `indexPath.row < arrayOfNames.count` so you won't access an element outside of your array.
Comment: count gets incremented because I'll get an out of bounce error for trying to access an element in my array
Answer: You cells are recreated. So you lose them. You could use the method PrepareForReuse to set the text back when they are recreated.
Comment: Can you guide me a little more on how to do that?..I can't find that method


Why does this edition use an upside-down up-bow to indicate down-bow?

Question: I have a piece here that uses upside-down up-bows to indicate down-bow, which I have never seen before. Since I know other editions of the same opus, I know that they have the meaning of a normal down-bow. Does anyone know anything about this? Yesterday I confused this example with the post about Upside-down down-bows, but here it obviously is a different matter. Does it have to to with the engraving technique of a certain time or area, since this print is from Russia as stated below?

Sebastian Lee; Op. 30 - St. Petersburg: M. Bernard, n.d.(ca.1873).
Answer: He explains this right at the start of the book (First Edition, from IMSLP).
Tirez = pull, Poussez = push. In other words, ^ is a different notation for a down bow.
Comment: @CarlWitthoft yes exactly! Seems like this comment applies mostly to itself, though a question is not a manual. ;-) +1 for adding this snippet, but it does not really answer the question. I was rather gaining towards some background information about why this upside-down symbol was used in certain editions instead of the common down-bow symbol.
Comment: RTFM FTW!  :-) .  Ain't it amazing what people miss?
Answer: There's rarely a 'why' answer to this sort of question.   You've now enriched your musical knowledge by knowing it happens.  Yes, it could be confused with a marcato symbol. But the notation has been explained in this edition.  Good.
Answer: The ^ symbol in music, for a violin, generally indicates a Marcato bow, where you push down on the string more or put more force. It's an accent, but instead of playing it like >, (where you emphasize the start of the note) it looks like ^ because you're pretty much emphasizing the entire note, not just the start. Think of it as a 50% sfz.
Comment: While this is correct in general, it doesn't describe the meaning of this symbol in this edition, where it indicates downbow.


A criteria for an open subset of a compactly generated space to be compactly generated

Question: Is the following statement true or false?

An open subset $U$ of a compactly generated space $X$ is compactly generated if each point of $U$ has an open neighborhood in $X$ with closure contained in $U$.

(Added 2/16/2023) Summary of the answers:
The answer depends on which notion of "compactly generated" (CG) space is used.  There are multiple variants of the concept, as mentioned here for example, or in Stefan Hamcke's comment.
Definition 1: $X$ has the final topology with respect to the inclusions from all its compact subspaces.  This is the definition in wikipedia.
Definition 2: $X$ has the final topology with respect to all continuous maps from arbitrary compact Hausdorff spaces.  This is the definition in nlab and is the one more commonly used in algebraic topology.
Eric Wofsey's answer: with Definition 1, the desired statement is true.  Also note as a consequence:

(With definition 1)  In a CG regular space, all open subspaces are CG.

David Carchedi's answer: with Definition 2, all the open subsets of a CG space are CG.
Comment: In $k$-spaces, every open subset is a $k$-space. You can find a proof in http://neil-strickland.staff.shef.ac.uk/courses/homotopy/cgwh.pdf. There are however compactly generated spaces which are not $k$-spaces, so this is maybe false for this class of spaces.
Comment: Each open subspace of a Hausdorff compactly generated space is compactly generated. Are you interested in non-Hausdorff case?
Comment: By compactly generated, do you mean *A set is closed if it intersects every compact $K$ in a set closed in* $K$, or *A set $S$ is closed if $t^{-1}(S)$ is closed in $K$ for any continuous map $t$ from a compact Hausdorff space $K$ to* $X$ ? For Hausdorff spaces both definitions agree. The latter one is actually for so-called *$k$-spaces*, but some people use it for compactly generated spaces.
Answer: ANY open subset of a compactly generated (possibly non-Hausdorff) space is compactly generated. Indeed, let $U$ be an open subset of a compactly generated space $X.$ Let $A$ be a subset of $U$ such that for any continuous map $f:K \to U$ from $K$ compact Hausdorff, $f^{-1}\left(A\right)$ is open in $K$. We want to show that $A$ is open in $U.$ But as $U$ is open in $X,$ this is iff $A$ is open in $X.$ So let $g:C \to X$ be a continuous map from a compact Hausdorff space. It suffices to show that $g^{-1}\left(A\right)$ is open in $C.$ Now, $g^{-1}\left(U\right)$ is an open neighborhood of $g^{-1}\left(A\right)$ in $C$ so it suffices to show that $g^{-1}\left(A\right)$ is open in $g^{-1}\left(U\right)$. But, as an open subspace of a compact Hausdorff space, $g^{-1}\left(U\right)$ is locally compact Hausdorff, and hence compactly generated. So it suffices to prove that for any map $h:D \to g^{-1}\left(U\right)$ with $D$ compact Hausdorff, $h^{-1}\left(g^{-1}\left(A\right)\right)$ is open in $D.$ But $$D \stackrel{h}{\to} g^{-1}\left(U\right) \stackrel{\tilde g}{\to} U$$ is a continuous map to $U$ from a compact Hausdorff space, where $\tilde g$ is the restriction of $g,$ so by assumption, $h^{-1}\left(\tilde g^{-1}\left(A\right)\right)$ is open in $D.$ But as $g^{-1}\left(U\right)$ contains $g^{-1}\left(A\right),$ this is the same as $h^{-1}\left(g^{-1}\left(A\right)\right),$ finishing the proof.
Comment: Note: this is assuming Definition 2 of compactly generated spaces (see the summary in the question).
Answer: This is true.  Suppose $A\subseteq U$ is closed with respect to the compactly generated topology on $U$ and let $V=U\setminus A$.  We wish to show that $V$ is open in the subspace topology on $U$, or equivalently that $V$ is open in $X$.  Let $K\subseteq X$ be compact; it suffices to show that $V\cap K$ is open in $K$.  Fix a point $x\in V\cap K$ and let $W\subseteq X$ be an open neighborhood of $x$ whose closure is contained in $U$.  Then $L=\overline{W}\cap K$ is closed in $K$ and hence compact, and is contained in $U$.  Since $A$ is closed with respect to the compactly generated topology on $U$, $A\cap L$ is closed in $L$, and so $V\cap L$ is open in $L$.  Since $x\in V\cap L$ and $L$ contains a neighborhood of $x$ in $K$ (namely $W\cap K$), $V\cap L$ contains a neighborhood of $x$ in $K$, and hence so does $V\cap K$.  Since $x\in V\cap K$ was arbitrary, this means $V\cap K$ is open in $K$.
Comment: Note for readers' benefit: This is assuming Definition 1 of compactly generated spaces (see the summary in the question).  I added the summary as the OP is long gone from the site.


É possível fazer um "texto para voz" com Pyqt?

Question: Estou aprendendo a mexer com Pyqt. Estou montando uma aplicação simples, onde vai haver uma notificação por voz quando uma nova solicitação estiver disponível.
Eu gostaria que minha aplicação escrita em Pyqt4 pudesse "transformar" um texto em voz.
Consegui fazer isso com HTML5 e Javascript, mas agora eu gostaria de saber se existe alguma forma de fazer algo parecido com Pyqt.
Existe alguma funcionalidade em Pyqt ou que possa ser usada junto com Pyqt para fazer Text to Speech (texto para voz)?
Answer: Eu acredito que o PyQt4 não tenha suporte para isto, e o pyqt5 talvez só venha suporte em mais no futuro (aparentemente QtSpeech é bem recente), então usar PyQt não é opção, todavia encontrei isto: https://pypi.python.org/pypi/pyttsx (está um pouco outdate 2012), mas ele é cross-platform e suporte Python2 e isto https://github.com/pndurette/gTTS
pyttsx
Drivers necessários por ambiente:

nsss - NSSpeechSynthesizer no Mac OS X 10.5 e superior
Detalhes: https://developer.apple.com/reference/appkit/nsspeechsynthesizer
sapi5 - SAPI5 no Windows
Faça o download em https://www.microsoft.com/en-us/download/details.aspx?id=27224
espeak - eSpeak em distros linux
Instalar no Debian e Ubuntu:
sudo apt-get install espeak

Para instalar manualmente: http://espeak.sourceforge.net/download.html

Depois pode instalar ele via PIP:
pip install pyttsx

Veja um exemplo:
import pyttsx

engine = pyttsx.init()
engine.say('Hello World!')
engine.runAndWait()

Links:

Repositório: https://github.com/RapidWareTech/pyttsx
Documentação: http://pyttsx.readthedocs.io/en/latest/

Nos releases do github https://github.com/RapidWareTech/pyttsx/releases e talvez no pip só apareça a versão 1.1, mas na doc já informa sobre a versão 1.2, no caso seria só instalar isto https://github.com/RapidWareTech/pyttsx/tree/master/pyttsx manualmente

A lib por si só não possui os idiomas instalado, quem tem que ter instalado é o driver usado, no caso do eSpeak acho que ele já vem com portugues, para configurar o idioma na lib pode-se usar algo como, o for é para obter todas as vozes:
engine = pyttsx.init()
voices = engine.getProperty('voices')
for voice in voices:
   engine.setProperty('voice', voice.id)  # troca a voz
   engine.say('The quick brown fox jumped over the lazy dog.')
engine.runAndWait()

Então você vai selecionar a voz que deseja, note que voices = engine.getProperty('voices') retorna a classe class pyttsx.voice.Voice, que tem as seguintes propriedades:

.age retorna a idade em anos como integer
.gender retorna uma String que contem o gênero de voz podendo ser female, male ou neutral
.id é o identificador da voz para poder usar no pyttsx.engine.Engine.setPropertyValue()
.languages retorna uma list de strings que mostra os idiomas suportados pela voz
.name retorna uma String contendo um nome "humano" para ajudar a assimilar idioma, genero, etc

No caso você pode usar voice.languages para obter o ID do idioma português e assim setar ele.
Se não existir o idioma em portugues na lista é porque o driver não suporta ou não tem instalado o idioma, no caso do linux o e eSpeak já vem com vários idiomas:

http://espeak.sourceforge.net/languages.html

O Windows também parece já possuir alguns incluindo os em português:

https://msdn.microsoft.com/en-us/library/hh378476(v=office.14).aspx

O que gostei no espeek é que ele tem suporte para Windows e MacOSX, só não sei se a lib é compatível

gTTS
O gTTS (Google Text to Speech) é uma interface que transforma texto em audio e salva em um .mp3
Para instalar rode o PIP:
pip install gTTS

Uso:
from gtts import gTTS

tts = gTTS(text='Hello', lang='en')

tts.save("hello.mp3") #Salva o arquivo

Em português:
from gtts import gTTS

tts = gTTS(text='Olá', lang='pt-br')

tts.save("ola.mp3") #Salva o arquivo

Idiomas suportados: https://github.com/pndurette/gTTS#lang_list


How To Deal With Datebase Size?

Question: I'm using the Domain Access module to share the same database with six different sub-domains. My concern is the database limit which is set to 1000 MB.
Is there a way to create new a database once the previous one reached  700 MB?
Comment: I think limit the database to 700MB is not a good idea, instead I would think to split your subdomains in separate Drupal installations.
Answer:
Use  the Memcache API and Integration module or the APC - Alternative PHP Cache module to store session and cache data
Use a log back-end that doesn't store data in the database
Don't store logs too long, if you have to keep them in the database.

If the above fails, just change your hosting company or hosting plan, to get bigger limit. Database space is cheap, nowadays.
Comment: @Bala nope, Boost provides only a HTML copy of your site for faster serving of static content. It does not prevent logs from being created, nor does it prevent sessions - if they happen, they happen, boost or not. It may help a bit with anonymous user sessions if they are initiated on pages that should be static (and are not excluded from cache in Boost config), but that's all.
Comment: Molot will boost module do that thing or not ?

How To Deal With Datebase Size?


How to find change occurance points in a Spark dataframe

Question: I have a simple dataframe as an example:
val someDF = Seq(
  (1, "A"),
  (2, "A"),
  (3, "A"),
  (4, "B"),
  (5, "B"),
  (6, "A"),
  (7, "A"),
  (8, "A")
).toDF("t", "state")

// this part is half pseudocode
someDF.aggregate((acc, cur) => {
    if (acc.last.state != cur.state) {
        acc.add(cur)
    }
}, List()).show(truncate=false)

"t" column represents points in time and "state" column represents the state at that point in time.
What I wish to find is the first time where each change occurs plus the first row, as in:
(1, "A")
(4, "B")
(6, "A")

I looked at the solutions in SQL too but they involve complex self-joins and window functions which I don't completely understand, but an SQL solution is OK too.
There are numerous functions in spark (fold, aggregate, reduce ..) that I feel which can do this, but I couldn't grasp the differences since I'm new to spark concepts like partitioning, and it's a bit tricky if the partitioning could affect the results.
Answer: You can use the window function lag for comparing with the previous row, and row_number for checking whether it's the first row:
import org.apache.spark.sql.functions._
import org.apache.spark.sql.expressions.Window

val result = someDF.withColumn(
    "change",
    lag("state", 1).over(Window.orderBy("t")) =!= col("state") ||
    row_number().over(Window.orderBy("t")) === 1
).filter("change").drop("change")

result.show
+---+-----+
|  t|state|
+---+-----+
|  1|    A|
|  4|    B|
|  6|    A|
+---+-----+

For an SQL solution:
someDF.createOrReplaceTempView("mytable")
val result = spark.sql("""
    select t, state
    from (
        select
            t, state,
            lag(state) over (order by t) != state or row_number() over (order by t) = 1 as change
       from mytable
    )
    where change
""")

Comment: is it possible to express this as a single SQL string, as in `spark.sql("SELECT * FROM myTable").show()` ?


apply running median for non equidistance daily measurements with potentially several KPI measurements per day

Question: Let us say I have daily measurements of a KPI with potential gaps (i.e. no measurements some days) and sometimes several measurements per day. How would one apply the running median (see for example here) in this situation? The intention is to show a trend of the underlying process's KPI. Should one average the values for days with several measurements (e.g. using the median)?
I have used statistical process control charts (SPCCs) and applied box-cox transformations but am not to sure whether the underlying assumption of normality is violated.
I read somewhere that one can use (monthly ?) bootstrapping and then apply SPCCs. Pretty sure I could program this but I lack the "statistical confidence" to judge whether it is worthwhile.
Thanks!
Comment: @cs0815, what do you think of my answer?
Comment: What do you hope the running median will do? What property of the process is it intended to reflect, or how do you plan on interpreting or using it?  There's no way to provide an objective answer without such information.
Comment: obviously this is already closed so not sure if commenting helps? Maybe whoever closed it, should at least give me a chance without shooting me down immediately? Any way it is quite common to have daily measurements of some KPI. A very basic approach is to take the monthly medians and look at the trend over time. I also used methods from statistical process control. The latter requires normal distributed data so I used the box cox transformation but I would also like to look at the trend of the data as moving median. Not sure, if this makes more sense?
Comment: You've been here awhile, so it's time to read our help to learn how SE sites work.  A closed question is not "shot down:" it is merely in suspension awaiting edits for clarification, at which point users can vote to reopen it.  This prevents people from wasting their time answering the question they *thought* was being asked only to discover later that a different question was intended.
Comment: @whuber I edited my question, which conveys the issue/intend.
Comment: You seem to be trying to ask two or three questions at once.  Could you explicitly connect SPCCs, the running median, the idea of averaging values, and bootstrapping for us so we can see how they are describing a coherent issue?  What is your objective here?
Comment: The overall issue is to detect any out of control process changes. This can (IMHO) be achieved with these 3 suggestions/approaches but I would really appreciate practitioners input.
Answer: Suppose the running median was intended to be the median of 7 values, observed at noon every day of the past week. With irregular observations, one alternative is to take the median of 168 values over all the hours of the past week. The question is then how to fill in the values at the many unobserved hours.
Depending on the KPI and the measurement procedure, you might fill in the value for an unmeasured hour with:

the last measurement before the unmeasured hour
the first measurement after the unmeasured hour
a linear interpolation between the above two measurements
any of the above adjusted for the time of day, e.g. adding the expected difference between noontime and nighttime temperatures to any nighttime temperature observation
a value from some hourly model fitted to the observed data

If you actually get data once a day at noon as intended, then the first four of these procedures will all give the same value that you would have gotten from the median of the seven daily measurements.
None of this refers to SPCCs, transformations, or an assumption of normality, but it may be enough for a transparent demonstration of trends.
Answer: It is hard to gauge how any method would work without even seeing sample data, but the possibilities include

summarizing the last so many values, by a median, or mean, or more generally a trimmed mean

(in this case, consider downweighting according to time elapsed)

summarizing each day, ditto, and just leaving blank days without measurements.

Whatever you do, graphing data and results is essential. (Fine if that is obvious.)
There is no assumption of normality underlying descriptive or exploratory smoothing, although a skewed conditional distribution will affect means more than medians. Trimmed means appear a little non-standard in this territory, but they allow tunable compromise between mean and median.
Box-Cox I personally consider oversold, although not by its original authors (no relation): in the majority of cases, there are two leading possibilities, leaving the data as they come and working on a logarithmic scale.
It is hard to know how you should think about outliers: in a business context, very high or very low values might tell you about past events you know about that aren't part of your decision-making, or they might be important detail you shouldn't discount.
Comment: @Nick - thanks Nick and sorry for taking so long before coming back to you. You mentioning "outliers" being interesting. I guess using a trimmed mean would be contradictory then?
Comment: Trimmed means tell you about the general level of the  _middle_ of your data, where _middle_ is defined by how much you trim. Calculating a trimmed mean is entirely consistent with recognising outliers: indeed it is motivated by realising that those outliers pull a plain mean towards them, which may not be desirable.
Comment: I think some weighting is probably essential given the circumstances described; downweighting by time elapsed would correspond to the first or second option in my answer.


jQuery dynamically output variables contents

Question: I will trim this down to make it as simple as possible. I have a variable named car and another one named car_output. Let's say value = 'car' and car_output = 'I like to drive my car'
So, I'm trying to get the contents of car_output placed inside a div div#car:
$('#'+value).html(value+'_output');

This shows within the div div#car the actual word "car_output" and not what the variable's contents are (I like to drive my car).
How can I get the variable's contents to be placed inside div#car?
`
Comment: you can't do that.... what you can do is instead of creating variables create a key value pair and use bracket notation to fetch dynamic keys
Answer: To access a "dynamic" variable in javascript you can access it from the window object, like this:
val = "test";
test_output = "Testing completed";

out = window[val + "_output"];
alert(out);

this will echo "Testing completed"
Comment: and the fiddle to test ==> http://jsfiddle.net/EF59p/
Answer: this does it
$( '#' + value ).html( eval( value + '_output' ) );

Sample
var value      = 'car';
var car_output = 'I like to drive my car';

$( '#' + value ).html( eval( value + '_output' ) );

you just have to eval() your code before
see JSFIDDLE
Comment: Depending on how value is populated runing eval could be quite dangerous.


order by and rownum in inner query

Question: I've 2 linked tables A an B 1 to n
For all records of A I'd like to have the record of B matching the following conditions

A.dep=B.dep
There is no record in B having the same year of A
I want the maching record of B with the year beeing the latest year before the year of A
If There is more than one record take the one with the latest updateDate
If There is more than one record take the one with the latest creationDate
If There is more than one record take the one with the latest Id

I tried the Following
Select ...
from A
     ...
     left join B same on A.dep=same.dep and A.year=same.year
     left join B last on A.dep=last.dep and A.year>last.year
where ...
      and same.id is null
      and (last.id is null or
           last.id = (select id from B where dep=A.dep and rownum=1 order by year desc, updateDate desc, creationDate desc, id desc))

But I've an error :
00907. 00000 -  "missing right parenthesis"

But all the parenthesis are ok.
Looks like the inner select doesn't support the order by
Any idea?
Comment: if you use aliases, that you also should it use  in your conditions  e.g. `A.dep=same.dep` instead of `A.dep=B.dep`
Comment: Yes thanks it's a typo
Answer: It is ORDER BY in a subquery.
If you run it in SQL*Plus (or some tool that shows error position), you'd get
SQL> select *
  2  from dept d
  3  where 1 = 1
  4    and d.deptno = (select e.deptno from emp e where e.ename = 'KING' order by e.sal);
  and d.deptno = (select e.deptno from emp e where e.ename = 'KING' order by e.sal)
                                                                    *
ERROR at line 4:
ORA-00907: missing right parenthesis

So:
SQL> select *
  2  from dept d
  3  where 1 = 1
  4    and d.deptno = (select e.deptno from emp e where e.ename = 'KING');

    DEPTNO DNAME                LOC
---------- -------------------- --------------------
        10 ACCOUNTING           NEW YORK

SQL>

Comment: How to fix what? Remove order by, it is irrelevant in this context.
Comment: Irrelevant in your query but in mine I'm using rownum=1 to retrieve the more recent record
Comment: Use ROWNUM or RANK analytic function, not ORDER BY + ROWNUM.
Answer: Indeed, inner sub-query does not support order by because it will give a partial resultset which will be manipulated further by the outer query.
Use order by to sort the result based on your requirements once the resultset corresponding to other filter conditions is obtained.
So, in your case, get the query to match your filter conditions first and then apply order by based on the requirements. But, if you do want the inner sub-query results to be ordered first before applying it to outer/main query, use a join. Create a resultset based on the inner sub-query first and join it with an alias to the other/main query results with appropriate join conditions.
Answer: Assuming b.id is unique, you can do:
select a.*, b.*
from a left join
     b
     on b.dep = a.dep and
        b.year <= a.year and
        b.id = (select max(b2.id) keep (dense_rank first order by b2.year desc, b2.updatedate desc, b2.creationdate desc, b2.id desc)
                from b b2
                where b2.dep = b.dep
               );

order by and rownum in inner query


How to set a checkbox using POST request with Services Module?

Question: I am trying to set a checkbox in a display using a post request by setting the value of the mark field to 1, specifically field_mark_as_discounted. But I can't seem to be able to achieve that. Note: The checkbox functions well when set manually. If anyone has come across this issue of setting a checkbox using post request please share.
Comment: long shot, but try setting it to `true`
Comment: tested it before; didn't work
Comment: one would think `1` should work, does your code work on other non boolean fields?
Comment: Are you **creating** new nodes, or are you **updating**?
Comment: yes the code works on all fields except the boolean and I am creating a new node, specifically a product display, which has the field.
Comment: is your field required or not? see my question http://drupal.stackexchange.com/questions/204946/updating-non-required-fields-via-services-json
Answer: This probably is the JSON you need for that field when POSTing:
<code>field_foo: { und: [ { value: 1 } ] }
</code>
Remember, the Services module is dependent on which widget you have selected on the Manage Fields page for your entity/bundle.
Comment: This is the same format that I have followed before with this field; moreover, I've checked the db and the field is saved correctly with same value as the node whose checkbox was manually clicked except in my case the node created using the post request does not have the checkbox clicked.


AsyncTask Android - Design Pattern and Return Values

Question: I'm writing an application that validates login credentials on an external webserver - so I have the basic issue of creating a login screen that when submitted will send an HTTP request to a server in the background and not cause the UI to hang - whilst providing a ProgressDialog to the user.
My problem lies in, I want to write a generic HTTP Request class that extends AsyncTask, so when I call .execute() I will then pass String parameters which may contain something like 'post', and when doInBackground is called this will see the 'post' string and then forward those parameters onto the respective call in my class.  Pseudo code would be something like
public class HTTPOperations extends AsyncTask<String, Void, String>
{
doInBackground(String... string1,additionalParams)
{
  if string1.equals "post"
      response = httpPost(additionalParams)
       return response;
}

httpPost(params)
{
// do http post request
}
}

This is all I could think of, other than creating a class for every HTTP Post/GET etc request I wish to make and extending ASyncTask...
Which leads me to my next problem, if the HTTP POST is successful and it returns an authentication token, how do I access this token?
Because new httpOperations.execute(), does not return the string from doInBackground, but a value of type
Sorry if this doesn't make sense, I can't figure this out at all.  Please ask for elaboration if you need it.  AsyncTask design patterns and ideas are hugely welcomed.
Answer: If you are designing a reusable task for something like this, you need to identify a reusable return type. Its a design decision on your part. Ask yourself, "Are my HTTP operations similar in both the mechanisms with which they are called and in which their data is processed?" If so, you can design a single class to do both. If not, you probably need different classes for your different remote operations.
In my personal use, I have an object i attach key value pairs to and the common return type is the HttpEntity. This is the return type for both HTTP Get and Post, and this seems to work ok in my scenarios because i throw exceptions in exceptional HTTP result situations, like 404.  Another nice aspect of this setup is that the code to attach parameters to a get or post are fairly similar, so this logic is pretty easy to construct.

An example would be something like this (psuedo):
public interface DownloadCallback {
   void onSuccess(String downloadedString);
   void onFailure(Exception exception);
}

Then in your code, where you go to do the download:
DownloadCallback dc = new DownloadCallback(){
   public void onSuccess(String downloadedString){
     Log.d("TEST", "Downloaded the string: "+ downloadedString);
   }
   public void onFailure(Exception e){
     Log.d("TEST", "Download had a serious failure: "+ e.getMessage());
   }
 }

 DownloadAsyncTask dlTask = new DownloadAsyncTask(dc);

Then inside the constructor of DownloadAsyncTask, store the DownloadCallback and, when the download is complete or fails, call the method on the download callback that corresponds to the event.  So...
public class DownloadAsyncTask extends AsyncTask <X, Y, Z>(){
  DownloadCallback dc = null;

  DownloadAsyncTask(DownloadCallback dc){
    this.dc = dc;
  }

  ... other stuff ...

  protected void onPostExecute(String string){
    dc.onSuccess(string);
  }
}

I'm going to reiterate that I think for the good of yourself, you should pass back HttpEntities.  String may seem like a good idea now, but it really leads to trouble later when you want to do more sophisticated logic behind your http calls.  Of course, thats up to you.  Hopefully this helps.
Comment: Thanks for the quick response.  The common return type would be a string, all of my calls I either wish to receive a response that will contain XML data, or a simple status code which I can then parse.  How do I go about doing this with the AsyncTask, I still don't quite understand where I get the return values from when using the situation described above.
Comment: In your async task, you override the `onPostExecute( result)` method.  AsyncTask automatically invokes onPostExecute when the doInBackground completes and passes it the value returned by doInBackground.  To 'get it out' of the AsyncTask, I'd recommend creating a callback interface.  This interface type would be passed into your asynctask constructor.  Then, in onPostExecute, you'd call the interfaces method.  This would allow you to build conditional logic into handling the results of your http downloads.
Comment: Thanks again. I'm unfamiliar with using an interface in a real example, I have never understood the concept fully.  Could you paraphrase what you mean?  Maybe with some pseudocode.  As I'm unsure how I actually access the constructor in my HTTPOperations AsyncTask as is it called by HTTPOperations().execute(strings...)
Comment: It may also be worth noting at the moment I am attempting to use my HTTPOperations class as a Singleton class...
Comment: I edited my response.  Singleton or not, you need to consider your mechanism for handling responses.  Callbacks are one way.  You could certainly do this with a router-type response handler but i've seen that get pretty messy and i don't have a simple example to share.
Comment: Thanks for that.  So when a I pass an interface into the constructor and then call a method in that interface, will that method then return where the method was defined?  For example instead of your Log.D code, I stored the string in a variable, could this variable then be accessed from the UI thread?  In effect 'getting the response out of' the AsyncTask.  Sorry for all the follow up questions, I just would love to get this mechanism correct.
Comment: where I defined the log.d call, you'd do something like runOnUIThread() to run the result on the ui thread.  Think of the logic you put in there as being the response code and there is, intentionally, some insulation between that and the logic of the class which made the call to your asynctask.
Comment: Thanks for all your help, great answer, I'll get on that
Comment: One last question, I've noticed you create an object of AsyncTask class and pass the interface to the constructor - on all the Goolg edeveloper examples, they always call an AsyncTask by doing TaskName().execute(params) instead... could you explain this?
Comment: Lets take [this example.](http://developer.android.com/reference/android/os/AsyncTask.html). The basic difference between what you want to do and what they are doing is that their example does the same thing on every onPostExecute().  The callback mechanism allows you to leave the majority of the AsyncTask the same and pass a callback object in on the constructor allowing you to have some control over what gets done as part of the result of the task.  In reality, this is only necessary because you are using AsyncTask.
Answer: suppose the data format with web api is json, my design pattern :
common classes
1.MyAsyncTask : extends AsyncTask
2.BackgroundBase : parameters to server
3.API_Base : parameters from server
4.MyTaskCompleted : callback interface
public class MyAsyncTask<BackgroundClass extends BackgroundBase,APIClass extends API_Base> extends AsyncTask<BackgroundClass, Void, APIClass> {
    private ProgressDialog pd ;
    private MyTaskCompleted listener;
    private Context cxt;
    private Class<APIClass> resultType;
    private String url;
    private int requestCode;

    public MyAsyncTask(MyTaskCompleted listener, Class<APIClass> resultType, int requestCode, String url){
        this.listener = listener;
        this.cxt = (Context)listener;
        this.requestCode = requestCode;
        this.resultType = resultType;
        this.url = url;
    }
    public MyAsyncTask(MyTaskCompleted listener, Class<APIClass> resultType, int requestCode, String url, ProgressDialog pd){
            this(listener, resultType, requestCode, url);
            this.pd = pd;
            this.pd.show();
    }

    @Override
    protected APIClass doInBackground(BackgroundClass... params) {
        APIClass result = null;
        try {
            //do something with url and params, and get data from WebServer api
            BackgroundClass oParams = params[0];
            String sUrl = url + "?d=" + URLEncoder.encode(oParams.getJSON(), "UTF-8");
            String source = "{\"RtnCode\":1, \"ResultA\":\"result aaa\", \"ResultB\":\"result bbb\"}";

            //to see progressdialog
            Thread.sleep(2000);

            result = new com.google.gson.Gson().fromJson(source, resultType);
        } catch (Exception e) {
            e.printStackTrace();
        }

        return result;
    }

     @Override
     protected void onPostExecute(APIClass result) {
        super.onPostExecute(result);

        try {
            if(pd != null && pd.isShowing())
                pd.dismiss();

            API_Base oApi_Base = (API_Base)result;
            listener.onMyTaskCompleted(result , this.requestCode);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}
public class API_Base {
    public int RtnCode;

    public String getJSON(Context context) throws Exception
    {
        return new com.google.gson.Gson().toJson(this);
    }

    public String toString(){
        StringBuilder sb = new StringBuilder();

        for (Field field : this.getClass().getFields()) {
            try {
                field.setAccessible(true);
                Object value = field.get(this);
                if (value != null) {
                    sb.append(String.format("%s = %s\n", field.getName(), value));
                }
            } catch (Exception e) {
                // TODO: handle exception
                e.printStackTrace();
            }

        }

        return sb.toString();
    }

}
public class BackgroundBase {

    public String getJSON() throws Exception
    {
        return new com.google.gson.Gson().toJson(this);
    }

}
public interface MyTaskCompleted {
    void onMyTaskCompleted(API_Base oApi_Base, int requestCode) ;
}

example, let's call two api in one activity
assume :
API 1.http://www.google.com/action/a
input params : ActionA
output params : RtnCode, ResultA
API 2.http://www.google.com/action/b
input params : ActionB
output params : RtnCode, ResultB
classes with example :
1.MyActivity : extends Activity and implements MyTaskCompleted
2.MyConfig : utility class, i set requestCode here
3.BackgroundActionA, BackgroundActionB : model classes for api's input params
4.API_ActionA, API_ActionB : model classes for api's output params
public class MyActivity extends Activity implements MyTaskCompleted {
    ProgressDialog pd;
    Button btnActionA, btnActionB;
    TextView txtResult;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        // TODO Auto-generated method stub
        super.onCreate(savedInstanceState);
        setContentView(R.layout.my_layout);
        btnActionA = (Button)findViewById(R.id.btn_actionA);
        btnActionB = (Button)findViewById(R.id.btn_actionB);
        txtResult = (TextView)findViewById(R.id.txt_result);

        btnActionA.setOnClickListener(listener_ActionA);
        btnActionB.setOnClickListener(listener_ActionB);

        pd = new ProgressDialog(MyActivity.this);
        pd.setTitle("Title");
        pd.setMessage("Loading");
    }

    Button.OnClickListener listener_ActionA = new Button.OnClickListener(){

        @Override
        public void onClick(View v) {
            //without ProgressDialog
            BackgroundActionA oBackgroundActionA = new BackgroundActionA("AAA");
            new MyAsyncTask<BackgroundActionA, API_ActionA>(MyActivity.this,
                                                            API_ActionA.class,
                                                            MyConfig.RequestCode_actionA,
                                                            "http://www.google.com/action/a").execute(oBackgroundActionA);
        }

    };
    Button.OnClickListener listener_ActionB = new Button.OnClickListener(){

        @Override
        public void onClick(View v) {
            //has ProgressDialog
            BackgroundActionB oBackgroundActionB = new BackgroundActionB("BBB");
            new MyAsyncTask<BackgroundActionB, API_ActionB>(MyActivity.this,
                                                            API_ActionB.class,
                                                            MyConfig.RequestCode_actionB,
                                                            "http://www.google.com/action/b",
                                                            MyActivity.this.pd).execute(oBackgroundActionB);
        }

    };

    @Override
    public void onMyTaskCompleted(API_Base oApi_Base, int requestCode) {
        // TODO Auto-generated method stub
        if(requestCode == MyConfig.RequestCode_actionA){
            API_ActionA oAPI_ActionA = (API_ActionA)oApi_Base;
            txtResult.setText(oAPI_ActionA.toString());

        }else if(requestCode == MyConfig.RequestCode_actionB){
            API_ActionB oAPI_ActionB = (API_ActionB)oApi_Base;
            txtResult.setText(oAPI_ActionB.toString());

        }

    }

}
public class MyConfig {
    public static String LogTag = "henrytest";

    public static int RequestCode_actionA = 1001;
    public static int RequestCode_actionB = 1002;
}
public class BackgroundActionA extends BackgroundBase {
    public String ActionA ;

    public BackgroundActionA(String actionA){
        this.ActionA = actionA;
    }

}
public class BackgroundActionB extends BackgroundBase {
    public String ActionB;

    public BackgroundActionB(String actionB){
        this.ActionB = actionB;
    }
}
public class API_ActionA extends API_Base {
    public String ResultA;
}
public class API_ActionB extends API_Base {
    public String ResultB;
}

Advantage with this design pattern :
1.one Advantage for multi api
2.just add model classes for new api, ex: BackgroundActionA and API_ActionA
3.determine which API by different requestCode in callback function : onMyTaskCompleted


Continuity of the location

Question: Consider $f(x,\theta),$ a density function for the random variable $X$ with a parameter $\theta$. Suppose $f(x, \theta)$ is continuous in $\theta$.
Is the location
$$\int_\mathbb{R} x f(x, \theta) dx$$
continuous in $\theta$?
Answer: The expectation does not have to be a continuous function of the parameter.  The following example illustrates what can go wrong.

For any number $0 \le p \le 1,$ let $g(x,p)=0$ when $x \le 1$ and otherwise
$$g(x,p) = (p+1)x^{-(p+2)} = \frac{\mathrm{d}}{\mathrm{d}x}\left(1 - x^{-(p+1)} \right) =  \frac{\mathrm{d}}{\mathrm{d}x}G(x,p).$$
This describes a family of positive distributions with distribution functions $x\to G(x,p)$ (Pareto distributions).  Moreover, for any fixed real number $x$, the function $p\to g(x,p)$ is continuous (it's actually differentiable).
Using these, construct a family by mixing two such distributions,
$$f(x,\theta) = \frac{1 - \theta}{2} g(-x,\theta^2) + \frac{1+\theta}{2} g(x,\theta^2),$$
where $-1 \le \theta \le 1.$  Because $g$ is continuous in its second variable, the functions $\theta\to f(x,\theta)$ are all continuous (actually, differentiable).

The parameter $\theta$ plays two roles in this family: as it grows closer to $0,$ it increases the heaviness of the tails, causing the absolute value of the expectation to increase; and it also determines the amounts of negative and positive parts in the mixture, causing the expectation to move from negative to positive as $\theta$ crosses $0.$
Their expectations are
$$\begin{aligned}
e(\theta) &= \int_{\mathbb R} x f(x,\theta)\,\mathrm{d}x\\
&= \frac{1 - \theta}{2}\int_{-\infty}^0 x g(-x,\theta^2)\,\mathrm{d}x  + \frac{1 + \theta}{2}\int_0^\infty x g(x,\theta^2)\,\mathrm{d}x\\
&=  -\frac{1 - \theta}{2}\int_0^\infty x g(x,\theta^2)\,\mathrm{d}x  + \frac{1 + \theta}{2}\int_0^\infty x g(x,\theta^2)\,\mathrm{d}x\\
&=\theta \int_0^\infty x g(x,\theta^2)\,\mathrm{d}x\\
&= \theta \int_1^\infty x (\theta^2+1)x^{-(\theta^2 + 2)}\,\mathrm{d}x\\
&= \theta\left(1 + \frac{1}{\theta^2}\right) = \theta + \frac{1}{\theta},
\end{aligned}$$
provided $\theta\ne 0.$  When $\theta = 0,$ the expectation is undefined.
Clearly, as $\theta$ increases from just below $0$ to just above $0,$ there is no way to define $e(0)$ to make this a continuous function.

If you would like an example where all the density functions are continuous in $x$ everywhere (which these are not: they are discontinuous at $\pm 1$), then convolve these with (say) a standard Normal distribution.  (These will be the densities of $X+Z$ where $Z$ is an independent standard Normal variable.) That adds $0$ to all the expectations while guaranteeing all the densities are infinitely differentiable.  This is what some of them look like:
Comment: Since the expectation is not defined at $\theta=0$, isn't the function $e(\theta)$ technically continuous on its domain? Definitely an interesting example though.
Comment: +1 interesting, I would have expected the opposite. I am bit rusty with my math, could you comment on what goes wrong in the following "proof"?
Start by the usual definition of pointwise continutiy, use linearity of the integral to come to $$\lim_{h \to 0} \left(\int_\mathbb{R} x \cdot \frac{f(x, \theta + h) - f(x, \theta)}{h} dx \right)$$
So the question boils down to whether we can exchange the limit with the integral. I thought initially yes, we can use e.g. dominated convergence to prove that in a neighborhood of 0, $$\frac{f(x, \theta + h) - f(x, \theta)}{h}$$ ...
Comment: ... (which must be close to $f(x, \theta) $) is less or equal to $k \cdot f(x, \theta)$ for some $k > 1$, and that would show that the function is dominated by a function in $L_1$, hence we apply lebesgue and we swap integral and limit and we're done. I guess this is incorrect and we cannot apply lebesgue's theorem here. Why not? :)
Comment: @Jacob It is more constructive either (a) to recognize the domain of $\theta$ is indeed $(-1,1)$ and address the question of nonexistence by asking whether this is a "removable singularity:" that is, whether one can redefine $e(0)$ to make the function continuous; or (b) to broaden the definition of the integral to permit computing $e(0)$ (which will be $0$); or (c) truncating the tails of these distributions so that $e(\theta)$ is always finite and defined: it will still have a discontinuity at $0.$ In short, you can't rescue the situation by restricting the domain.

Continuity of the location