Clustering samples from non-stationary multivariate distributions

Question: I have dataset consisting of samples from a n-multivariate probability distribution, i.e each sample is a n-dimensional vector. The data source is known to be non-stationary in nature. I have been looking for ways to cluster the samples into subsets such that each subset represents at least a covariance or weakly stationary distribution. What are some effective ways of doing so?
Essentially all of the clustering methods I've come across thus far are based on geometry instead of statistical basis.
Comment: Then you must have overlooked EM clustering...
Comment: Statistics requires assumptions. You can do EM with other distributions as well, it's just that nobody seems to use that in practise. Either way it is not "based on geometry". And in fact, **EM uses covariances**.
Comment: Correct me if I'm wrong but EM assumes a mixture of gaussian. I prefer not to make such assumptions.
Answer: Partially answered in comments:
Then you must have overlooked EM clustering... – Anony-Mousse
Correct me if I'm wrong but EM assumes a mixture of gaussian. I prefer not to make such assumptions. – gamerx
Statistics requires assumptions. You can do EM with other distributions as well, it's just that nobody seems to use that in practise. Either way it is not "based on geometry". And in fact, EM uses covariances. – Anony-Mousse


Replcae inner space in Python

Question: Iam new to Python, And I need to remove space between string and a digit only not between two strings.
eg:
Input : Paragraph 25 is in documents and paragraph number in another file.
Output : Paragraph25 is in documents and paragraph number in another file.
How this can be done in Python ? I tried regex
<code>re.sub("paragraph\s[a-z]", "paragraph[a-z]", Input)
</code>
But its not working.
Comment: Regexs are your friends
Comment: It helps if you post what you have tried so far.
Comment: What do you mean by "it's not working"?
Answer: <code> >>> re.sub(r'\s+(\d+)', r'\1', 'Program 25 is fun')
'Program25 is fun'
</code>
That might work in a pinch. I'm not the most familiar with regexes, so hopefully someone who is can chime in with something more robust.
Basically we match on whitespace succeeded by numbers and remove it.

Replcae inner space in Python


What kind of isometry is a composition of a glide reflection with itself? Justify

Question: Is there a simple algebraic proof?
Thanks!
Comment: Well, if you compose a glide reflection with itself, what you get is a translation - which is an isometry.

It's not hard to see this on euclidean spaces; even easier in the plane. But to do this with an algebraic proof?

Try do proof this using only geometric arguments; if you have a glide reflection then it has a line in which it reflects; this line must divide the plane in two components, in such a way that when you apply the reflection again, the original geometric object which is being reflected is in the original "side" of the plane, but translated.
Comment: yeah i checked geometrically, i am trying to solve it algebraically as well by using the fact that
Mk composed with Tab = Tab composed with Mk
where AB is parallel to the line of reflection k. Tab is a translation and Mk is a reflection across line k AND Mk composed with Tab is glide reflection.
Answer: Let the glide reflection be $T$. Without loss of generality the reflection part of $T$ is reflection in the $y$-axis. Assume that the translation part is by the vector $(a,b)$.
The reflection takes $(x,y)$ to $(-x,y)$. The translation part now takes us to $(-x+a,y+b)$. So $T$ takes $(x,y)$ to $(-x+a,y+b)$.
Do it again. The reflection part takes $(-x+a,y+b)$ to $(x-a,y+b)$, and the translation part takes this to $(x,y+2b)$. So $T^2$ is translation parallel to the $y$-axis by $2b$.
Comment: this is more or less correct if I understand your idea correctly(unless you missed the word 'GLIDE REFLECTION') but I need to prove that composition of glide reflection with itself is a translation NOT that two reflections make a translation (that is indeed a very basic result)
Comment: I was being sloppy, had vectors in mind but did not saay so. Have rewritten. A glide reflection is a reflection followed by a translation. The algebra above is that if $T$ is a glide reflection, then $T^2$ is a translation parallel to the axis of reflection.
Comment: great i see now, thanks!


Handling of Null Values

Question: I had an bizarre situation while debugging my code today.
As the screen shot says, I had an object obj yields a value from an System.Data.OracleClient ExecuteScalar().
The yielded value was assigned to variable obj which clearly shows its a null, but the execution had gone inside a if (obj != null) statement which technically should not. Anything idea? I wonder I did the code correct?
Answer: Try using DBNull.Value instead
if(obj != DBNull.Value)

Comment: Is obj really DBNull.Value? In the screenshot I see a IsNull and Value property.
Comment: @Dirk To clear that up: if a value is truly `null` then it has no preperty at all. It would only display `null` as the value of `obj` in a single line.
Comment: This is what the confusion was
Comment: @DrCopyPaste And DBNull.Value doesn't have a IsNull or Value property. Actually my answer below answers the question about the return type.
Answer: DBNull.Value can be used,
null is not an instance of any type. Rather, it is an invalid reference.
However, System.DbNull.Value, is a valid reference to an instance of System.DbNull (System.DbNull is a singleton and System.DbNull.Value gives you a reference to the single instance of that class) that represents nonexistent* values in the database.
Answer: The documentation of ExecuteOracleScalar says that it "executes the query, and returns the first column of the first row in the result set returned by the query as an Oracle-specific data type."
From what I can tell the result could be an OracleNumber for example, which has the properties IsNull and Value as your screenshot shows.
Comment: you explanation is correct. I ran the SQL, which returned no rows, technically a null value. The SQL was supposed to return an `OracleNumber` value, but `ExecuteOracleScalar()` returned an OracleNumber type with its Value attribute as `null` since there were no rows.

Handling of Null Values


Windows 7 Updates installed in registry

Question: Does any one know where the information about Windows 7 Updates is stored in the Registry because its no longer in:
HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Uninstall
Answer: If you're just looking for a list of installed updates, you can use the "systeminfo" command at the command line, the list is contained within the "Hotfixe(s)" section.
Comment: Actually no. Im trying to write a software. That works for previous version of windows just not 7 (Havent Tried Vista)
Comment: I would +1 but req 15 rep. Cause i didnt know about the systeminfo command :D
Answer: Check HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\Packages.
To list them in a cmd window, open an elevated command prompt and paste in:
REG QUERY "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Component Based Servicing\Packages" /s /v InstallClient /t REG_SZ /f WindowsUpdateAgent | grep -oPh "KB[0-9]{4,8}" | sort | uniq
This isn't an ideal way of retrieving the installed updates, since it relies on UNIX tools being installed to filter in the KBs, and sort & dedupe the list. But if you have Github Desktop or Git SCM installed you probably have those tools in your PATH already.

Windows 7 Updates installed in registry


Understanding the Importance of "Sufficiency" within Statistics

Question: I am trying to better understand what it means to be a "sufficient statistic".
"In statistics, a statistic is sufficient with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the parameter" . "
Source: https://en.wikipedia.org/wiki/Sufficient_statistic
Based on this definition, here is how I conceptualize it:
In a unimodal distribution, the "mean" provides more information about the unimodal distribution - relative to the amount of information provided by the "mean" in a biomodal distribution. Thus in this case, the "mean" is more of a sufficient statistic in a unimodal distribution compared to a biomodal distribution.
For example, in a distribution of basketball player heights that is unimodal - the "mean" would well describe the average height of a basketball player. But if you were to have a distribution of heights corresponding to penguins, ostriches and giraffes - and this distribution was not unimodal, it is more likely that the "mean" would not sufficiently characterize the height of the average measurement in this sample.
Is this interpretation correct?
Comment: Sufficiency has a very clear definition in terms of Fisher information. It also very rarely takes place, the "very rarely" being wrt the range of statistical models.
Comment: Does this answer your question? [Sufficient statistic, specifics/intuition problems](https://stats.stackexchange.com/questions/81993/sufficient-statistic-specifics-intuition-problems)
Answer: The term 'sufficient statistic' is pretty theoretical (i.e., it would not typically be taught in applied stats classes) and I am not sure you are hoping for a theoretical answer.
But, what you have above (about normal and binomial) is a bit off. Although it is a theoretical term, the general idea of it is not that complicated (actually way simpler probably than you are thinking about it).
This is the nutshell example (with small loss of preciseness to make the explanation simple). If your goal is to estimate the mean of a population for any distribution, and you have a random sample $y_1,y_2,\ldots,y_n$ from that distribution, then a (one of many) sufficient statistic for estimating the population mean the sum of the $n$ numbers and the sample size $n$. All that means is that if you know those two numbers (the sum + sample size) then you do not need to know any other information about the $n$ numbers to estimate the population mean.
As a second example, if your goal is to estimate the variance of any population (= distribution), then if you know the values of $n$, the sum of $y_1,y_2,\ldots,y_n$ and also the sum of the squared values $y_1^2,y_2^2,\ldots,y_n^2$ then you do not need to know any other information about the $n$ numbers to estimate the population variance. This is because the sample variance can be written as a function of
$\sum_{i=1}^ny_i$ and $\sum_{i=1}^ny_i^2$ and $n$.
Again, if you are looking for a more theoretical answer, this is not it.


complex numbers -equation

Question: $z,w \in \mathbb{C}, z \neq 0, w \neq 0. $ It's known also that $zw \in \mathbb{R}$.
Prove that $w = a\overline z$ when $a \in \mathbb{R}$.
I tried to solve it as follows:
$z = (a+ib), w=(c+id).$
$zw \in \mathbb{R}$ than:
$zw = (a+ib)(c+id) = ac - bd + i (ad + cb)$
than $ad+cb = 0$.
but how can I continue from here to prove that $w = a\overline z$?
Comment: $$\frac{w}{\bar z}=\frac{zw}{z\bar z}=\frac{zw}{|z|^2}$$
Comment: @mathlove That's an overpowered answer! +1
Comment: @zainpatel Don't worry you got a +1 from me too. I doubt one of you copied the other's answer since it seems the time difference between your posts is of some seconds.
Comment: @mathlove not sure who got there first. :-p
Answer: We have $zw = b$ for some $b \in \mathbb{R}$ hence $$w = \frac{b}{z} = \frac{b\overline{z}}{z\overline{z}} = \frac{b\overline{z}}{|z|^2}$$
Then labelling $a = \frac{b}{|z|^2} \in \mathbb{R}$ we have
$$w = \frac{b\overline{z}}{|z|^2} = a\overline{z}$$ as required.
Comment: In the last part you ment $w = a \overline z$ and not $ zw = a \overline z $ ?
Comment: Yes, I meant $w = a\overline{z}$, corrected, thanks!  :-)
Answer: You are very close.
I will, however, make a slight alteration to your work so that "$a$" does not appear twice in the proof.
Let $z = (p+iq), w=(r+is).$
$zw = (p+iq)(r+is) = pr - qs + i (ps + qr)$
If $zw \in \mathbb{R}$ then $Im(zw)=0 \Rightarrow ps+qr = 0$.
Rearrange to make $ps=-qr \Rightarrow s=-\frac {qr}p$
$w=r+is=r-i\frac {qr}p$
$w=\frac rp (p-iq)=a (p-iq)=a \overline z $
Answer: You can use the exponential expression of complex numbers:
$$z=r_1e^{i\theta_1}\land w=r_2e^{i\theta_2}$$
$$zw=r_1r_2e^{i(\theta_1+\theta_2)}\in\mathbb{R}\Leftrightarrow\theta_1+\theta_2\equiv0\mod{2\pi}$$
Thus: $$\theta_2\equiv-\theta_1\mod{2\pi}$$
Hence $$w=r_2e^{-i\theta_2}=\frac{r_2}{r_1}\bar{z}$$

complex numbers -equation


How to pass `ly:music` to a `markup-command`

Question: I am attempting to create a lilypond markup command which will display a guitar chord fingering above the chord on a stave. The code I have so far follows:
#(define-markup-command (chord_and_fingering layout props
  chord fingering) (ly:music? string?)
  "…documentation…"
  (interpret-markup layout props
    #{ \markup { \center-column {
      \fret-diagram #(string-append "f:1;h:13;" fingering)
      \score {
        \new Staff {
          \clef "treble_8"
          \set fingeringOrientations = #'(left)
          #chord \bar "|."
        }
        \layout {
          \override Staff.TimeSignature #'stencil = ##f
          \override Staff.Clef #'transparent = ##t
        }
      }
    }}#}
))

It works okay, but I cannot figure out how to pass a music expression directly to it. That is, I can use it like this with a temporary variable:
a_C_major_chord = {
  <c-3 e-2 g-0 c'-1 e'-0>1
}

a_C_major = \markup {
  \chord_and_fingering
    #a_C_major_chord
    #"6-x;5-3;4-2;3-o;2-1;1-o;"
}

But would like to use it like this, without:
a_C_major = \markup {
  \chord_and_fingering
    #{<c-3 e-2 g-0 c'-1 e'-0>1}
    #"6-x;5-3;4-2;3-o;2-1;1-o;"
}

This however doesn't work. Could someone tell me where I am going wrong and what is the correct syntax for this?
Thank-you for your time and help.
Answer: Well, you can use either
\notemode {<c-3 e-2 g-0 c'-1 e'-0>1}

or (you were almost there)
##{<c-3 e-2 g-0 c'-1 e'-0>1#}

Remember: # throws you from LilyPond input modes into Scheme and #{ introduces stuff in LilyPond syntax into Scheme.  Here you are combining both and so need ##{ at the start.
Comment: That's great. `\notemode` doesn't seem to work for me here, but the alternative is perfect, there's just a small typo in your answer; the closing `}` in the second example must also be pre-fixed with a `#` so `##{...#}` is the construct. Thanks for the help.


Identify the set of points which satisfy $ |z|=\text{Re}z+1 $

Question: Let $ z=x+iy $
I want to Identify the set of points that satisfy $ |z|=\text{Re}z+1 $.
Here's what I have found:
$ \sqrt{x^{2}+y^{2}}=x+1\thinspace\thinspace\thinspace\iff $
$ x^{2}+y^{2}=\left(x+1\right)^{2}\thinspace\thinspace\thinspace\iff $
$ y^{2}-2x-1=0\thinspace $
Now how can I identify the set of points based on the equation? I tried to use desmos, it looks like hyperbola, but its not. So im not sure...
Thanks in advance
Answer: Geometrically, the equation $|z|=\text{Re}(z)+1 $ indicates that the horizontal distance from $z$ to the vertical line $x=-1$ is equal to its distance to the origin, which defines the parabola with the focus at the origin and the directrix $x=-1$.
Answer: You're correct that the equation is $y^2 = 2x + 1$. This gives us $x = \frac{y^2 - 1}{2}$. This is a parabola because $x$ is a quadratic function of $y$.


Guassian Sum Solving based on Improper Integral

Question: We all know that:$$\displaystyle \int_{-\infty}^{\infty} e^{-x^{2}} \ dx = \sqrt{\pi}$$.
Now if we consider the sum:
$$\mathfrak{A} = \sum_{n=-\infty}^{\infty} 2^{-n^{2}}$$
Then how can we show $\mathfrak{A}$ in the form
$\left( q^{-m}; q^{-k} \right)_{\infty} \left( -q^{-n}; q^{-k} \right)_{\infty}^{2}$?
In which $q, m, n, k \in \mathbb{Z}^{+}$ and $(\cdot; \cdot)_{k}$ is the q-Pochhammer symbol.
Comment: Your sum is just a value of the Jacobi theta function $$\Theta(x)=\sum_{n\in\mathbb{Z}} e^{-n^2 \pi x}.$$ Look at the Wikipedia page http://mathworld.wolfram.com/JacobiThetaFunctions.html.
Answer: Using the Jacobi triple product
$$\sum_{n=-\infty}^{\infty} x^{n^{2}}z^{n} = \prod_{i=1}^{\infty} (1-x^{2n+2})(1+x^{2n+1}z)(1+x^{2n+1}z^{-1})$$
when $|x| < 1$ and $z$ nonzero, we substitute $x = 2^{-1}$ and $z = 1$:
$$\mathfrak{A} = \sum_{n=-\infty}^{\infty} 2^{-n^{2}} = \prod_{i=1}^{\infty} (1-2^{-2i-2})(1+2^{-2i-1})^{2}$$
We can then see that
$$\mathfrak{A} = \prod_{i=0}^{\infty} (1-2^{-2i-4})(1+2^{-2i-3})^{2} = \left( 2^{-4}; 2^{-2} \right)_{\infty} \left( 2^{-3}; 2^{-2} \right)_{\infty}$$


How to log out from google account in android ?

Question: I am developing application that uses login using google Auth2.0 with android account manager. I am login successfully and fetching data from my google using different apis but I don't know about how to log out from my application and when logout want to shows login screen  once again.
Answer: I don't think you can log-out, you will have to show the AccountChooser again
try
Intent intent = AccountPicker.newChooseAccountIntent(null, null, new String[] {"com.google"}, false, null, null, null, null);
 startActivityForResult(intent, SOME_REQUEST_CODE);

Comment: Thanks but what is "SOME_REQUEST_CODE" parameter.please,can you give me e.g
Comment: You can make this up yourself, but try 6789
Answer: Usually i saved the account name on SharedPreferences and on log-out just remove the account name from the SharedPreferences. Saving the account name from onActivityResult
if (resultCode == Activity.RESULT_OK && data != null && data.getExtras() != null) {
                String accountName = data.getExtras().getString(AccountManager.KEY_ACCOUNT_NAME);
                if (accountName != null) {
                    SharedPreferences settings = getPreferences(Context.MODE_PRIVATE);
                    SharedPreferences.Editor editor = settings.edit();
                    editor.putString(PREF_ACCOUNT_NAME, accountName);
                    editor.commit();
                    //do the rest after saving the account name on SharedPreferences
                }
            }

And log out(my log out occur on a different activity):
private void logOut(){
    SharedPreferences sharedPreferences = getSharedPreferences("MainActivity",Context.MODE_PRIVATE);
    if (sharedPreferences.getString(PREF_ACCOUNT_NAME,null)!=null){
        SharedPreferences.Editor editor = sharedPreferences.edit();
        editor.remove(PREF_ACCOUNT_NAME);
        editor.commit();
        //here show the log-in screen again
    }
}

Comment: Hey What is "PREF_ACCOUNT_NAME" parameter
Comment: its just a string variable, like: final String PREF_ACCOUNT_NAME = "accountName";
Answer: While logging in you have saved data i.e. your access token in shared preferenes. So when you want to LogOut clear shared preferences. It is the only way to logout.


Surjectivity of representable sheaves

Question: Let $S$ be a base scheme and let $(Sch/S)_{fppf}$ be a big fppf site. Let $U$ be a scheme over $S$. Suppose that there is a surjective morphism $\Phi_{U}:U\rightarrow U$. Then can we show that the induced morphism of sheaves $h_{U}\rightarrow h_{U}$ is locally surjective? It seems that this is false.
Note that $h_{U}={\rm{Hom}}(-,U)$ is a representable sheaf. A map of sheaves $F\rightarrow G$ on $(Sch/S)_{fppf}$ is locally surjective if for every scheme $U\in{\rm{Ob}}((Sch/S)_{fppf})$ and every $s\in G(U)$, there exists a covering $\{U_{i}\rightarrow U\}_{i\in I}$ such that for all $i$, $s|_{U_{i}}$ is in the image of $F(U_{i})\rightarrow G(U_{i})$.
Answer: Let $S:={\rm Spec}(k)$ be a field and let $U={\rm Spec}(k[t]/t^2)$.
The ring $k[t]/t^2$ is a $k$-algebra and there is a map of $k$-algebras $k[t]/t^2\to t$ sending $t$ to $0$ so we get morphisms $U\to{\rm Spec}(k)$ and ${\rm Spec}(k)\to U$.
Let $a:U\to U$ be the composition of these two morphisms. This is surjective.
Now if the answer to your question is positive, there is a faithfully flat morphism $b:V\to U$ and an element $\rho\in{\rm Hom}(V,U)$ such that $a\circ\rho=b$ (applying local surjectivity to ${\rm Id}_U\in{\rm Hom}(U,U)$). In other words, the morphism $a_V:U\times_U V\to V$ has a section. However, if the morphism $a_V$ has a section then so has the morphism $S\times_U V\to V$ obtained by base-change from the morphism $S={\rm Spec}(k)\to U$. The morphism is $S\times_U V\to V$ is a closed immersion, so it is an isomorphism, since it has a section. Isomorphisms descend along faithfully flat morphisms, so  we deduce that ${\rm Spec}(k)\to U$ is an isomorphism, which is false. So this provides a counterexample.

Surjectivity of representable sheaves


What happens if I use OLS in a multiple regression but the sample is not random?

Question: I know that, to use OLS estimators in linear regressions, there are few assumption to be satisfied. However, it is not clear to me what would happen if I would use OLS in a multiple regression without having a random sample, so that (Xi, Yi) would not be iid. Which sort of problem may I face?
Comment: Your model will be biased
Comment: A good place to look is [this](http://econweb.ucsd.edu/~jhamilto/hp.pdf) paper. Section 6 to end (the first five sections are not that relevant to your question)
Comment: @user603 I think my econometric background is too limitated to understand the contents of that paper. Nothing simpler?
Comment: @jchaykow, how would you define model bias? I know what a biased parameter estimator is, but not quite sure about a biased model.
Comment: @RichardHardy  a model that is trained on biased data will overfit certain non-representative subset of the overall population.
Comment: @jchaykow, *biased data* is yet another notion without a common definition, but let it be. I got your point.
Comment: @RichardHardy I see what you mean, I'm being loose with my language I think.
Answer: First, OLS is nothing more than an algorithm for fitting a linear model of the form
$$
y = \mathbf{X\beta} + \epsilon
$$
In other words, you are positing that the phenomenon $y$ is a linear function of the variables $\mathbf{X}$, plus some additively separable disturbance term.
If this is a good assumption, then there is some true, constant $\mathbf{\beta}$, and you apply some estimator -- such as OLS -- to estimate what it is.
If your sample is non-random -- there is some correlation between your $\mathbf{X}$'s and your error term -- then OLS estimates of $\mathbf{\hat\beta}$ will not be equal in expectation to the true $\mathbf{\beta}$.  This is to say that they are biased.
In other words, if you were to take many many samples from the population of $\mathbf{X}$ and $y$, your average $\mathbf{\hat\beta}$ would not equal $\beta$.
Comment: I think non i.i.d. does not imply dependence (let alone correlation) between $X$ and $\epsilon$, although it does not prohibit it either. E.g. you could have serially correlated errors that are independent of $X$ and still call it non-i.i.d., no?
Comment: Clarify what you mean by nonrandom.  If you mean that you've got repeated observations of individuals, and those individuals are autocorrelated in time, your estimate of beta will still be unbiased, though inference is complicated.  If your sample is nonrandom because some unobserved factor is both correlated with your X's and y, beta will be biased.
Comment: Yeah, that's right.  But the title of the question says "non-random sample."  It is possible that OP has longitudinal data, and just needs to correct the standard errors.  The question is somewhat unclear.
Comment: I will try to explain better.

I would like to know, theoretically, what would happen if I would try to estimate OLS having a non random sample, violating the assumption of iid. I think that if iid is satisfied, then the expected value of β^ would be equal to the real β, otherwise it would be not. But I am not sure
Comment: @bobo55, take a look at the OLS assumptions to make sure you got those correctly. See e.g. [this thread](http://stats.stackexchange.com/questions/16381/what-is-a-complete-list-of-the-usual-assumptions-for-linear-regression/16460#16460), especially the first answer.
Comment: It seems to me that I can say that, if I have a nonrandom sample, OLS estimator would be biased and inconsistent. In particular, it is not true that E[β^] = β.
Comment: Likewise if Your data exhibits heteroskedasticity, your estimates will still be unbiased, just inefficient.
Comment: You're right. Using nonrandom, I don't mean repeated observations but observations that own some characteristics, eg individuals taller than 1.80m.
Answer: When the sample is not random, you have to consider whether the way you got the sample introduced bias. That is, the way data was gathered IRL can affect the extent to which the sample is representative of the population.
For example, say you want to predict who someone is going to vote for based on their media habits. You get the data from asking your friends. The problem is that your friends are probably not going to be representative for the population at large.
Why? One reason could be that we tend to become friends with people who share similar media preferences (maybe you became friends partly because you both love the same youtube channel). Another could be that friends tend to have the same socioeconomic status, and socioeconomic status affects which types of media that are consumed.
In this case, when you do your OLS, your regression coefficient will reflect your friends, but it's very hard to say whether it reflects the population at large. If you're only interested in your friends, that's fine. If you want to generalize, you're in trouble.
According to (Mercer et al., 2017), for non-random samples you basically have to consider whether your non-random sample reflects the population... in terms of confounders.
For example, if all your friends have the same gender as you, sampling your friends is going to be a problem because gender likely affects media habits.
But an all male sample might not be a problem. E.g. if you're testing a new pill for erectile dysfunction, you're probably OK going with an all male sample. Basically, it depends on the theoretical knowledge of your field.
When we start to appeal to the theoretical knowledge of our field, we are moving out of statistics and into the world of causal inference (see e.g. Pearl's Book of Why).
Random sampling (of the population) is a way to not have to deal with any of this. With random sampling (from the population) you can say "I know the way I gathered the data didn't introduce bias, because it was at random".
Randomization protects you from the known, the unknown, and the unknown unknown sources of bias.


Without using mod_rewrite, what is the best way to redirect all subdomains to https?

Question: I am currently using mod_rewrite to redirect all subdomains to https, but multiple sources have suggested that

Using mod_rewrite to do this isn't the recommended behavior. See
  RedirectSSL

The apache docs suggest Redirect and RedirectMatch:

A common use for RewriteRule is to redirect an entire class of URLs.
  For example, all URLs in the /one directory must be redirected to
  http://one.example.com/, or perhaps all http requests must be
  redirected to https.
These situations are better handled by the Redirect directive.
  Remember that Redirect preserves path information. That is to say, a
  redirect for a URL /one will also redirect all URLs under that, such
  as /one/two.html and /one/three/four.html.

Unfortunately, both of these documents (and anything else I have found) are only considering matching a single domain, rather than all subdomains. RedirectMatch seems like a good candidate for a relatively simple task like this, but I can't seem to get "http" to match as I expect:
RedirectMatch permanent "^http://(.*)$" "https://$1"

My guess as to why that doesn't work is because the redirectMatch is looking for a path and not a URI, but at this point I am stuck. Is it possible to accomplish a redirect of all subdomains to https without mod_rewrite, or is that the best available method?
Comment: Are you kidding me with this duplicate flag? The first three words of the title are *without using mod_rewrite*. I literally could not have been more specific about how the mod_rewrite canonical answer is not applicable here.
Answer: Normally, for sub domains of different VHosts, I append the following line to each.
Redirect permanent / https://bar.example.com/

I normally just add one of these for each subdomain. If you have a very large number of sub domains, this method may not be very scalable, but it's simple and gets the job done.
Comment: I'm marking this as correct, as given the extent to which people have ignored the specifics of this question it would seem that there is not a (sans-mod_rewrite) alternative to just listing every subdomain as you suggest. Thank you for actually reading!


Rich notification play video clip

Question: I'm trying to get rich notification and play video on the notification.
I success to show an image and not found swift sample code.
What need to be done to add video clip / mp4 notification support?
which function need to add to the NotificationService class ?
Thanks Yakir
Comment: It looks like https://stackoverflow.com/a/66205763/12783209 already answered your question.
Answer: There is no need to add extra code and no need to use AV player also. we can achieve that by initialising UNNotificationAttachment with url which we get after URLSession.shared.downloadTask
<code>URLSession.shared.downloadTask(with: attachmentURL) { url, _ , _ in
    let attachment = try UNNotificationAttachment(identifier: "identifier", url: url, options: nil)
    content.attachments.append(attachment)
}
</code>
While updating the content in NotificationServiceExtension. Notification will load the video if it is video url.
Note: - please make sure video should be small size or as suggested by Apple.

Rich notification play video clip


Viewing an integral as an expected value

Question: Let $\omega\in (0,1]$ be represented in binary as $\omega=0.d_1(\omega)d_2(\omega)\cdots$ where each $d_i(\omega)$ is either $0$ or $1$ (a tail of zeros is prohibited). We define $r_n(\omega)=2d_n(\omega)-1$ and $s_n(\omega)=\sum_{i=1}^n r_i(\omega)$.
We may use $(r_n)$ to model a simple random walk: Pick a $\omega\in(0,1]$. There is a particle initially at the origin. At the $i$th stage, where $i=1,2,3,\cdots$ the particle moves a unit to the right or left depending on whether $r_i(\omega)$ is $1$ or $-1$ respectively. Clearly, $s_n(\omega)$ represents the position of the particle after $n$ steps.
It is proved in my book that $\int_0^{1}s_n(\omega)d\omega=0$. I understand that proof. After that there is a line:

If the integral is viewed as an expected value, then $\int_0^1s_n(\omega)d\omega=0$ says that the mean position after $n$ steps of a
  random walk is $0$.

I don't understand this line. I interpret $s_n$ as a sum of $n$ independent RVs which have identical two-point uniform distributions. As such $s_n$ is discrete and its expected value should be given by a summation. The book has not mentioned either random variables or expected value till this point. Can someone explain this line?
Answer: Let's consider the case where $X$ is a discrete random variable, eg. it takes its values in $\{1,\ldots,n\}$. There are in general two ways to express the expectation of a random variable of the form $f(X)$, where is $f$ a (deterministic) function, say from $\{1,\ldots,n\}$ to $\{1,\ldots,m\}$:
(1) $\mathbf{E} f(X)=\sum_{y=1}^m y \mathbf{P}[f(X)=y]$
(2) $\mathbf{E} f(X)=\sum_{x=1}^n f(x) \mathbf{P}[X=x]$
(1) is basically the definition of the expectation of a random variable (applied to $f(X)$). (2) follows from (1) by an easy theorem that we sometimes call transfert formula (it's so easy that we usually don't mention it).
Now in your case, the variable $X$ is called $\omega$ and it's a continuous random variable. The formula (1) still makes sense:
$$\mathbf{E} s_n(\omega)=\sum_{i=-n}^n i \mathbf{P}[s_n(\omega)=i]$$
However, to write (2), you now need an integral because the underlying measure is not discrete anymore:
$$\mathbf{E} s_n(\omega)=\int_{0}^1 s_n(t) \,dt$$
Comment: What is the pdf for $\omega$?
Comment: @Shahab: $\omega$ follows a uniform distribution over $(0,1]$, so its pdf is 1 on $(0,1]$ and 0 elsewhere. It makes sense to look at the last equality as $\mathbf{E} s_n(\omega)=\int_0^1 s_n(\omega)\times 1 \,d\omega$
Comment: Honestly it is a bit confusing to use $\omega$ both as a random variable and as an index in the integral. I will change this
Comment: Thanks. I have a small additional doubt. You are essentially saying that $s_n\circ \omega$ is a continuous RV. What is its PDF?
Comment: @Shahab: $s_n \circ \omega$ is *not* continuous (let alone because all its mass is contained in $\mathbf{Z}$), it has no PDF (wrt Lebesgue measure). The "real" formula for $\mathbf{E} s_n \circ \omega$ is the first one (with the sum). But there is no contradiction in the fact that there exists *some* integral formula to compute the expectation of a discrete random variable
Comment: Thanks, understood. BTW what you call the "transfert formula" (is the spelling correct?) is also known as the [law of the unconscious statistician](https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician).


FieldValueSet part underlined while trying to update item in SharePoint Online list

Question: I am trying to update an item from the list via MS Graph.
Tried to follow https://learn.microsoft.com/en-us/graph/api/listitem-update?view=graph-rest-1.0&tabs=cs, but the code below underlines the Color and Quantity and I can't convince it to work.
Screenshot with the issue here
Also tried to follow .NET Graph SDK Updating Sharepoint Online List Item Values, but could not make it to work as well.
Tried with new ListItem as variable too as could be seen in some older cases, but now it just wants the FieldValueSet and would not accept the ListItem.
var fieldValueSet = new FieldValueSet
{
Color = "Fuchsia",
Quantity = 934
};

    await graphClient.Sites["yourtenant.sharepoint.com:/sites/ITOddeleni:"].Lists["TeamsRequest"].Items[item.Id].Fields
.Request()
.UpdateAsync(fieldValueSet);

Answer: Finally, my colleague helped me out to come up with this :-)
var fieldValueSet = new FieldValueSet();
                                fieldValueSet.AdditionalData = new Dictionary<string, object>();
                                fieldValueSet.AdditionalData.Add("firstname.lastname@example.com", "Edm.String");
                                fieldValueSet.AdditionalData.Add("ColumnName", "DesiredValue");

                                await graphServiceClient.Sites["yourtenant.sharepoint.com:/sites/ITOddeleni:"].Lists["TeamsRequest"].Items[item.Id].Fields
                                    .Request()
                                    .UpdateAsync(fieldValueSet);

Comment: This doesn't work though - I'm still getting an error that the ColumnName is not recognized
Answer: The accepted answer is not working anymore.
var fieldValueSet = new FieldValueSet();
fieldValueSet.additionalDataManager().put("Title", new JsonPrimitive("new Title value"));


How do you access the electronics on a hollow-body electric guitars ?

Question: Some models of electric hollow-body / electric arch-top guitars do not have seem to have any kind of access to the electronics (switches, pots, caps, wiring) beside the "f" shaped sound holes.
I have seen guitars with no pickguard on the front of the guitar, and no plastic plate which can removed with screws (like on a Les Paul). For example this guitar : http://members.shaw.ca/jeffflowerday/Images/PRSArchtop.jpg.
How are these guitars electronics parts serviced ?
Comment: Short answer? Very, very carefully XD. If no one else gets to this one before I make it home tonight I'll write it up.
Answer: Pulling out the pickups, and using plastic tubing to hold onto the volume/tone pots as they slide in to the guitar.
For one example, see http://www.youtube.com/watch?v=p29QN4ycHMU (Putting new pickups in a Gretsch Electromatic)
Answer: Through the pickup cavities and the f-hole. I've used a mirror probe and long, skinny tools to install a pickup inside of a parlor guitar; it must be hard as hell to do this work without a full-sized soundhole.
You'd need tools like an inspection mirror, tweezers or skinny pliers, long screwdrivers, and skinny flashlights.
Comment: And the first thing you want to do is write down the lengths of all the wires, because it's a pain in the neck replacing components and fiddling everything back in ... only to find the wire to the tone pot is half an inch too short. (bitter experience!)
Comment: I took my semi-hollow-body to a guitar tech, and he replaced the electronics by fishing them out of the f-holes.  I was kind of impressed, as it's kind of like those guys who build ships in a bottle.
Answer: I have a small piece of 1/8" plywood with holes drilled to match the controls for my H'body. i just have remember to solder everything backwards (mirror image). Spenser forceps help quite a bit too, they aren't just for roaches ;-p
Answer: I solved this exact problem.  I purchased a les Paul backplate, used my dremel tool to cut a hole a quarter inch smaller on all sides, mounted the removed wood from the hole and glued it with gorilla glue to the back of the plate.  I mounted it with screws from an old strat pickguard.   Now I have a hole where I can get to the electronics.  I notice absolutely no sound difference on my Ibanez Artcore AF55 and I can change pickups in minutes.  I also replaced a worn out switch temporarily with a telecaster switch, and I kind of like it and it is still there.


Autoencoder doesn't work (can't learn features)

Question: I am completely new to machine learning and am playing around with the theanets package. What I am currently trying to do is to get an Autoencoder to reproduce a series of Gaussian distributions:
def gen_inputs(x=28, sigma=2.0):
    target = np.array([skimage.filters.gaussian_filter(i, sigma)
                for i in ((np.eye(x*x)*2)).reshape(x, x, x*x).transpose()]).reshape(x*x, x*x)
    return target

I then attempted to use an Autoencoder (28*28, 9, 28*28) to train it. The parameters were as follows:

learning_rate = 0.01
input_noise = 0.01
I used SGD with sigmoid activation function, along with linear output function.

But my network couldn't reproduce the input. It seems to always converge to an average distribution of weights, resulting in random noise-like results. The loss function (MSE) converges as it should. Here are the results:
Comment: Are you trying to repdoduce a gaussian distribution? But there is no structure in a noise. How can use reproduce it? Its a simple GIGO system.
Comment: Also even if there was, to go directly from 784 features to 9 is a huge compression. I would suggest take a subset of the mnist dataset and try less steep dimensionality reduction using greedy layer-wise pretraining.
Comment: Try convolutional autoencoder.
Comment: @yasin.yazici What? How many parameters you need to represent a a bi-dimensional gaussian distribution? Five. Two means, two variances and a covariance.
Answer: (Primary author of theanets here.) As hinted in the comments on your question, this is actually a difficult learning problem! The network is, as indicated by the optimized loss value during training, learning the optimal filters for representing this set of input data as well as it can.
The important thing to think about here is that the weights in the network are being tuned to represent the entire space of inputs, not just one input. My guess is that you're expecting the network to learn one gaussian blob feature, but that's not how this works.
From the network's perspective, it's being asked to represent an input that is sampled from this pool of data arbitrarily. Which pixels in the next sample will be zero? Which ones will be nonzero? The network doesn't know, because the inputs tile the entire pixel space with zero and nonzero pixels. The best representation for a set of data that fills the space uniformly is a bunch of more or less uniformly-distributed small values, which is what you're seeing.
In comparison, try limiting your input data to a subset of the gaussian blobs. Let's put them all in a diagonal stripe of pixels, for instance:
import climate
import matplotlib.pyplot as plt
import numpy as np
import skimage.filters
import theanets

climate.enable_default_logging()

def gen_inputs(x=28, sigma=2.0):
    return np.array([
        skimage.filters.gaussian_filter(i, sigma).astype('f')
        for i in (np.eye(x*x)*2).reshape(x, x, x*x).transpose()
    ]).reshape(x*x, x*x)[10::27]

data = gen_inputs()

plt.imshow(data.mean(axis=0).reshape((28, 28)))
plt.show()

net = theanets.Autoencoder([784, 9, 784])
net.train(data, weight_l2=0.0001)

w = net.find('hid1', 'w').get_value().T
img = np.zeros((3 * 28, 3 * 28), float)
for r in range(3):
    for c in range(3):
        img[r*28:(r+1)*28, c*28:(c+1)*28] = w[r*3+c].reshape((28, 28))
plt.imshow(img)
plt.show()

Here's a plot of the mean data (the first imshow in the code):

And here's a plot of the learned features (the second imshow):

The features are responding to the mean of the entire dataset!
If you want to get the network to learn more "individual" features, it can be pretty tricky. Things you can play with:

Increase the number of hidden units, as suggested in the comments.
Try training with an L1 penalty on the hidden-unit activations (hidden_l1=0.5).
Try forcing the weights themselves to be sparse (weight_l1=0.5).

Good luck!


Prove that if $\gcd(a,b)=1$ then $\gcd(ab,c) = \gcd(a,c) \gcd(b,c)$

Question: Let $a, b, c$ be integers. Prove that if $\gcd(a,b)=1$ then $\gcd(ab,c) = \gcd(a,c) \gcd(b,c)$
First time asking here. I'm not sure what your policies are on general homework help but I truly am stuck.
So far I have shown $\gcd(a,c) \gcd(b,c)$ as an integer combination of $ab$ and $c$. So if I can show that $\gcd(a,c) \gcd(b,c)$ divides $ab$ and $c$ I can use the proof that if an integer $d$ is a common divisor of $a$ and $b$, and $d=ax+by$ for some $x$ and $y$, that $d=\gcd(a,b)$. However I don't really know where to start with this. Any help would be appreciated.
Comment: Welcome to Math.SE! Homework questions are definitely acceptable (and common) here, and you presented yours in the way we like (*i.e.*, showing what you've done so far, where you're having trouble, etc.). You'll usually get answers in the form of hints (like the one I'm writing) for homework questions instead of explicit solutions.
Comment: Someone is flagging my comment, and I don't know why. But I'll expand: in number theory classes, you start by understanding divisibility and gcd rules, and use these to prove the fundamental theorem of arithmetic (unique factorization). But this question is independent of unique factorization, and can be proved with or without it. I din't know which proof to guide the OP towards without knowing whether or not he or she knows the fundamental theorem of arithmetic.
Comment: Nope. I'd prefer a simpler solution versus something requiring advanced knowledge, plus I doubt they'd accept a solution like that anyway.
Comment: Do you know the fundamental theorem of arithmetic yet?
Comment: http://math.stackexchange.com/questions/535961/prove-that-if-gcda-b-1-then-gcda-cdot-b-c-gcda-c-cdot-gcdb-c
Comment: See also
http://math.stackexchange.com/questions/536554/gcdab-c-equals-gcda-c-times-gcd-b-c
http://math.stackexchange.com/questions/535961/prove-that-if-gcda-b-1-then-gcda-cdot-b-c-gcda-c-cdot-gcdb-c
http://math.stackexchange.com/questions/806096/let-a-m-n-in-mathbfn-show-that-if-gcdm-n-1-then-gcda-mn-gcda
http://math.stackexchange.com/questions/846522/if-gcda-b-1-then-gcdab-c-gcda-c-cdot-gcdb-c
Answer: I'm going to write $(m,n)$ for $\text{gcd}(m,n)$ throughout what follows.
You want to show that $(a,c)(b,c) \mid ab, c$. From the definition of $(m,n)$, it's easy to show that $(a,c)(b,c) \mid ab$ (since $(a,c) \mid a$ and $(b,c) \mid b$). So it really boils down to showing that $(a,c)(b,c) \mid c$.
Since $(a,c), (b,c) \mid c$, you can write $c = r(a,c) = s(b,c)$ for some integers $r$ and $s$. Therefore, to show that $(a,c)(b,c) \mid c$, it's enough to show that $(b,c) \mid r = c/(a,c)$. This is equivalent to showing that $p \nmid (a,c)$ for any prime number dividing $(b,c)$. This follows from $a$ and $b$ being coprime. (Why?)
Answer: Imagine the prime factorization of $a$ and the prime factorization of $b$. The primes in $a$ are all different from those in $b$ (because $a$ and $b$ are coprime).
Think about what GCD does. It cherry-picks the common primes of the inputs and multiplies them together. If $a$ and $b$ contain entirely different primes from one another, then separating the GCD into the two GCD's has no effect.
Try to formulate a proof from this.
Good luck
Comment: This is exactly what the OP said he didn't want in response to my comment above
Comment: Oh, I hadn't read your comment. I think the OP was intimidated by the need to go read other references when the idea is actually quite simple. This is not exactly "advanced knowledge" but the OP might have gotten that gist.


What is a good freeware MIDI editor for minor changes running on Windows?

Question: I compose music on my CVP-501. I record the tracks on a MIDI (.mid) file and record them as a wave (.wav) file. However, being human, I make mistakes. Plus, my setup for recording could be a lot better.
 A MIDI editor could fix both of these problems. I could fix the (minor) mistakes and export the file as audio. I realize that MIDI files save the notes and that audio files save the sound, but I have seen enough converters to know it's not impossible to convert between the two.
I am asking for good, free MIDI editors that will run on a Windows machine. I would like an audio conversion feature, but it's not a requirement (like I said, I've seen converters before).
Thanks in advance!
Comment: do you need the source?  I've got a closed source one (windows) on http://pianocheetah.com (my personal site)
Comment: No, not really. I only need it to be free.
Comment: http://midieditor.sourceforge.net/ - not used it before though: http://en.wikipedia.org/wiki/List_of_MIDI_editors_and_sequencers
Answer: LMMS works for window too, and is free for use.
Answer: Try Anvil Studio, it's free and probably has all you need for basic editing, and more.


Class like Struct

Question: I have task here: I must create a class which will be like class Struct. I have just started to learn Ruby and don't know how to create it. In my attempts, I was able to create a method that takes arguments and creates an array. Then I have to go through the array and all of the arguments, make the argument class, in which I can bring some value. But I don't know how to create methods that can create classes in Ruby.
I am asking for help if you have an example or know where to find it, I will be grateful!
my attempts:
<code>class Hobbit

  def new(*params)
    "#{params.inspect}"
  end

end
</code>
Comment: How exactly do you want your clase to behave?
Comment: My task looks "Create the class  which will have the same behavior as 'Struct' class"
Comment: So basically, it's your homework?
Comment: Yes, but, in fact, I do not know how to create a method that will create the classes. I know how to create a class with parameters, etc.
But how to create a class that will create the classes, I do not understand (
Comment: What classes do you want to create ? You should explain in your question how you think you can solve your problem. It is not quite clear for the moment. My advice, the clearlier you can formulate a problem, the easier you can find ways to solve it.
Comment: OK, in short I need to implement on Ruby class Struct. And what i want it is an example of a method that can create classes
Answer: So I assume all you want is

And what i want it is an example of a method that can create classes

Check out this question and the accepted answer which shows you how to create a class dynamically given a class name. Let me know if that's what you're looking for.

Class like Struct


R: How to omit tested warning message from test report when testing for result AND warning

Question: I want to test in R if a function

returns the correct value
throws the correct warning during calculation

For that, I created a reproducible example.
There a two scripts, the first one (e.g. test-warning-and-result.R) works fine and without any errors:
library(testthat)

f <- function(x) {
  if (x < 0) {
    warning("*x* is already negative")
    return(x)
  }
  -x
}

test_that("warning and result", {
  x = f(-1)
  expect_that(x, equals(-1))
  expect_warning(f(-1), "already negative")
})

However, when I run the tests from an external script (e.g. run-test.R), it logically throws a warning at "x = f(-1)"
library(testthat)
test_dir(".")

Picture of test results
Since I know there will be a warning and am testing for it, I'm searching for a way to omit the warning within test_that() from the test report. Ideally, I would not have to run the function twice but in one test.
Any Ideas would be appreciated
Comment: Maybe you can use `options(warn=-1)` to delete warnings in your other script. I think it's a replicate of this post -> https://stackoverflow.com/questions/16194212/how-to-suppress-warnings-globally-in-an-r-script
Comment: Hey, thanks for your answer. I don't want to turn off the warnings globally, since the users need to get some warnings that I throw on specific incidents.
I just don't want to see the warning in the test result, since I'm aware that this warning will be thrown (and even test for it).
Answer: Ok after having one night's sleep on it I found a simple sollution:
Don't store the functions result in a variable x. Nest the two tests into each other, with the expect_warning outside
Change from
test_that("warning and result", {
  x = f(-1)
  expect_that(x, equals(-1))
  expect_warning(f(-1), "already negative")
})

to
test_that("warning and result", {
  expect_warning(expect_that(f(-1), equals(-1)), "already negative")
})


How to solve such equations involving probability

Question: I am trying to understand how the equation (2) is obtained from the equation (1) that involves probability as shown in the following case:
$P_d = \text{Pr}\Bigl(h_d<h_e\Bigr)$  ----(1)
where $\text{Pr}$ denotes the probability, $h_d$ and $h_e$ are independent exponential random variables with variances $\sigma^2_{d}$ and $\sigma^2_{e}$ respectively.
$P_d = \frac{\sigma^2_e}{\sigma^2_e+\sigma^2_d}$  ----(2)
My query is how the equation (2) is obtained from equation (1).
Any help in this regard will be highly appreciated.
Answer: $P_d=\int_0^{\infty} \int_0^{y} \lambda e^{-\lambda x} dx \, \mu e^{-\mu y}dy$ where $\lambda$ and $\mu$ are the parameters of the exponential  distributions of $h_d$ and $h_e$ respectively. Write down $\lambda $ and $\mu$ in terms of the variance and compute this integral.
Comment: Thank you so much for your answer sir...but can you please specify  how the integration limits are obtained....
Comment: If you fix $h_e$ to be $y$ then $h_d$ ranges from $0$ to $y$ since we want $h_d <h_e$.  Basically we condition on $h_e$, write down the probability, multiply by the density of $h_e$ and integrate. @karu
Comment: Thanks a lot sir for such a precise solution and explanation....


Add theme settings field to theme-settings.php in d8 is not working for me

Question: I followed the documentation on drupal.org to add a field to the theme-settings.php but there must be a mistake in code.
I have a filed called advertisement and a theme called tv so I tried to copy and paste the code and altered the names where to alter theme the files I created in particular are as follows:
/themes/tv/templates/system/thme-settings.php
#}
function tv_form_system_theme_settings_alter(&$form,
\Drupal\Core\Form\FormStateInterface &$form_state, $form_id = NULL) {
  // Work-around for a core bug affecting admin themes. See issue
 #943212.
  if (isset($form_id)) {
    return;

$form['tv_advertisement'] = array(
    '#type'          => 'textfield',
    '#title'         => t('Widget'),
    '#default_value' => theme_get_setting('tv_advertisement'),
    '#description'   => t("Place this text in the widget spot on your
 site."),
  );
}
$tv_advertisement = theme_get_setting('tv_advertisement');

in /themes/tv/config/install/tv.settings.yml I put this:
tv_advertisement: blue bikeshed

My /themes/tv/tv.theme file contains:
<?php
/**
 * @file
 * Theme functions
 */

// Include all files from the includes directory.
$includes_path = dirname(__FILE__) . '/includes/*.inc';
foreach (glob($includes_path) as $filename) {
  require_once dirname(__FILE__) . '/includes/' . basename($filename);
}

 function tv_preprocess_node(&$variables) {
 $variables['tv_advertisement'] =
 theme_get_setting('tv_advertisement');
 }

The last thing I did was
drush cr

to clear the cache
When I go to the themes settings page there are only the default fieds displayed but the new one does not show up
If anyone has a clue what I am doing wrong and is able to enlighten me I would be very pleased.
Thank you
Answer: Try to remove this code:

  if (isset($form_id)) {
    return;

The hook interface of hook_form_system_theme_settings_alter is hook_form_system_theme_settings_alter(&$form, \Drupal\Core\Form\FormStateInterface $form_state).
You can remove the third parameter:

 $form_id = NULL
Comment: even if i remove this the variable still ge's not rendered in a twig.html file  The settings page looks good but if I use {{tv_advertisement}} in a twig file for example there is no rendering of the var
Comment: Did you use it in the template **node.html.twig** or in another template ?  I assume if you put it in the preprocess tv_preprocess_node you only get it in node.html.twig.
Comment: how to get the variable rendered in other templates? What the name suggestions are or how to determine theme by template names?
Comment: You can set your debug variable to true like specified in the Drupal documentation -> https://www.drupal.org/docs/8/theming/twig/debugging-twig-templates and after go to the generated html to see HOOK in comments. Also, you can see theme hook preprocess explanation in the Drupal documentation https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Render!theme.api.php/function/hook_preprocess_HOOK/8.2.x
Comment: Shorter is better function tv_preprocess_node(&$variables) for node
function tv_preprocess_page(&$variables) for page
function tv_preprocess_field(&$variables) for field

Never thought that this is that simple! :-)


Maior quantidade de aparições de um elemento em um array de objetos - JavaScript

Question: Olá,
tenho um array de notícias(objetos)
lista: Array<Noticia> = [];

Desse array preciso descobrir o autor que mais publicou noticias.
Uma noticia tem:
ID | Título | Tags | Autor | Conteúdo.
Minha ideia inicial foi:
public totalPublicacoes(){
 var autorM = '';
 var autor = this.lista[0].autor;
 var qtM = 0;
 var qt = 0;
 for(let i=0; i<this.lista.length; i++){
   if(autor == this.lista[i].autor){
     qt ++;
     if(qt > qtM){
       qtM = qt;
       autorM = this.lista[i].autor;
     }
   }
 }
 console.log(autorM);}

Com isso minha ideia inicial foi pegar o autor do primeiro objeto do array
verificar se ele era igual ao próximo elemento autor do array
caso sim eu somava a qt de vezes que ele apareceu
caso a qt atual fosse maior que a quantidade maior de aparições
eu atualizava o valor da maior aparição
e guardava o nome do autor que em tese mais publicou.
Porém não está funcionando, podem me ajudar?
Answer: Bom dia Gabriel tenta assim:
seuarraydenoticias.reduce(function (autores, autor) {
  if (autor.Autor in autores) {
    autores[autor.Autor]++;
  }
  else {
    autores[autor.Autor] = 1;
  }

  return autores;
}, {});

else vai retornar um novo array com o nome do autor e a quantidade.
Espero ter ajudado amigo.
Comment: Nossa deu certo! muito obrigado! só uma duvida, o tipo desse autores é obj certo? sendo assim terei que quebra-ló para poder comparar os valores certo?
Comment: Na verdade estou tentando agora separar esses valores para poder comparar e dizer quem publicou, mas está difícil uma vez que não sei qual o nome do valor que diz sobre a contagem de publicações..
Answer: Pessoal consegui resolver, caso alguém algum dia passe pela mesma situação:
public maisPublicou() {
const frequency = this.lista
  .map(({ autor }) => autor)
  .reduce((autores, autor) => {
    const count = autores[autor] || 0;
    autores[autor] = count + 1;
    return autores;
  }, {});
var qt = 0;
var autorM = '';
for (var key in frequency) {
  if (frequency[key] > qt) {
    qt = frequency[key];
    autorM = key;
  }
}
return autorM;}


Are these statements about p-values correct?

Question: I've seen the following equivalent statements about p-values:

"It is the probability of wrongly rejecting the null hypothesis if it is in fact true." [1]
"The P value or calculated probability is the estimated probability of rejecting the null hypothesis (H0) of a study question when that hypothesis is true." [2]

Some people I know insist these are wrong, and say these are equivalent to type I error. But the type I error is in fact determined by the significance level. Are these right or wrong?
[1] http://www.stats.gla.ac.uk/steps/glossary/hypothesis_testing.html#pvalue
[2] http://www.statsdirect.com/help/default.htm#basics/p_values.htm
Comment: [This](http://stats.stackexchange.com/questions/44769/understanding-p-value?rq=1) previous discussion might help.
Answer: There are two p-values of interest. The critical p-value, also known as the $\alpha$-level or significance level, is decided and fixed before the study / analysis is performed. This critical p-value is in fact the probability of Type I error. Your examples are talking about the critical p-value, and I agree with those who are saying they are incorrect or at least misleading, because in common usage "the p-value" refers to the next type of p-value.
The observed p-value, or calculated p-value, is calculated as part of a statistical test. The observed p-value is the probability under the null hypothesis of observing data that leads to a test statistic as or more extreme than the test statistic actually observed in the experiment. The definition of "extreme" varies depending on your test, e.g. one-sided versus two-sided. The common usage of the phrase "the p-value" usually refers to the observed p-value.
We reject the null hypothesis if the observed p-value is less than or equal to the critical p-value. This is because, under the null hypothesis, the observed p-value is uniformly distributed on $[0,1]$. Thus the probability that we reject the null when in fact the null is true is equal to the critical p-value.
Edit: The full section in [1] is talking about the observed p-values, but the one line you quoted only makes sense when talking about the critical p-value. If you were to force it to be about observed p-values and be correct, it would be something like "It is the probability of a new, identical study wrongly rejecting the null hypothesis if you set your alpha-level equal to it." Strictly speaking, once you've done your analysis, the probability of wrongly rejecting the null hypothesis is either 0 or 1, depending on whether or not the null hypothesis is true and whether or not you rejected it!
Comment: Thanks. But [1](http://www.stats.gla.ac.uk/steps/glossary/hypothesis_testing.html#pvalue) seems to make the statement I quoted about (observed) p-values, not significance levels. Hence the confusion.
Comment: I've updated my answer to address this.


VBA - Contando o número de células preenchidas na linha

Question: Olá, pessoal.
Estou tentando criar um código responsável por ler a quantidade de células preenchidas nas linhas.. Tentei desta forma porém não tive sucesso.
<code>  For b = 1 To rMaior
         contador = Worksheets("1").Range("b:b").Cells.SpecialCells(xlCellTypeConstants).Count
                MsgBox contador
  Next b
</code>
Obrigado.
Answer: Olá,
Creio que sua fórmula possui um pequeno erro... tente usar o nome da planilha conforme abaixo:
<code>MsgBox Plan1.Range("b:b").Cells.SpecialCells(xlCellTypeConstants).Count
</code>
Espero ter ajudado!
Answer: Altere seu código para isso:
<code>  For b = 1 To rMaior
         contador = Worksheets("1").Rows(b).Cells.SpecialCells(xlCellTypeConstants).Count
         MsgBox contador
  Next b
</code>
Esse código considera que você quer contar n linhas da planilha chamada 1. Além disso, deve ter uma variável chamada rMaior declarada e que especifica a última linha que deverá ser contada. Assim, o código percorrerá da linha 1 até rMaior contando o número de células não vazias por linha.


Is it possible to specify the indices for K-fold cross-validation with the tune function in R?

Question: I would like to use the tune() function in R to specify the hyperparameters in an SVM model. The particular SVM implementation that I am using is the one that is provided in the e1071 package.
By default, tune() runs a K-fold cross-validation, and chooses the testing and training sets for each fold by itself. I am wondering whether I can override this behavior and specify the indices of the testing and training set for each fold myself.
If not, does anyone know if there is a package or combination of packages which allows me to run a K-fold CV on SVM and specify the indices of the training and testing set.
Answer: The caret package allows you to specify the training and testing indexes for each fold/resample, using the indexes and indexesOut arguments of the trainControl argument for train.
By default, caret uses kernlab for svms, but you can use the custom argument for trainControl to specify any predictive modeling function you wish.


L2 regularisation in DNN context

Question: could you explain me in DNN (deep neural networks) context the meaning of "m" and "w" in the following formula (L2 regularization)?
Cost function = Loss + Lambda/2m * sum||w||(exp[2])
I know that m is the number of data points while w the weight. This in a "regression context". I'm a little bit confused to figure it out in the context of DNN.
Thank you in advance
Comment: they have the same meaning for neural nets. what is confusing for NN?
Answer: As you noticed, $m$ stands for the number of parameters, and $w$ stands for the weights. If you understand it in the context of regression, the same idea applies to neural networks. The generalized linear model (like linear regression, logistic regression, etc) is
$$
E[y|X] = \sigma(\, w_1 x_1 + w_2 x_2 + \dots + w_m x_m + b \,)
$$
where $w_i$ are the regression coefficients (weights), $b$ is the bias term, and $\sigma$ is the inverse of the link function (or activation function, in neural network terms). This basically is a single neuron of a neural network, the difference is that neural networks have many such neurons and many layers where each layer takes some input, and its outputs are served as inputs (the $x$'s) to the next layer. So for a neural network, the $m$ would be the sum of all the parameters for all the neurons.
However, notice that the formula you quote can be simplified. Without going into details, the bare bones $L_2$ regularization is
$$
\text{loss}(y, f(X)) \, + \lambda\, \| \boldsymbol{w} \|^2_2
$$
Scaling by $m$ is needed only to make the $\lambda$ term easier to interpret, but it doesn't really matter because you find the $\lambda$ using a hyperparameter tuning algorithm and don't really care what it is.
For a recap on regularization, you can check Andrew Ng's lecture.
Comment: Thank you very much! More clear thanks to your sentence: "m would be the sum of all the parameters for all the neurons".

L2 regularisation in DNN context


Trying to put two runat=server forms in one web form asp.net

Question: i'm building a site for course project in Web forms asp.net
and i did a master page for all the website
i want to have a text box that acts as a search in the website
but it needs to be in
<form runat="server">

this is master code

            <form runat="server">

                <asp:ContentPlaceHolder ID="head" runat="server"></asp:ContentPlaceHolder>
                <title>Dor Lugasi Guitar Shop</title>
                <link href="Styles.css" rel="stylesheet" type="text/css" />
                <right>

                    <asp:LinkButton ID="lnkHome" runat="server" PostBackUrl="~/Pages/Home.aspx" >Home</asp:LinkButton>
                    <asp:LinkButton ID="lnkLogin" runat="server" PostBackUrl="~/Pages/Login.aspx">Login</asp:LinkButton>
                    <asp:LinkButton ID="lnkContact" runat="server" PostBackUrl="~/Pages/Contact.aspx" >Contact</asp:LinkButton>
                    <asp:LinkButton ID="lnkGuitarLab" runat="server" PostBackUrl="~/Pages/GuitarLab.aspx">Guitar Lab</asp:LinkButton>
                    <asp:ImageButton style="float:left;margin-left:10px;" ID="btnCart" runat="server" Height="55px" ImageUrl="~/Images/cart.png" Width="55px" PostBackUrl="~/Pages/Cart.aspx" />
                    <asp:TextBox ID="txtSearch" runat="server" BackColor="Silver" BorderColor="Black" TextMode="Search" placeholder="               Search" onkeypress="txtSearch"></asp:TextBox>
                    <asp:ImageButton ID="btnSearch" runat="server" ImageUrl="~/Images/src.png" OnClick="btnSearch_Click" Width="25px" />

                    <asp:Label style="float:left;margin-left:10px;" ID="lblLoggedUser" runat="server" Font-Bold="True" Font-Size="Large" ForeColor="Red"></asp:Label>
                    <asp:LinkButton ID="btnLogOut" runat="server" OnClick="btnLogOut_Click" Visible="False" CssClass="auto-style5" Height="24px" Width="99px">Log Out</asp:LinkButton>
                    <asp:LinkButton style="float:right;margin-right:10px;" ID="lnkManageProducts" runat="server" PostBackUrl="~/Pages/ManageProducts.aspx">Manage Products</asp:LinkButton>

                    <asp:LinkButton style="float:right;margin-right:10px;" ID="lnkManageUsers" runat="server" PostBackUrl="~/Pages/ManageUsers.aspx">Manage Users</asp:LinkButton>

            </form>

        <form runat="server">

                <asp:contentplaceholder id="ContentPlaceHolder1" runat="server">
                </asp:contentplaceholder>

                </left>

        </form>

but im getting a:
A page can have only one server-side Form tag.

before i added this  search textbox and button i had
<form runat="server">

...
.....
....
.....

</form>

and it worked
but at the login page when i pressed Enter it applied to the search form instead of the login form
Comment: You got to wire it up to another "page". A `search.aspx` page that your code behind handles. Here's an example I found: http://www.frederikvig.com/2009/06/using-multiple-forms-on-an-asp-net-web-forms-page/
Answer: eventually i ended up making a single form for the page and just put the  specific form that had buttons in
<asp:Panel ID="search_panel" runat="server" DefaultButton="btnSearch">
                    <asp:TextBox ID="txtSearch" runat="server" BackColor="Silver" BorderColor="Black" TextMode="Search" placeholder="               Search" onkeypress="txtSearch"></asp:TextBox>
                    <asp:ImageButton ID="btnSearch" runat="server" ImageUrl="~/Images/src.png" OnClick="btnSearch_Click" Width="25px" />
                    </asp:Panel>

i hope thats the right way
Comment: Yes, in Web Forms, just like the error message indicates, you can only have one form with the `runat=server` attribute. It's dumb, but that's Web Forms. Did you choose to use Web Forms, or were you forced to? If your teacher is still teaching Web Forms in 2017, then they need to get with the times and teach modern web tooling. At least ASP.NET MVC, but it'd be even better to teach a front end system like Angular, Vue, React etc and combine that with a REST based API framework like ASP.NET Web API.
Comment: i totally agree with you.
in my spare time im teaching myself angular
but the point of the course is not the site building, communication and the site is just  to implement the basic stuff


Rank of augmented matrices.

Question: Suppose there are two matrices as follows:
$M=[A~B]$, $N=[C~B]$,
where $A\in\mathbb{R}^{2\times 4}$, $B\in\mathbb{R}^{2\times 6}$, $C\in\mathbb{R}^{2\times 2}$.
Suppose Im$(A)=$Im$(C)$, where Im$(X)$ denotes the image (column space) of $X$.
My question is
"Is the rank of $M$ the same as the rank of $N$?"
(In case the rank of $B$ is not full rank.)
Comment: When you write $a \times b$, does this mean $a$ rows and $b$ columns?
Answer: Yes. Let The rank of $M$ is the dimension of the span of the columns of $A$ and the columns of $B$. The rank of $N$ is the dimension of the span of the columns of $C$ and the colmuns of $B$. Since the columns of $A$ and the columns of $C$ span the same subspace by assumption, we are done.

Rank of augmented matrices.


how can I draw the pushdown automaton for a language

Question: How can I draw the pushdown automaton for the language $\{w \in \{a,b,c\}^{*}:w \text{ is of the form } y c y^{R}, y \in \{a,b\}^{*}\}$?
Answer:

let's call the starting state COPY
on input character $a$ or $b$, PUSH this into the stack and stay on state COPY
if character $c$ is the input, move to state CHECK
from state CHECK, input $c$ leads to state ERROR
similarly, both cases (tape: $a$, POP: $b$), $\ $(tape: $b$, POP $a$) lead to state ERROR
the other two inputs (tape: $a$, POP: $a$), $\ $(tape: $b$, POP $b$) stay on state CHECK
finally (tape: EOT, POP: empty) goes to state ACCEPT
Comment: I will post it as an answer,so that you can see if that what I have drawn is right.
Answer: That's what I have tried.Do you mean that I should do something like that?
Comment: Yes, something like that.
I don't exactly know how  a stack (pushdown) automaton is pictured usually, I mean how to indicate the PUSH/POP informations and how to distinguish the stack input (POP) from the tape input in the drawing...
Comment: if you write it down, it's usually clearer. If the exercise was specifically to draw, then add some picture..
Comment: If I want to show that a language is accepted by a pushdown automaton,do I have to draw one or could I just write something like that,what you have written in your previous post?


How to create a node from an other node template?

Question: I would like to have a form for the creation of a node (let's name it news) when you are on the page of an other type of node (let's name it movie). So on /node/31 (of type movie) I would like to have the node content and a form to create a news
I thought about using a Webform but after all it felt like trying to hack the Webform with a trick. Moreover since the form already exists (at path node/add/node_name), I might be able to reuse it but I don't know if it is really possible.
What should be the right way to do it ? With a webform ? By exposing the form for the specific node ? By creating a custom form ? Maybe an other solution I didn't think about ?
Answer: Use Form Block

Enables the presentation of user registration, site wide contact, or
  node creation forms in blocks

Select your content type for the form. Place block in region and for visibility select the other content type.
If you rather render the block using a template file, ex: node--movie.html.twig you can use Twig Tweak and print it out using {{ drupal_block('block_id') }}
Comment: The module is working (not with Twig Tweak, I've got a message saying that the block is defective but the module is still working).
Answer: That's actually a task for Inline Entity Form.

Provides a widget for inline management (creation, modification,
  removal) of referenced entities. The primary use case is the parent ->
  children one (product display -> products, order -> line items, etc.),
  where the child entities are never managed outside the parent form.
  Existing entities can also be referenced.

It will simply add some widgets for entity/node reference fields that then let you create and edit the referencing entities/nodes in a node form.
Answer: You may try drupal_entity_form provided by the Twig Tweak module.
{{ drupal_entity_form('node', values={type: 'news'}) }}


Pausar programa com Timer

Question: Preciso usar ele para esperar 2 minutos antes de executar um método. Já pesquisei e não consigo achar nada claro o suficiente para me ajudar.
Já usei o Thread.sleep() mais ele trava a thread toda, e não é isso que eu busco.
Obrigado aos que ajudarem.
Answer: Isso dever ser simples.
Timer timer;

public Executar(int segundos) {
    timer = new Timer();
    timer.schedule(new MinhaFuncao(), segundos*1000);
}

class MinhaFuncao extends TimerTask {
    public void run() {
        //faz tudo que precisa fazer aqui
        timer.cancel(); //Termina a task
    }
}

Dai você chama o executar passando 120 segundos para o método.
OBS: esse exemplo utiliza o timer da java.util.Timer http://docs.oracle.com/javase/7/docs/api/java/util/Timer.html
Se você for usar o javax.swing.Timer doc: http://docs.oracle.com/javase/7/docs/api/javax/swing/Timer.html
Então você deve fazer assim
Timer timer;

int delay = 120*1000; //2 minutos

ActionListener tarefaAgendada = new ActionListener() {
    public void actionPerformed(ActionEvent evt) {
        //faz o que tem que fazer
    }
};

timer = new Timer(delay, taskPerformer);
timer.setRepeats(false); //faz rodar apenas 1 vez, sem isso ele ficar invocando a action de 2 em 2 minutos
timer.start(); //inicia a task

Comment: @Higor fico feliz que solucionou seu problema. Lembre-se de marcar a resposta como correta para ajudar outros usuários que possam ter a mesma dúvida que você.
Comment: O construtor timer pede 2 parâmetros aqui. Um inteiro e um ActionListener. Consequentemente o código nem rodou.
Comment: @Higor esta usando o timer da  java.util.Timer ?
Comment: @Higor veja minha edição na resposta
Comment: Eu estava usando Timer da java.swing.Timer, troquei e consegui usar aqui. Obrigado, abraços.

Pausar programa com Timer


Proof for weak (1,1)-estimate for Hilbert Transform

Question: I am following the proof given in the book Fourier Analysis by Javier Duoandikoetxea and I am unsure about some elements of the proof given for the weak $(1,1)$ estimate for Hilbert Transform.
Firstly, the theorem is stated for a Schwartz $f$. So is it correct to say that proving "$H$ is weak $(1,1)$" is equivalent to saying that we are only proving the $(1,1)$ estimate when $f \in L^1(\mathbb{R}) \cap \mathcal{S}(\mathbb{R})$?
Secondly, I am unsure why is the bad part $b$ of the function $f$ is in $L^2(\mathbb{R})$. It is not hard to see that the good part $g$ is in $L^1(\mathbb{R}) \cap L^\infty(\mathbb{R})$ since its norm equals that of $f$ and it is above bounded by the bound set by the Calderon-Zygmund decomposition. But I am unsure why the bad part
$$b (x):= \sum_{j} b_j(x)  :=  \sum_{j} \left(f(x) - \frac{1}{ \lvert I_j \rvert } \int_{I_j} f\right) \chi _{I_j}(x),$$
is in $L^2(\mathbb{R})$, where the collection $\{I_j\}$ is of disjoint intervals given by the Calderon-Zygmund decomposition.
Lastly, why is the following statement true?

Even though $b_j \not\in \mathcal{S}$, when $x \not\in 2I_j$ the formula $$Hb_j(x) = \int_{I_j} \frac{b_j(y)}{x-y} \mathrm{\, d}y$$ is still valid.
Answer: I believe the following answers:

$b$ is in $L^2(\mathbb{R})$ since $b = f- g$ where $f \in \mathcal{S} \subset L^2(\mathbb{R})$ and $g \in L^2(\mathbb{R})$

To see why the statement holds true, let $b_j^\varepsilon$ be the mollification such that $b_j^\varepsilon|_{I_j} = b_j, \operatorname{supp} b_j^\varepsilon \subset 2I_j.$ Then $b_j^\varepsilon \xrightarrow{L^2(\mathbb{R}) }  b_j$ and so $Hb_j^\varepsilon \to Hb_j$, by boundedness of $H$ on $L^2(\mathbb{R}). $ Now we have $\forall  x \not\in 2I_j: $
$$ Hb_j(x) = \lim_{\varepsilon \to 0^+} Hb_j^\varepsilon(x) = \lim_{\varepsilon \to 0^+} \lim_{s \to 0^+}  \int_{ \lvert y \rvert > s}\frac{1}{\pi} \frac{b_j^\varepsilon(y)}{x - y} = \lim_{\varepsilon \to 0^+}   \int_{\operatorname{supp} b_j^\varepsilon}\frac{1}{\pi} \frac{b_j^\varepsilon(y)}{x - y} =\int_{I_j} \frac{1}{\pi} \frac{b_j(y)}{x - y},$$
where the latter equality follows from $b^\varepsilon _j \xrightarrow{a.e.}_s b_j $ and $ \left\lvert \tfrac{b_j^\varepsilon}{x-y} \right\rvert \leq \frac{2\lVert b_j^\varepsilon  \rVert_{L^\infty(2I_j) }}{ \lvert I_j \rvert } \in L^2(2I_j)  $. Then by dominated convergence, the limit goes inside, and we have the required equality.
Comment: Apologies for coming back to an old question. I had the same question as you, though I am not sure about this answer. It seems that you are saying the mollification of $b_j$ is Schwartz. I know it is smooth, but can we be certain of the rapid decay?
Comment: @RobbieA Recall that mollifiers are smooth so convolution with smooth functions is smooth. Finally, $b^\epsilon_j$ is compactly supported so it is a compactly supported smooth function. Such functions are always Schwartz (and in fact dense).
Comment: ah yes your're right. I forgot about the compact support. Thank you


Implementing the procducer-consumer pattern with .NET 4.0

Question: With alle the new paralell programming features in .NET 4.0, what would be a a simple and fast way to implement the producer-consumer pattern (where at least one thread is producing/enqueuing task items and another thread executes/dequeues these tasks). Can we benfit from all these new APIs? What is your preferred implementation of this pattern?
Answer: Implement IProducerConsuerCollection or derive from BlockingCollection. Samples are provided in the documentation for these items.


Characteristic polynomial of matrix with real coefficients

Question: I am reading a lecture note about matrix.
Let $p_A(x)$ be the characteristic polynomial of matrix $A$.
Can we say if $p_A(x) = p_A(-x)$, then $p_A(x)$ is a real coefficient polynomial?
Comment: If $p(x) = p(-x)$ then $p$ is even (by definition) but we can find even polynomials with complex coefficients.  $p(x) = ix^2$ would be an example.
Answer: Note that your title is ambiguous. I have taken it to mean that the matrix has real entries....
It is restrictive, but perhaps not as much as you think. In any case, a real matrix has a characteristic polynomial with real coefficients.  Now, the characteristic polynomial of
$$
\left(
\begin{array}{rrrr}
0 & 1 & 0 & 0 \\
0 & 0 & 1 & 0 \\
0 & 0 & 0 & 1 \\
-4 & 0 & 0 & 0 \\
\end{array}
\right)
$$
is
$$  x^4 + 4  $$ while the roots are
$$  1+i, 1-i, -1+i, -1-i  $$


Difference between econometrics tags?

Question: Whats the difference between the usage of the tags econometrics and applied-econometrics?
Im seeing alot of overlap in the use of the two tags and am wondering if there is a functional difference between the two for the site.
Answer: There is a potentially useful distinction here: "applied-econometrics" can cover specific questions on implementing models with data, issues with software use (not programming), or also discussion/request of specific applied econometrics studies.
Then, "econometrics" could stand for the more general theoretical questions about properties of estimators, theoretical models etc.
It will be up to users to be careful enough to actually respect and apply this distinction -so if one searches the questions and finds that they don't, it would be an argument in favor of merging/synonymize them.
On the other hand, these are indeed two distinct worlds in econometrics, and in academia the difference is well understood, so perhaps we should keep them distinct in any case.
I have just wrote a short tag-wiki for "applied econometrics" in an attempt to help the situation.

Difference between econometrics tags?


What is the diffrence beetween Days and TotalDays?

Question: Can anybody tell me what is the difference between these two functions in C#? TotalDays and Days because I'm not sure which once I should use in my code? Sorry for the low information on this text, but there is not much I can talk about.
Comment: Read the documentation. [`TotalDays`](http://msdn.microsoft.com/en-us/library/system.timespan.totaldays(v=vs.110).aspx) is a `double` because it represents whole and fractional days whereas [`Days`](http://msdn.microsoft.com/en-us/library/system.timespan.days(v=vs.110).aspx) is an `int` and represents only whole days.
Comment: Sometime SO is easier/faster/etc than original documentation. Especially slow Microsoft msdn servers
Answer: Since I haven't found a duplicate, I post my comment here:
Always read the documentation first. TotalDays is a double because it represents whole and fractional days whereas Days is an int which represents only whole days.
That is even mentioned explicitly in the remarks sections of TimeSpan.Days/TotalDays:

The Days property represents whole days, whereas the TotalDays
property represents whole and fractional days.

One thing to note, as opposed to the other properties in TimeSpan like Hours/TotalHours there is no limit on Days. So it doesn't end with 30 or 365 (like Hour which ranges from -23 through 23) since there is no larger unit than year. So Days will always be the same number as (int) ts.TotalDays.

A TimeSpan doesn't have a sensible concept of "years" because it
depends on the start and end point. (Months is similar - how many
months are there in 29 days? Well, it depends...) [J. Skeet]
Comment: Some useful examples here: https://zzz.buzz/2017/12/07/csharp-traps-and-pitfalls-timespan-days-and-totaldays/
Comment: The inclusion of `Days` is surprising because the number of seconds-per-day is actually variable (e.g. Leap Seconds). Looking at the source-code for the two properties I see the .NET code performs naive division on Ticks assuming a fixed number of ticks per day, which I think is incorrect.


Gradient of the expectation of a function w.r.t. distribution parameters

Question: In section 2.2 of Kingma & Welling's paper on variational auto-encoders authors write the following equality for the gradient of the expectation of a function with respect to the parameters of the probability distribution:
$$
\nabla_\phi \mathbb{E}_{q(\mathbf{z}|\phi)}[ f(\mathbf{z}) ] ~=~
\mathbb{E}_{q(\mathbf{z}|\phi)}\,[
   f(\mathbf{z})\, \nabla_{q(\mathbf{z}|\phi)} \log q(\mathbf{z}|\phi) ]
\hspace{41mm}(1)
$$
The authors then note that the RHS can be approximated by
$$
\mathbb{E}_{q(\mathbf{z}|\phi)}\,[
   f(\mathbf{z})\, \nabla_{q(\mathbf{z}|\phi)} \log q(\mathbf{z}|\phi) ]
~\simeq~
\frac{1}{L}\sum_{i=1}^L f(\mathbf{z}^{(i)})
\, \nabla_{q(\mathbf{z}^{(i)}|\phi)} \log q(\mathbf{z}^{(i)}|\phi)
\hspace{10mm}(2)
$$
where ${z}^{(i)} \sim q(\mathbf{z}|\phi)$.
My question is why the rewrite in equation (1) is even needed. For instance, could one approximate the LHS in equation (1) directly with something like:
$$
\nabla_{\phi} \left\{\frac{1}{L}
\sum_{i=1}^L f(\mathbf{z}^{(i)}) \, q(\mathbf{z}^{(i)}|\phi)
\right\}
~=~
\frac{1}{L}\sum_{i=1}^L f(\mathbf{z}^{(i)}) \,
\nabla_\phi \, q(\mathbf{z}^{(i)}|\phi)
\hspace{10mm}(3)
$$
Another question I have is whether the rewrite in equation (1) is an example of "differentiating under the integral" which is mentioned by Feynman in one of his memoires.
Answer: Without knowing anything in partuclar about auto-encoders: (1) $E_{q(z|\phi)}[f(Z)]$ this notation is not really 'standard' but usually means the expression
$$ E_{q(z|\phi)}[f(Z)] = \int_Z f(z) q(z|\phi) dz $$
I.e. it is a quirky way of writing the (factorization) of the conditional expectation $E[f(Z)|\Phi=\phi]$ where the common distribution $f_{Z|\Phi}(z|\phi)$ is given by $q(z|\phi)$. Now we want to proceed like this:
  $$\partial_\phi E_{q(z|\phi)}[f(Z)] = \partial_\phi \int_Z f(z) q(z|\phi) dz =  \int_Z f(z) \partial_\phi q(z|\phi) dz$$
It is mathematically not clear ad hoc that you may do this (and there are examples where this in general, i.e. $\partial_\phi \int f(x,\phi) dx = \int \partial_\phi f(x,\phi) dx$ is false!). The reason is that one needs to verify some assumptions that allows us to interchange two limits (differentiation and integration) but in probability theory these assumptions are usually met or assumed in turn. The assumption more precisely is that we can bound the functions
  $$ \frac{f(z) q(z|\phi + \delta) - f(z) q(z|\phi)}{\delta}$$
uniformly in all $z$ and $\delta$ ($\delta$ being close to $0$) by a single function that is still integrable (then we can use the Lebesgue dominated convergence theorem in order to interchange the limits). In your case it will (probably) work like this: Use the mean value theorem to write
  $$\frac{f(z) q(z|\phi + \delta) - f(z) q(z|\phi)}{\delta} = f(z) \partial_\phi q(x|\phi + \theta)$$ for some $\theta$ in the interval $[-\delta, \delta]$. Then find an integrable bound function for this derivative.
In short: (1): Yes, this seems to me as if it is one of these examples where you need to pull the differentiation into the integral.

My question is why the rewrite in equation (1) is even needed.

Let us assume that you directly approximite the lhs. Who tells you that this approximation is any good?? We have results of the form
''We can approximize the expected value $E[W]$ of a random value $W$ by $\frac{1}{L} \sum_{i=1}^L w_i \cdot f_W(w_i)$ this and that good if we have a sufficient amount of samples $w_1, ..., w_L$.''
So we could use these results on $\partial_\phi E_{q(z|\phi)}[f(Z)]$ but only if we know that this weird expression (the derivative of a random variable that somehow depends on a second parameter in this parameter) is an expected value of a random variable again and by pulling the derivative into the integral you do precisely that: Realize that this is the normal expectation of a random variable again, namely $E[f(Z)\cdot \partial_{q(z|\phi} \log q(z|\phi)]$.
In short: you need to rewrite (1) to (2) in order to see that the approximation is any good.
On your (3): Even in all the usual well known theorems (like the central limit theorem, the proof that the mean is a consistent estimator, etc) you need to divide the sum by the number of samples, otherwise the sum does not have a chance to converge but just vanishes towards infinity.
Answer: I don't know if this is too late but I would like to add my two cents.
Let consider the simple case where $\phi$ and $z$ are 1 dimensional
$\nabla_{\phi}\mathbb{E}_{q(z|\phi)}[f(z)] = \lim_{\delta \rightarrow 0} \frac{\int_{z=-\infty}^{z=\infty}f(z)q(z|\phi + \delta)dz \ - \int_{z=-\infty}^{z=\infty}f(z)q(z|\phi)dz}{\delta} = \lim_{\delta \rightarrow 0} \frac{\int_{z=-\infty}^{z=\infty}f(z)[q(z|\phi+\delta) - q(z|\phi)]dz}{\delta} \quad (4) $
If you want to approximate
$\int_{z=-\infty}^{z=\infty}f(z)q(z|\phi)dz \approx \frac{1}{L}\sum_i f(z_i)q(z_i|\phi) \quad (5)$
then in order to use Eq. (4), you need to guarantee that the variance of RHS in  Eq. (5) much smaller than $\delta$. This way you have to work with $d\delta^2$ while in Eq. 2, you only have to work with $d\delta$. Your method will require $n^2$ samples to achieve the same error with the method in Eq. 2.


What values of $a$ and $b$ does this system have infinitely many solutions?

Question: As a disclosure, this question is more for me to confirm that I did my work correctly. More specifically, the "solution" provided to me claims there are two values of $a$ and $b$ that yield infinite solutions, but I found only one.
That said, this is the problem:
For what values of $a$ and $b$ does this system have infinitely many solutions
\begin{equation} A =
\begin{pmatrix}
    a & 0 & b & |2 \\
    a & 2 & a & |b \\
    b & 2 & a & |a \\
\end{pmatrix}
\end{equation}
I reduced the matrix to
\begin{equation} A =
\begin{pmatrix}
    a & 0 & b & |2 \\
    0 & 2 & a-b & |b-2 \\
    0 & 0 & b-\frac{b^{2}}{a} & |2-b+a-2\frac{b^{2}}{a} \\
\end{pmatrix}
\end{equation}
For this system to have infinite solutions, then one row must be $0$. In order for that to happen, I determined that $b=0$, $a=-2$.
However, the solution I have claims $b=0$, $a=-2$, or $a = b \neq 0$. I don't see how the latter is possible based on my reduction. Either I did something wrong, or the solution is wrong. Any insight would be appreciated.
Comment: That is also one way to check, but that could also imply there are no solutions. I don't believe it guarantees that there are infinite solutions.
Answer: You can check easily if the solution is wrong. (It is not.) Put $a=b$. Then you have:
\begin{equation} A =
\begin{pmatrix}
    a & 0 & a & |2 \\
    a & 2 & a & |a \\
    a & 2 & a & |a \\
\end{pmatrix}
\end{equation}
Reducing to (III - I) and (II - I)
\begin{equation} A =
\begin{pmatrix}
    a & 0 & a & |2 \\
    0 & 2 & 0 & |a-2 \\
    0 & 0 & 0 & |0 \\
\end{pmatrix}
\end{equation}
For $a\neq 0$ this has infinitely many solutions. (For $a=0$ it has none.)
Thus there is a problem in your computation.
Comment: Yep, you're right. I have a $b^{2}$ where it should be $b$ in $a_{34}$. Thanks. Must have done one too many problems today.
Comment: I am glad it was useful and you found the place were the problem arises.


Java program asking for 10 numbers and puts them in an array. How do I ask for input? Is my code correct so far?

Question: This program asks for the user to input 10 numbers and is converted into an int array. If the array is lucky (contains the numbers 7, 13, or 18) then it prints out the sum of all the numbers in the array.  If it does not contain those then it is false and it only prints the sum of all the even numbers.
How do I correctly ask for this input?  How do I get the sum of the even numbers in the array?  Is any of the other code incorrect?
import java.util.Scanner;

public class FunArrays {

public static void main(String[] args) {
    // until you do user input, you should test your methods using "test" as the input.
    int[] test = {1,2,3,4,5,6,7};
    luckyNumber1 = {7};
    luckyNumber2 = {13};
    luckyNumber3 = {18};

    int[] a=new int[9];
    Scanner sc=new Scanner(System.in);
        System.out.println("Please enter numbers...");
        for(int j=0;j==9;j++)
        a[j]=sc.nextInt();

}

public static int sum(int [ ] value) {
      int i, total = 0;
      for(i=0; i<10; i++)
      {
          total = total + value[ i ];
      }

      return (total);
 }
public static int sumOfEvens (
public static boolean isLucky (int[] array) {

    if ( (int == luckyNumber1) || (int == luckyNumber2) || (int == luckyNumber3 )
        return true;

    else
        return false
}

// write the static methods isLucky, sum, and sumOfEvens

}

Comment: You want us to proof your code?
Comment: My initial questions and yes if you could.  That's up to you though! @DummyCode
Answer: You have an array of size 9. Instead it should be of size 10. So, Change the initialization to
int[] a=new int[10];

Use modulo operator % or remainder operator in Java to find out even or odd numbers.
    int evenSum = 0;
    for(int j=0;j<a.length;j++){
    if(a[j]%2 == 0){
       //even numbers
       evenSum = evenSum + a[j];
    }else{
       //odd numbers
      }
    }
    return evenSum;

Answer:   int[] a=new int[9];
    Scanner sc=new Scanner(System.in);
        System.out.println("Please enter numbers...");
        for(int j=0;j==9;j++)
        a[j]=sc.nextInt();

}

There's something wrong with your loop, it should be
for (int j = 0; j < 9; j++)

With this fix it should correctly prompt the users for 9 numbers. Change to 10 for the array and the loop it would takes in 10 number.
Answer: Sum of evens
static int sumOfEvens(int array[]) {
    int sum = 0;
    for(int i = 0; i < array.length; i++>) {
        if(array[i] % 2 == 0)
            sum += array[i];
    }
    return sum;
}

Input
for(int j = 0; j < a.length; j++)
    a[j] = sc.nextInt();

Comment: The above code is having a bug, the `if` condition should be `array[i] % 2 == 0`. Please update your answer.
Comment: Can you explain the input again or why you did it like that?
Comment: @Octopus Thanks for the catch! Didn't test.
Comment: @user3354072 I did it like that because you want to iterate through each index and set it equal to `sc.nextInt()`. Starting at `0` and going until you hit the `length`
Comment: @DummyCode You are saying the OP the correct and easiest way. but, the OP`s way of checking the condition as `j==9` also will work in this specific case of array of length `10`. OP has initialized the array wrongly. it should be `int[10]` not `int[9]`. See my answer!
Comment: @dummycode I'm having trouble updating my code on the site.  Is there  a way I could get it to you? And also my Eclipse program won't run it? It's just showing me ant builds that I can't use...
Answer: import java.util.Scanner;

public class MyMain1 {

    public static void main(String[] args) {
        // until you do user input, you should test your methods using "test" as
        // the input.
        int[] luckyNumbers = { 7, 13, 18 };

        int[] a = new int[10];
        Scanner sc = new Scanner(System.in);
        System.out.println("Please enter numbers...");
        for (int j = 0; j <= 9; j++) {
            a[j] = sc.nextInt();
        }
        sc.close();

        boolean sumAll = false;

        for (int i : a) {
            for (int j : luckyNumbers) {
                if (i == j) {
                    sumAll = true;
                    break;
                }
            }
            if(sumAll) {
                break;
            }
        }

        if (sumAll) {
            System.out.println("Summing all : " + sumAll(a));
        } else {
            System.out.println("Summing Only Even : " + sumEven(a));
        }

    }

    public static int sumAll(int[] value) {
        int total = 0;
        for (int j : value) {
            total = total + j;
        }
        return total;
    }

    public static int sumEven(int[] value) {

        int total = 0;
        for (int j : value) {
            if (j % 2 == 0) {
                total = total + j;
            }
        }
        return total;
    }

}