What happens when you apply SVD to a collaborative filtering problem? What is the difference between the two?

Question: In Collaborative filtering, we have values that are not filled in. Suppose a user did not watch a movie then we have to put an 'na' in there.
If I am going to take an SVD of this matrix, then I have to put some number in there - say 0. Now if I factorize the matrix, I have a method to find similar users (by finding out which users are closer together in the reduced dimensional space). But the predicted preference itself - for a user to an item will be zero. (because thats what we entered on the unknown columns).
So I am stuck with the problem of collaborative filtering vs SVD. They seem to be almost the same, but not quite.
What is the difference between them and what happens when I apply an SVD to a collaborative filtering problem? I did, and the results seem acceptable in terms of finding nearby users, which is great, but how?
Answer: $\DeclareMathOperator*{\argmin}{arg\,min}$
Ok, when you say SVD, presumably you're talking about truncated SVD (where you only keep the $k$ biggest singular values). There are two different ways to look at the truncated SVD of a matrix. One is the standard definition:

First you do the SVD: $\underset{n\times m}{X} = \underset{n\times n}{U} \overset{n\times m}{\Sigma} \underset{m\times m}{V^T}$, where $U$ and $V$ are rotation matrices, and $\Sigma$ has the singular values along the diagonal.
  Then you pick the top $k$ singular values, zero out the rest, and hack off irrelevant rows and columns to make a $k$-rank approximation to the original: $X \approx \tilde{X} = \underset{n\times k}{\tilde{U}} \overset{k\times k}{\tilde{\Sigma}} \underset{k\times m}{\tilde{V}^T}$

This is all fine and dandy (and easy to implement in R or matlab), but it doesn't make sense when talking about matrices with missing values. However, there's an interesting property of the $k$-truncated SVD--It's the best $k$-rank approximation to the original! That is:

$ \tilde{X} = \argmin_{B : rank(B)=k}  \displaystyle\sum\limits_{i,j} (X_{ij} - B_{ij})^2$

This property seems easy to generalize to the missing value case. Basically you're looking for a $k$-rank matrix that minimizes the element-wise mean squared error across the known entries of the original matrix. That is, when you're training the system, you ignore all of the missing values. (For tips on how you might actually go about finding a $k$-rank approximation, here are some places to look).
Then, once you've come up with a suitably "close" $k$-rank approximation to the original, you use it to fill in the missing values. That is, if $X_{ij}$ was missing, then you fill in $\tilde{X}_{ij}$. Tada! You are now done.
Answer: It seems like there are a lot of approaches on how to deal with missing values. The following paper with review in Section 1.3 may be a good starting point.
Answer: I need more reputation to comment on Stumpy Joe Pete's answer therefore I post this as an answer.
Stumpy thanks for the answer although I think it needs a bit of clarification. Particularly I mean this sentence:
Basically you're looking for a k-rank matrix that minimizes the element-wise mean squared error across the known entries of the original matrix.
First - wouldnt the highest rank always minimize this, or actually reconstruct the original X matrix?
Secondly - Why would you only take the known entries. Intuitively it makes sense, but the procedure is actually also fitting the empty places which were replaced with some reasonable numbers.
My approach would be to carry out something like a cross validation:

Fill in the empty places with 0s or means or another reasonable number.
Replace one of the n known elements with 0 or a reasonable number
Carry out SVD reconstruction of rank k
Check the value of the known reconstructed element.
repeat for all possible known elements and calculate MSE
repeat for all possible k and choose the one with lowest MSE.
Comment: 1. You want to choose a low k to avoid overfitting (much lower than whatever the dimensions of X are). This is basically for the same reason that linear regression is a better choice than a quintic for fitting a dataset of 6 points. 2. You don't _know_ what the unknown entries are supposed to be, so you can't measure "the element-wise MSE" across them. My procedure fills up the missing values with numbers that were derived by minimizing error against the known values (and constraining that the matrix must be low-rank).


On sheafification and stalks

Question: I have a problem but it may be easy for you.
So, please give me a lecture.
Thank you.
Let $\mathcal{F}$ be a presheaf on a topological space $X$ and ${}^a\mathcal{F}$ a sheafification of $\mathcal{F}$:
${}^a\mathcal{F}(U):=\{s:U\to \bigoplus_{x\in X}\mathcal{F}_x\ |\ s\text{ is a section of }\pi:\mathcal{F}_x\ni a\to x\in X\}$,
where $U$ is an open set in $X$ and $\mathcal{F}_x$ is a stalk of $\mathcal{F}$ associated with $x\in U$.
I understood that ${}^a\mathcal{F}$ is a sheaf, but I did not understand that $({}^a\mathcal{F})_x \simeq \mathcal{F}_x$ for any $x\in X$.
According to some texts, it is clear by definion. Why?
Comment: Possible duplicate of [Sheafification of a presheaf through the etale space](https://math.stackexchange.com/questions/146996/sheafification-of-a-presheaf-through-the-etale-space)
Comment: Where did you get that definition from? First time I see it. When I was introduced to it we defined ${}^a\mathcal{F}(U)$ as a subset of $ \prod_{x\in U}\mathcal{F}_x$. It's maybe the same thing, I'm just curios.
Comment: Also I'm not sure about your condition in the set. A section of a morphism is usually something like this: you have $\pi:A \to B$ and $s:B\to A$ with $\pi\circ s=id_B$. But in your condition  $s$ and $\pi$ don't map to the same spaces. Can you clarify?
Comment: As remarked by @Maik, your definition of ${}^a\mathcal{F}(U)$ is completely false.
Answer: As said in the comments, your description of $\mathcal{F}^a$ is not completely correct. First, it should be the disjoint union instead of the direct sum of the $\mathcal{F}_x$. Then all sections of $\pi$ are not allowed, only those that satisfy this condition :
$$ (1) \quad \forall x\in U, \exists V\ni x \text{ a neighborhood of $x$ in $U$ and }t\in\mathcal{F}(V) \text{ such that } \forall y\in V, s(y)=t_y $$
Otherwise you have too many sections, for example, if $\mathcal{F}=\mathcal{C}$ is the sheaf of continuous function on a space $X$, with your definition a section in $\mathcal{F}^a$ would consists of the choice of a germ of continuous function at every point, without conditions that these germs glue (and they might define a function which is not continuous).
Hence the good definition is the following :
$$\mathcal{F}^a(U)=\{s:U\rightarrow\coprod_{x\in U}\mathcal{F}_x | s \text{ is a section of $\pi$ and satisfies condition $(1)$}\}$$

Now it is easy to see that $\mathcal{F}^a_x=\mathcal{F}_x$. Indeed, if $s_x$ is a germ of a section in $\mathcal{F}^a_x$, then you can find a representative $(U,s)$ where $s\in\mathcal{F}^a(U)$. Now by condition $(1)$, there exists $t\in\mathcal{F}(V)$ such that $\forall y\in V, s(y)=t_y$. But this implies that $(U,s)$ and $(V,t)$ define the same germ. So $s\in\mathcal{F}_x$.
To be perfectly rigorous, check that what I just described is a well-defined map $\mathcal{F}^a_x\rightarrow\mathcal{F}_x$ which is the inverse of the obvious map $\mathcal{F}_x\rightarrow\mathcal{F}^a_x$.
Comment: How does $(U,s)$ and $(V,t)$ define the same germ? They live in different spaces.
Comment: @AlJebr The explanation was given in the line just below. Is it not clear ?
Comment: I'm sorry, where is the explanation? I see that $s$ lives in $\mathcal F^a(U)$ but $t$ lives in $\mathcal F(V)$. How can $(U,s)$ and $(V,t)$ define the same germ if the direct limit is taken over different spaces? I guess I should ask: What do you mean by they define the same germ?
Comment: @AlJebr The explanation was "To be perfectly rigorous, check that what I just described is a well-defined map $\mathcal{F}^a_x\to\mathcal{F}_x$ which is the inverse of the obvious map $\mathcal{F}_x\to\mathcal{F}^a_x$." What I meant is the class of $(V,t)$ is a germ in $\mathcal{F}_x$, and the canonical map $\phi:\mathcal{F}\to\mathcal{F}^a$ maps this germ to a germ in $\mathcal{F}^a_x$ which is still represented by $(V,t)$ (or rather $(V,\phi(t))$). Then $(U,s)=(V,\phi(t))$ in $\mathcal{F}^a_x$ and the map $(U,s)\to (V,t)$ well defined and the inverse of $\phi$.
Comment: Ok. Thanks. I think I get it now. This is all new to me so I think I was getting thrown off by the abuse of notation. I'm not used to it yet. I think I see it now though. Thanks.
Comment: I'm now learning about the inverse image sheaf and it is very heavy notation since we have direct limit of direct limit. Any advice on how to deal with all this notation?
Comment: @AlJebr You should ask a separate question with a clear explanation on where are your difficulties. Just a piece of advise : don't use the direct limit of direct limit definition, and use mainly the adjunction property.
Comment: I think it should be infinite product not infinite coproduct here...
Comment: @Chris Do you mean an infinite product in the definition of $\mathcal{F}^a$ ? Well no, it really is a coproduct there. Did I misunderstood your comment ?
Comment: @Roland Yes I mean infinite product in the definition of $F^a$. If it is infinite coproduct then finitely many of the $s_x$ in the element $(s_x)$ are non zero which would be bad for sections no?
Comment: @Chris As I said in the answer, this is not the coproduct in the category of abelian groups, this is the disjoint union (in other words, the coproduct in the category of sets). But read carefully the construction : $\mathcal{F}^a(U)$ is not the coproduct. This is rather the set of maps from $U$ into the coproduct which satisfies two conditions. The first one means that for each $x\in U, s(x)\in \mathcal{F}_x$ (where $\mathcal{F}_x$ is a subset of $\coprod_{x\in U}\mathcal{F}_x$ in the obvious way, but this is really a disjoint union here).
Comment: But this is true that $\mathcal{F}^a(U)$ can be seen as a subset of $\prod_{x\in U}\mathcal{F}_x$ (and this time I mean product). But, sections of $\mathcal{F}^a(U)$ are not maps from $U$ to the product. There are several ways to write this set. I used "map into the coproduct" way instead of the "product" way because this was closer to what the OP had in mind and also because the second condition is better written with this notation.
Answer: You have to use the topology of the etale space. It is the coarse topology such that for every $s\in {\cal F}(U)$, $s:U\rightarrow {}^a{\cal F}$ where $s(x)$ is the image $s_x$ of $s\in {\cal F}_x$ is continue. A base open subset of ${}^a{\cal F}$ is defined by $\{s_x\in {\cal F}(U)\}$ where $s$ is an element $s\in {\cal F}$.
Consider the map $f:{}^a{\cal F}_x\rightarrow {\cal F}_x$ such that for every $u\in {}^a{\cal F}_x$, take a section $s\in {}^a{\cal F}(V), V\subset U$ such that $u$ is the image of $s$ in ${}^a{\cal F}_x={\cal F}_x$. There exists $t\in {\cal F}(W)$ such that $u=t_x$. Write $f(u)=t_x$. $f$ is well-defined, if you consider $t'\in {\cal F}(W')$ such $t'_x=u=t_x$, there exists $A\subset W, A\subset W'$ such that the restriction of $t$ and $t'$ to $A$ are equal this implies that $f$ is well defined.
$f$ is surjective: for every $u\in {\cal F}_x$, consider $s\in {\cal F}(U)$ whose image in ${\cal F}_x$ is $u$, $t:U\rightarrow {}^a{\cal F}$ defined by $t(y)=s_y$ is an element of ${}^a{\cal F}(U)$, and the image of the element in ${}^a{\cal F}_x$ induced by $t$ by $f$ is $u$.
$f$ is injective. Suppose that $f(u)=f(v)=t_x$, without restricting the generality, we can suppose that there exists $s,s'\in{}^a{\cal F}(U)$ whose image in ${}^a{\cal F}_x$ is respectively $u$ and $v$. Remark that the map $p:U_t=\{t_x,x\in U\}\rightarrow U$ defined by $p(t_x)=x$ is injective. Write $W=s^{-1}(U_t)\cap{s'}^{-1}(U_t)$, it is an open subset which contains $x$, we deduce that the restriction of $s$ and $s'$ to $W$ are equal since $p$ is injective. This implies that $u=v$.
Answer: I use the notation in here.
We want to show that for any $x\in X$, $ \phi_x$ is bijection.
injectivity:
Let $ \phi_x(s_x)= \phi(t_x)$
(where $s\in \mathcal{F}(U), t\in \mathcal{F}(V)$).
Since $ \phi_x(s_x)=(\tilde{s})_x, \phi_x(t_x)=(\tilde{t})_x$, $\tilde{s}|_W=\tilde{t}|_W$ for some open neighbourhood $x\in W\subset U\cap V$ by definition of germ. So for any $p\in W$, $(p, s_p)=\tilde{s}(p)=\tilde{t}(p)=(p, t_p)$, therefore $s_p=t_p$. Especially, you get $s_x=t_x$,
considering a case $p=x$.
surjectivity:
Take arbitrary $s_x\in \mathcal{F}^{+}_x$
(where $s\in \mathcal{F}^{+}(U)$).
We write $s(x)=(x, t_x)$(where $t\in \mathcal{F}(V)$), then by continuity of $s$ on $U$, for some open neighbourhood $x\in W\subset U$ such that $s(W)\subset [V, t]$ since $[V, t]$ is open neighbourhood of $s(x)$. That is, for every $y\in W$, $s|_W(y)=s(y)=(y, t_y)$.
therefore we get $ \phi(V)(t)|_W=s|_W$ and,
\begin{align}
    s_x
    =( \phi(V)(t))_x
    = \phi_x(t_x)
    \in \operatorname{Im}  \phi_x.
\end{align}

On sheafification and stalks


Let $|E|_e$ denote the outer measure of a set $E$ in $\mathbb R^n$. If $E_k \nearrow E$, then $\lim_{k\to \infty} |E_k|_e = |E|_e$.

Question: Why does the author let $V_m = \bigcap_{k=m}^\infty H_k$? I think $V_m = H_m$ so there is no need to define $V_m$. Can we just use $H_m$ and let $H=\bigcup H_m$?
Answer: The point of taking the intersections $V_m = \bigcap_{k=m}^\infty H_k$ is so that the $V_m$ are increasing. Note that without any further assumptions, the $H_m$ are not necessarily increasing, even though they have the same outer measures as the $E_m$. Hence we do not necessarily have $H_m = V_m$. So we replace $H_m$ by the increasing sequence $V_m$ to apply Theorem 3.26.

Just as a simple example to see that the $H_m$ are not necessarily increasing, suppose that we are working in $\Bbb R$ and let $E_m = B_{1-1/m}(0)$, the ball of radius $1-1/m$ centered at $0$. Take $H_m = E_m\cup\{m\}$. Then clearly $H_m$ is measurable, and $|H_m| = |E_m|_e$, but $H_m\not\subset H_{m+1}$, so the sequence $H_m$ is not increasing. However, $V_m = E_m$ is increasing.
Comment: I'm not OP but I appreciated the example. Thanks :)


How to customize a content type css?

Question: I have a pdf sharing site with a content type which called PDF. This content type has many fileds (such as author, description, rating, etc). These fields display in simple rows but I want to change either their form and css. for example I want to show Rating number much bigger (and in other color) on the top of each post. what can I do?
Am I able to define new css classes for these fields?
Answer: create a new template file and add any classes you want. Here's a good introduction to D7 templates.
your template should be named like this: node--node_type.tpl.php. In your case, I think it should be node--pdf.tpl.php. You can use node.tpl.php as a starting point
Answer: One more way is to use a node preprocess hook in your template.php.
It might look something like
YOURTHEME_preprocess_node(&$var){
 if($var['type'] == 'PDF'){
   drupal_add_css(...);
  }
}

refer this for details

How to customize a content type css?


Android how to invisible my listview?

Question: I am using my own data base and get specific column and display listview. Here using addTextChangedListener method like this:
adapter = new ArrayAdapter<String>(this, R.layout.main , R.id.product_name, optionlist);
    listview.setAdapter(adapter);

 inputSearch.addTextChangedListener(new TextWatcher() {
   @Override
      public void onTextChanged(CharSequence cs, int arg1, int arg2, int arg3) {
            // When user changed the Text
         MainActivity.this.adapter.getFilter().filter(cs);
        }
    @Override
      public void beforeTextChanged(CharSequence arg0, int arg1, int arg2, int arg3) {
        }
        @Override
            public void afterTextChanged(Editable arg0) {
    }
});

My Question is: When I search data in edit text that time only specific data visible other wise no data display in my listview how to solve my problem here.
Here search time data visible like this othe wise no data visible

I want this no search no data visible in list view:

but here data visible like this:
Comment: it's better to clear the adapter than hiding listview.
Answer: That's difference between Filter and Search. You are filtering data, with no filter applied, everything is selected. Easiest would be to call setVisibility(GONE) on ListView when TextView is empty. Or you can place a boolean switch in Adapter to return item count as 0, when switch is set true.
Answer: add following to your listview in xml
android:cacheColorHint="#00000000"

Answer: Only add following to your java class:
@Override
        public void onTextChanged(CharSequence cs, int arg1, int arg2, int arg3) {
            // When user changed the Text

            lv.setVisibility(View.VISIBLE);
            MainActivity.this.adapter.getFilter().filter(cs);
            String str = inputSearch.getText().toString();
            if (str.equals("")) {
                lv.setVisibility(View.GONE);
            }
        }

Answer: Its better you clear the adapter and set empty string to adapter
@Override
    public void onTextChanged(CharSequence cs, int arg1, int arg2, int arg3) {
        // When user changed the Text

        lv.setVisibility(View.VISIBLE);
        MainActivity.this.adapter.getFilter().filter(cs);
        String str = inputSearch.getText().toString();
        if (str.equals(""))
        {
            List<String> list = new ArrayList<String>();
            list.add("");
            ArrayAdapter<String> adapter = new ArrayAdapter<String>(TestSpinner.this,R.layout.spinner_item,list);
            adapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
            listView.setAdapter(adapter);
        }
    }

Android how to invisible my listview?


CommunicationException: Erro no cliente ao fazer a solicitação HTTP (HTTP.SYS) a WebService de terceiros

Question: Estou desenvolvendo para o eSocial há mais de um ano já, e, estou encontrando o seguinte erro em alguns clientes, ao tentar fazer envio ao serviço:

System.ServiceModel.CommunicationException:
Erro ao fazer a solicitação HTTP para https://webservices.producaorestrita.esocial.gov.br/servicos/empregador/enviarloteeventos/WsEnviarLoteEventos.svc. Isso pode estar relacionado ao fato de o certificado de servidor não estar corretamente configurado com HTTP.SYS no caso HTTPS. Isso também pode ter sido causado por uma incompatibilidade da associação de segurança entre o cliente e o servidor.
  ---> System.Net.WebException: A conexão subjacente estava fechada: Erro inesperado em um envio.
      ---> System.IO.IOException: Não é possível ler os dados da conexão de transporte: Foi forçado o cancelamento de uma conexão existente pelo host remoto.
        ---> System.Net.Sockets.SocketException: Foi forçado o cancelamento de uma conexão existente pelo host remoto em System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)...

Na maioria dos computadores o programa está funcionando perfeitamente, só em alguns computadores esse problema acontece. Minha plataforma alvo é o .NET Framework 4.7.
Eu mesmo já respondi duas perguntas sobre esse erro aqui no SO:

(1) Problemas na comunicação com o webService disponibilizado pelo governo
  (2) Erro comunicação https com WebService

E, pelas pesquisas que eu fiz, há duas soluções para o problema:

Configurar a propriedade ServicePointManager.SecurityProtocol (https://stackoverflow.com/a/33084791/8133067);
Instalar as atualizações do Windows Update (https://stackoverflow.com/a/45494549).

No caso da primeira solução: O serviço do eSocial usa o protocolo de segurança TLS 1.2, mas nem sempre essa é a configuração padrão da máquina. Eu já fiz isso no código e não resolveu na maioria dos casos:
 ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls |
                                        SecurityProtocolType.Tls11 |
                                        SecurityProtocolType.Tls12;

No caso da segunda solução: Nas pesquisas que fiz o problema podia acontecer em computadores com Windows 7 desatualizado, e teoricamente a atualização via Windows Update resolveria, mas, na maioria dos casos não resolveu, e já peguei até alguns casos desse erro no Windows 10.
Eu até fiz um programa de testes, com o log de mensagens e o log de trace ligados, para ver se eu conseguia mais alguma informação, mas, a única coisa diferente que descobri (por causa do tracing) é que o erro ocorre quando a mensagem é enviada pelo canal (de comunicação com o WebService).
Segue o código que estou usando para instanciar e chamar o serviço:
 const string UrlSvcBase = @"https://webservices.{0}.esocial.gov.br/servicos/empregador/{1}";
 string urlServico = String.Format(UrlSvcBase, "producaorestrita", @"enviarloteeventos/WsEnviarLoteEventos.svc");

 var address = new EndpointAddress(urlServico);
 //var binding = new BasicHttpBinding(BasicHttpsSecurityMode.Transport);
 var binding = new BasicHttpsBinding();  //Disponível desde .NET Framework 4.5
 binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;
 // Manual de Orientação do Desenvolvedor v1.4, página 39, item '5.4.5. Validações aplicadas':
 // "O tamanho limite da mensagem SOAP é 750 kbytes."
 // O valor padrão da propriedade MaxReceivedMessageSize é 65.536,
 // que é alterado então para 750 KB * 1024 = 768.000 bytes.
 // Caso contrário, ocorre a exceção:
 //   Exception: System.ServiceModel.CommunicationException
 //   InnerException: System.Exception {System.ServiceModel.QuotaExceededException}
 //   "The maximum message size quota for incoming messages (65536) has been exceeded.
 //    To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element."
 //   HResult: -2146233087 (0x80131501)
 binding.MaxReceivedMessageSize = 768000;

 ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls |
                                        SecurityProtocolType.Tls11 |
                                        SecurityProtocolType.Tls12;

 var wsClient = new WsEnviar.ServicoEnviarLoteEventosClient(binding, address);
 // Variável 'certificado' é do tipo X509Certificate2.
 wsClient.ClientCredentials.ClientCertificate.Certificate = certificado;

 wsClient.Open();
 // Variavel 'lote' é do tipo XElement.
 var retorno = wsClient.EnviarLoteEventos(lote);
 wsClient.Close();

Alguém tem alguma ideia do que pode causar esse erro, ou alguma ideia do que mais posso fazer para tentar descobrir a causa do problema?
Comment: Aparentemente existe um bug no .Net, que ele não faz o fallback para as versões anteriores quando uma versão não é suportada. Experimente colocar manualmente a versão 1.0 e verifique se funciona. Eu inspecionei o pacote com o fiddler, o sistema deveria emitir 3 requisições de handshake em caso de erro, uma para cada versão, mas isso não está ocorrendo, na primeira com falha já estoura a exceção.
Comment: Boa tarde Pedro, também estou nesse projeto e estamos utilizando o .Tls.
Realizei um teste agora e não houve nenhum problema no envio, já tentou utilizar ele ? Ambientes testados: Windows 7, Windows Server 2008 R2
Comment: Oi Felippe, sim, fiz um teste ontem mesmo usando as opções *Tls | Tls11 | Tls12* e também não funcionou...
Comment: Editei a pergunta para incluir as novas opções que tentei na propriedade *ServicePointManager.SecurityProtocol*.
Comment: como está a sua references do ServicoEnviarLoteEventos ?
Comment: Não entendi muito bem a pergunta Felippe. Você quer saber como está o código que instancia e chama o serviço? Eu instancio um objeto da classe `ServicoEnviarLoteEventosClient`, passando o binding HTTPS e a URL, informo o certificado digital para conexão e então chamo o método `EnviarLoteEventos()` do objeto, passando o lote XML como argumento. Vou atualizar a pergunta para incluir o código também.
Comment: Existem algumas diferenças entre nossas implementações, na parte de configuração de url e de certificado.
Url eu popula no próprio ServicoEnviarLoteEventos, uma classe que tem na classe References.cs
Certificado também no próprio ServicoEnviarLoteEventos
Comment: Você poderia informar onde você leu que a versão do tls do servidor do eSocial é 1.2? Pois eu não encontrei isso nos manuais..
Comment: @EProgrammerNotFound, no manual do desenvolvedor realmente não está especificada versão, apenas diz que o serviço usa a tecnologia TLS. Eu cheguei à conclusão que era a versão 1.2 porque se você acessar a URL do serviço usando o IE, clicar com o botão direito do mouse sobre a página e clicar em Propriedades, vai aparecer a informação *Connection: TLS 1.2*; e também se você rodar um teste nesse site aqui: https://www.ssllabs.com/ssltest/analyze.html?d=webservices.producaorestrita.esocial.gov.br, o resultado dirá que o serviço está configurado para os protocolos TLS 1.0, 1.1 e 1.2!
Comment: @EProgrammerNotFound, mas eu acho que você estava certo! Eu fiz um teste no cliente que estava com esse problema, informando apenas o protocolo TLS 1.0 e nenhum mais: `ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;`, e funcionou!!! Se você escrever esse seu comentário em forma de resposta ficarei feliz em marca-la como resposta e como (muito) útil! Muito obrigado! Eu já estava há semanas me batendo com esse problema, e tinha até usado o Wireshark, mas não tinha chegado a conclusão nenhuma...
Comment: Estive estudando um pouco mais e aparentemente esse controle de fallback precisa de uma flag TLS_FALLBACK_SCSV, acontece que essa flag introduz um problema de segurança e por isso os servidores não a habilitam. Intuitivamente, acredito que é por isso que o .Net não testa as outras versões. Creio que deve ser feito manualmente como o amigo que postou a resposta ai em baixo.
Comment: Se você quiser estudar um pouco mais a respeito e postar uma resposta detalhada para ajudar a comunidade, segue alguns materiais interessantes: [rfc7507](https://tools.ietf.org/html/rfc7507) e [outra questão do cryptoexchange](https://crypto.stackexchange.com/questions/19673/how-does-tls-fallback-scsv-help) e [rfc5246](https://tools.ietf.org/html/rfc5246#appendix-E)
Answer: Conforme o comentário do @EProgrammerNotFound, parece ser um bug do .NET. Comigo acontecia a mesma coisa ao transmitir o E-Financeira ao Webservice SPED em algumas máquinas Windows 7 e Windows 2008.
Acabei criando uma rotina quecria uma Stack com os protocolos que quero utilizar (TLS 1.0, TLS 1.1. e TLS 1.2). Em seguida vou desempilhando e atribuindo um protocolo por vez e vou tentando enviar. Se falhar, passo para o próximo protocolo:
ServicePointManager.SecurityProtocol = (SecurityProtocolType) 0;

// Pilha com protocolos que vou utilizar
Stack<SecurityProtocolType> protocolosDisponiveis = new Stack<SecurityProtocolType>();
protocolosDisponiveis.Push(SecurityProtocolType.Tls);
protocolosDisponiveis.Push((SecurityProtocolType)768); // TLS 1.1
protocolosDisponiveis.Push((SecurityProtocolType)3072); // TLS 1.2.
var TentarNovamente = false;

do
{
    ServicePointManager.SecurityProtocol = this.protocolosDisponiveis.Pop();

    try
    {
        // Tenta chamar o serviço se der certo ok eu saio fora
        ChamarServico();
        TentarNovamente = false;
    }
    catch (CommunicationException ex)
    {
        TentarNovamente = true;

    }
}
while(TentarNovamente && protocolosDisponiveis.Count() > 0);

Comment: Legal Jonas, eu até pensei em fazer algo parecido, mas, devido à falta de tempo acabei, pelo menos por enquanto, só usando o TLS 1.0 sempre mesmo. Mas acho que vou fazer algo parecido com o que você fez, valeu! Eu só não entendi porque você remove todos os protocolos no início do código, um a um... não seria melhor ter feito `ServicePointManager.SecurityProtocol = SecurityProtocolType.SystemDefault` (*SystemDefault = 0*)? Dentro do loop, no caso de falha eu também faria a mesma coisa, já que só o protocolo atual estará presente na propriedade `SecurityProtocol`.
Comment: Também, como sugestão para deixar o seu código mais didático para quem possa parar aqui nesse post, você podia acrescentar uma linha do tipo `ChamarServico()` lá no bloco `Try`, para ficar bem claro que é ali que o serviço será chamado.
Comment: Na verdade, agora pensando, você não precisaria remover nem usando uma linha só, como eu sugeri, porque em cada iteração do loop você já joga um único protocolo na propriedade `ServicePointManager.SecurityProtocol`, então isso por si só já remove todos os outros! Aproveitando, deixa eu te perguntar, na versão do .NET que você está usando o enum `SecurityProtocolType` ainda não tinha os valores `SecurityProtocolType.Tls11` e `SecurityProtocolType.Tls12`?
Comment: Boa, Pedro! Vou seguir a dica e testar o comportamento. O tempo e prazo legal não permitiram que estudasse melhor as possibilidades. A ideia original em remover os protocolos atribuídos por padrão na máquina era evitar que o .NET tentasse estabelecer a conexão usando algum destes. Mas você tem razão. Obrigado.
Comment: Boa, Pedro. Testei aqui e funcionou seguindo a tua sugestão. Alterei o código exemplo acima. Sobre a versão do .NET, utilizamos o 4.0. E realmente ainda não possui as enumerações. Obrigado pelas dicas.


save/get data from map field type

Question: What I want to do is give a possibility to the user to save as many images as he likes. For this, I have created a specific field type
<code>    $fields['photo_ids'] = BaseFieldDefinition::create('map')
  ->setLabel(t('Photos'))
  ->setDescription(t('The photo ids of the Item entity.'))
  ->setSettings([
    'max_length' => 50,
    'text_processing' => 0,
  ])
  ->setDefaultValue('');
</code>
here I want to save, add, delete IDs of pictures. Not sure if this is the best approach to this problem. The problem is I can't find the way in Drupal 8 to serialize/deserialize and save these data into the database;
Comment: Sounds like a multi-value field?
Comment: yes I plan to store all picture ids in this field
Comment: Why don't you use the built-in field api and ui? Check "setCardinality" for your BaseFieldDefinition approach.
Comment: because I don't know about it :), could share an example or link of how to do it?
Comment: There are plenty on youtube. Search for something like "content type field create drupal 8".
Comment: The built-int field UI? You haven't seen that?
Comment: That all has a learning purpose, I am creating a new module and using entity type that's why I would like to do it programmatically.
Answer: You should not create your content type programmatically, if there's no special reason. Take a look at any intro like this https://www.youtube.com/watch?v=Nfu6nckjuno - then create a content type and fields with the excellent admin UI.
Serialization to DB is done by the Drupal core.

save/get data from map field type


Winforms / .Net - dynamically creating install .exe or script

Question: Does anyone know how I can Dynamically create an installation package that installs files into a folder that is determined at installation-runtime, not by the user, but by obtaining the UserAppDataPath for that specific user on that specific PC.
By dynamically I mean that I must create the installation package programatically as part of another process.
Any help ideas greatly appreciated!
Comment: What does "installs files into a folder" mean? If you want just to copy some files you could write a small console app and call that. If you want to register some COMs, again a small console app. If you want to create some default settings at a certain location... small console app. It depends on what "installs files" means.
Comment: Youu need to clarify what you mean by dynamically - in fact I think the overall objective needs to be a bit clearer, any decent installation tools will give you environment variables for things like the user app data path
Comment: thanks for comments , see comment below for clarification
Comment: thanks for comments, to clarify here (also see below): 'installs' - yes I should have said copies but I had Windows Installer on the brain at the time. Console App - I could use a console app or just a WSH script but was really looking for something where the user didn't have to unpack files and execute something - e.g. a septup.exe.
Answer: To build the installer package, you can use the WiX toolset. Your app could build the .wxs source file for it in the user's temp folder and invoke candle.exe (the WiX compiler) and light.exe (the WiX linker) to produce the proper .msi.
To get the location of the user's appdata path, use the Environment.GetFolderPath method with the ApplicationData parameter.
It does sound strange to me that you want to build the installer package on the user's machine. Unless you are building a tool that the user can use to generate installer packages from content they generated, there's no reason for you to do that. MSI (and the WiX toolset) allows you to build an installer package on your machine that would target the installing user's app data folder on their machine.
Comment: thanks, I will look at WiX - to clarify : files are output from an Expression Web 3 Add-in that in turn need to be distributed by the Add-In user to other users. We don't want those end-users to have to worry about unpacking a zip file and then finding the right .exe, WSH script or other app to run.  We wanted a pre-packed .exe that would simply extract the files and put them where they are supposed to go. The installer would not be built on the end-users machine but on the Add-In user's. Maybe thats clearer, maybe not.
Comment: That actually makes the whole scenario more clear. I would suggest you update the question with this description. Also, it is still unclear whether the package that was prepared by the add-in user needs to be uninstallable and whether it should be visible to the end-final user in the Add/Remove Programs dialog. (In other words - what's your clean up scenario)
Comment: The reason I am asking i because the WiX might be an overkill for your scenario, depending on the cleanup/uninstall scenario.


Como corro o Ooo4Kids no Ubuntu?

Question: O Ooo4Kids deixou de ser desenvolvido e ao tentar correr no Ubuntu (a partir do 15.04) obtenho o seguinte erro:

/usr/lib/ooo4kids-1.3/program/soffice.bin: /usr/lib/ooo4kids-1.3/program/> ../basis-link/ure-link/lib/libstdc++.so.6: version `CXXABI_1.3.8' not
  found (required by /usr/lib/x86_64-linux-gnu/libicuuc.so.55)

Obrigado,
Comment: Muchas gracias, no hay problema con este solucion ! merci beaucoup ! thanks a lot, it did it !
Answer: Esse problema ocorre porque o Ooo4Kids usa um libstdc++.so.6 próprio em vez de usar o do sistema, que seria o mais apropriado.
A forma mais simples de resolver o problema é apagar (ou mover) o libstdc++.so.6 do Ooo4Kids:
sudo mv /usr/lib/ooo4kids-1.3/basis-link/ure-link/lib/libstdc++.so.6 /usr/lib/ooo4kids-1.3/basis-link/ure-link/lib/libstdc++.so.6.old

e criar um link para o libstdc++.so.6 do sistema:
sudo ln -s /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /usr/lib/ooo4kids-1.3/basis-link/ure-link/lib/libstdc++.so.6

Isso deverá resolver o problema.

Como corro o Ooo4Kids no Ubuntu?


How to choose the right pick?

Question: How should I choose the right pick thickness and what are the differences between them?
I'm using a 0.5 m pick for extra light strings, but I don' t like it very much.
What is the best pick thickness for strumming ?
Comment: In the music store you can probably get an assortment of picks with all kinds of sizes en thicknesses. Maybe that's a good starting point to experiment.
Comment: acoustic or electric?
Comment: @Topomorto It's an Acoustic
Answer: This is entirely down to your own preference - there is no "best"
If you don't like a 0.5, try a 0.75 for a bit and see if you prefer that. I find I like different thickness picks for different types of playing on different guitars, but my choice is different to my friends' pick choices.
Answer: Pick style and thickness and material are largely a matter of personal preference.  You should try many different type picks to see what works best for you.  As your playing skills evolve, your choice of pick may change.
I will talk about different type picks but also want to touch on some ideas on how you might use a pick as part of your strumming technique because the way you hold the pick and the way you engage it with the strings will make a big difference in the overall level of satisfaction you experience with any pick.
Most beginners find it easier to strum with lighter gauge picks.  The lighter picks will flex as you strum down and then strum up. The lightest gauge pick I have ever found is a nylon Jim Dunlop pick in .38 gauge.   As you refine your strumming technique, you will develop the ability to flex your wrist and change the angle of attack as you alternate from down strum to up strum and you may develop a preference for the greater control of a medium gauge pick.
I find that for mostly strumming, a light to medium pick works best (for me). But for picking individual notes (especially fast) a heavier gauge works better because there is no recoil latency. I usually mix in strumming with picking out individual strings so unless it's a fast strumming only song - I usually use a medium to medium heavy pick. But I have been playing for awhile.
Besides gauge - picks come in many different materials.  Many popular picks are made of a material known as celluloid.  It can be flexible and is a little slippery.  Hard plastic is not a material I like for picks. Softer plastic picks are not bad in my opinion.  Nylon is a commonly used material for picks.
Picks also come in different shapes and sizes.  The most common size is the one that I seem to always gravitate to.  It's roughly the size of a US quarter but kind of a rounded triangle shape.  I have tried really large and really small picks but I always go back to "standard" size.
The degree of satisfaction you get from any pick will be highly influenced by your strumming technique and the way you hold your pick.  The best way that works for me to hold the pick is between thumb and first finger with the pick perpendicular (at a right angle) to my thumb with my first finger curled to point almost in the opposite direction of my thumb.  I leave about 1/3 of the pick exposed.
Holding the pick a little closer to the tip will help you learn not to insert it too deeply into the strings.  You just want to brush the strings with the tip of the pick so you aren't fighting against the strings so much.  It takes practice.
You want to develop a smooth strumming motion that is somewhat controlled.  This keeps the friction level consistent as you strum.  I encourage you to anchor your elbow on the top of the guitar but avoid the temptation to use your elbow as a "pivot point" for swinging your arm.
Instead, try to relax your wrist and use your wrist to strum up and down in a smaller arc - not your entire forearm. Try to imagine using a paintbrush to paint a wall with up and down strokes. You will alternate the angle of your wrist and the brush (pick) according to the direction your wrist is moving (up or down). Also, the smaller amount of movement required by keeping your forearm anchored and strumming with mostly wrist, will give you greater control.
Also using a lighter, relaxed grip on the pick (instead of a death grip) will allow it to flex a bit between your fingers. One thing that has helped me use a more relaxed grip on my pick when strumming is using picks with some type of texture on the grip which allows me to use a lighter grip without fear of the pick slipping out of my hands. My favorite pick for this is the Dunlop MaxGrip (better than the original textured dunlop picks) made out of nylon. Below is a picture Dunlop MaxGrip Pick. It's almost impossible for one of these to slip out of your hand - no matter how loosely you hold it. It comes in different gauges.

Good luck, have fun, and keep posting questions.
Comment: I play better without a pick , i don't know why , holding a pick make me feel nervous!
Comment: @Eva that's understandable.  It takes getting used to using a pick which means practice.  The first time we tried to eat with utensils (or chop sticks) it felt a little awkward, but once we got used to it - it becomes second nature.   But you don't have to use a pick if you choose not to.  If I try to strum without a pick for long I get cuts in my skin (darn steel strings) so I prefer a pick unless I'm finger picking and then I prefer bare fingers.
Answer: In my opinion, thickness of pick does not matter all that much. Maybe it's because of the extra light strings that you're using that you do not like the sound you're getting? Imho, string thickness changes your tone more than pick thickness. The answer that Rockin Cowboy wrote was really good and exhaustive, but was maybe overcomplicating things: just try out different types of picks and see which ones you like. [But in the end, we agreed though] Nowadays, I make my own picks by cutting them out of credit cards etc. (like this) just because it is very useful and I never have to buy picks any more and I never really mind thickness.
So:
1) maybe try using thicker strings
2) try using as much different picks as you can, just to see what you as a player like to handle the most
Comment: Agreed - that was the premise of my opening paragraph.  I suppose I could have left it at that but my suspicion is that if feel is the issue, technique will be part of the solution.  I tend to read between the lines (particularly with beginners) and provide additional information that I know might help the asker solve their problem (which is not always what they think it is).
Comment: Thickness of pick matters a great deal to many guitarist.  That's why they make picks from .38 all the way to 2.0 and everything in between.  There are 100's of  different types for different tastes, and purposes.  I can play with any pick just like I can play any guitar.  But I have very strong preferences and I use different picks for playing different things.  If you like the picks you make from credit cards - great!  In the long run it will save you money.  I have tried them and would only use them if there was no other choice.  I don't think tone is the issue. I think feel is the issue.
Comment: @RockinCowboy You're correct in saying that for some people thickness of pick does matter, but as you said, most of the time that's for the feel of it. I thought the OP was mainly focusing on the sound of it - so it's probably a different interpretation of the question at hand? But in the end the solution is clear: try everything!
Answer:
How should I choose the right pick thickness...?

The best way to choose the right pick for you (since it's all about personal preference and your particular playing style and guitar setup) is to buy a variety of single picks and try them out.
Go to a music store and ask about single picks. They often have a plastic box with dividers and a huge variety of picks organized for not much money per pick. Don't just buy random picks (well you could, but I think there's a more intelligent way to find the best pick). I would stick to the traditional pick shape. Buy a few different materials and/or manufacturers, and buy a range of thicknesses for each material/manufacturer.
For example, you might (or even should) try the Fender celluloid picks, but don't just buy a random Fender celluloid pick, instead, buy a range of thicknesses or buy one of each thickness. Thin, medium, and thick are probably the most popular ones. Extra thick is available but is pretty extreme.
Dunlop nylon picks are also very popular, and are available in six different "guages" (labelled by milimeters of thickness). Pick a style and buy one of each. Look for other options and perhaps spring for a few unusual picks if something catches your eye. It often helps to know what you don't like when trying to find what you do like.
When you get all the picks home, spend some time playing with each different kind. If you realize you like a material, stick with that for a bit and try different thicknesses, and/or vice-versa. You don't have to find the perfect pick on your first time out, but you will probably at least have a favorite one out of the selection you purchased. That's your new baseline, and you can try out a few other picks as time goes on and either make a change or stick with your first favorite. You'll probably know it when you find the one.

...and what are the differences between them?

Assuming we only talk about the standard shaped picks, the material and the thickness control the overall stiffness of the pick. That's the primary difference between different picks.
Picks that are more stiff (thicker, harder material):

Can play a bit louder
Provide more resistance and feedback to your fingers
Can be a little more difficult to play - especially chords
Tend to give a fuller sound

Picks that are less stiff (thinner, softer material):

Are usually easier to strum chords with
Feel more spongy
Offer less of range of loudness
Tend to give a thinner sound

The shape of the pick does matter. One advantage with the traditional pick shape is it actually lets you use different shapes all in one pick. Normally people play with the pointy end, but you can flip the pick around and either play with the rounded end, or you can play with one of the corners of the rounded end (the last one is how Eric Johnson uses a traditional shaped pick). If you learn to use the traditional shape, it will give you the best place to start when selecting other picks.

How to choose the right pick?


Comparar duas Lista

Question: Tenho uma lista
List<Comentarios> comentarios
List<Comentarios> comentariosSP

A primeira lista eu recebo os dados de um webserver, a segunda de um SharedPreferences.
Eu tentei compara-las
comentarios.equals(comentariosSP)

mas ele sempre retorna false, mesmo eu sabendo que as listas são idênticas.
lembrando que eu preciso verificar se os dados dos atributos são os mesmos. EX:
comentarios.get(0).getComentario().equals(comentariosSP.get(0).getComentario())

Tem como fazer isso sem percorrer toda a lista?
Answer: Você pode usar o método equals() da List se sobre-escrever o método equals() da classe Comentarios de forma a indicar quando dois objectos Cometarios são iguais:
@Override
public boolean equals(Object o) {
    // verifica se é o mesmo objecto
    if (this == o)
        return true;
    // verifica se é null
    if (o == null)
        return false;
    // verifica tipo e cast
    if (getClass() != o.getClass())
        return false;

    Comentarios comentarios = (Comentarios) o;
    // Comparação atributo a atributo
    // Note que cada um dos atributos têm também de implementar correctamente o método equals()
    return Objects.equals(atributo1, comentarios.atributo1)
            && Objects.equals(atributo2, comentarios.atributo2) && .....;
}

Nota: Duas listas são iguais se tiverem o mesmo número de itens, iguais e  pela mesma ordem.
Comment: Muito obrigado! serviu como uma luva para mim!
Answer: Quando você compara com o método comentarios.equals(comentariosSP), você está comparando o endereço de memória em que a lista está alocada.
utilizando o equals() da List você compara os valores em sí.

Comparar duas Lista


Show that map $p$ is continuous

Question: We have $Y \subset C[0,1]$ be the set of continuous functions on $[0,1]$ which are polynomials of degree $\leq 2$, i.e. $ax^2+bx+c$.
Consider map $p:\mathbb{R}^3_\text{usual} \to Y$ mapping $(a,b,c)$ to $f(x) = ax^2+bx+c$
Now we want to show that the maps $p:\mathbb{R}^3_\text{usual} \to Y_{L_1}$ and $p:\mathbb{R}^3_\text{usual} \to Y_{L_\infty}$ are continuous. (Here, $Y_{L_1}$ is the  under the $L_1$ topology.)
I tried doing $p^{-1}(B(f,\epsilon)) = \{ (a,b,c):\int_0^1 \lvert f(t) - at^2-bt-c \rvert < \epsilon \}$ but I don't know how to proceed.
Answer: Note that $p$ is a linear map, so it suffices to show $p$ is bounded under the two norms. That is, we need to find constants $K_1$ and $K_\infty$ such that
\begin{align*}
\|p(a, b, c)\|_1 &\le K_1\|(a, b, c)\| \\
\|p(a, b, c)\|_\infty &\le K_\infty\|(a, b, c)\|.
\end{align*}
We have,
$$\|p(a, b, c)\|_1 = \int_0^1 |ax^2 + bx + c| \, \mathrm{d}x \le \int_0^1 |a|x^2 + |b|x + |c| \, \mathrm{d}x = \frac{|a|}{3} + \frac{|b|}{2} + |c|.$$
Since $|a|, |b|, |c| \le \sqrt{a^2 + b^2 + c^2}$, we have
$$\|p(a, b, c)\|_1 \le \frac{\|(a, b, c)\|}{3} + \frac{\|(a, b, c)\|}{2} + \|(a, b, c)\| = \frac{11}{6}\|(a, b, c)\|,$$
so we may choose $K_1 = \frac{11}{6}$, i.e. $p$ is continuous as a map to $Y$ under the $L^1$ norm.
I'll let you think about the $L^\infty$ norm. Let me know if you need more help.

Show that map $p$ is continuous


How can i remove title requirement for content type

Question: For example i have content type article i need to remove requirement for title, right now its a Node module element and i can not edit it. I found module title but when i installed id and tried to change title, my site crashed.
Is there any solution to achieve this?
Should i remove title on content type edit page and replace it with editable field?
UPDATE:
Some of you maybe do not understand me right). So i have content type Answer and this c.type has two fields title and body. When someone want to answer atm he needs to fill title field every time he wants to answer, but i need to give user a choice to just fill only body field OR body and title if user want to fill the title. The title field should be displayed on node edit every time!
So only way i see viable is to create new field and  hide title with automatic-node-title module...
Comment: Why do you want to remove the requirement? Do you just need to hide the title or remove it completely?
Comment: i have a content type answer, so if user want to answer he can not add a titl, i dont need to remove it
Comment: What @jan said, i'd recommend auto-nodetitle module then as well...
Comment: @neok, I agree with jan. You can use [Auto Node title](http://drupal.org/project/auto_nodetitle) and you can hide the title in the node view page using template override or `display:none` from css.
Comment: i updated my question
Answer: I dont' think you can entirely remove the title from the nodes but you may hide it and automatically populate it, as is below:
Auto node titles and Title override
I have used the first one before.
Comment: i dont need to hide it, just need to remove requirement, but gonna check those modules
Comment: I think the title field in the node table schema is not allowed to accept null values, which would be a possibilty if you'd remove the requirement. So hiding the title and use it just in the backend would be the way to go...
Comment: i updated my question, you can see what i am actually need, i believe this is not the goal i am looking for.
Comment: Your requirement is not common. As what @Volker pointed out it cannot accept null values. Perhaps you can pull this off using a custom module, similar to auto node title but with a little extra -- toggles title to hide/unhide as required.
Comment: ok, this module has helped me.
Comment: Did you manage to hide/unhide the title field on demand? It might help us in the future as well.
Answer: How to hide or remove title for content type.
1 Install Automatic Entity Label
2 Install Token
For Content Type News go to
D7 admin/structure/types/manage/news/auto_label
D8/9 admin/structure/types/manage/news/auto-label
Then select an option that fit your solution, in my case I selected:
Automatically generate the label and hide the label field
In the Pattern for the label enter field token to replace the Title.
in my case I entered [node:filed_slogan].


How to discriminate between non-nested models?

Question:
So first this question asks "how I would discriminate between model(1) and model(2)".
It appears that both models are non-nested so I would come up with a hybrid model consisting of Xt,Zt and Qt regressors. I know I need to test for Zt=0 and Qt=0 but I'm not sure if I need to test for Xt=0. My logic tells me no as the point is to compare model(1) and model(2), since they both contain Xt as regressor this won't tell me if model(1) is better than model(2) and vice-versa. But someone please correct me if I'm wrong.
For the second part where: Given the available information, would it be desirable for you to come up with a third and dominant model?
First a third model would be desirable only if model(1) and model(2) are not "good" or we would want a better model. But from the way the question is formulated it cannot be the second one, so I believe I should test each individual models to see if they are "good".
For this part the only thing I can think about is doing the following for each equation:
t-test(Individual Significance Test)
F-test(Joint Significance Test)
Apart from these two tests there aren't really any other information that can be used to tests the models.
So if model(1) and model(2) fail any of these tests then a third and dominant model would be desirable. Is my understanding correct?
Answer: I would look also at the efficiency of the estimators, i.e. at their variance-covariance matrix. Efficiency is important since it measures the uncertainty characterizing the point estimates.
The two models self-select with respect to a single regressor. So it is a duel between the two regressors. Put them in the arena together, and see who remains alive.
That's the "hybrid" model you mention - used here as a "testing device" rather than as a nesting model. If one of the two regressors becomes statistically insignificant in the presence of the other, I would go with the model that contains the second -it is more strongly engaged with the dependent variable.
Since this enhanced model nests the initial two, if you are prepared to make some distributional assumption on the error terms, you can conduct straightforward Likelihood Ratio Tests, pairwise.
Also, there are formal procedures to compare "overlapping" models like yours (strictly speaking, they are midway between "nested" and "non-nested" models, since they have an explanatory variable in common), but require distributional assumptions for the distribution of the error terms in the two models (conditional on the regressors). You can consult Vuong 1989. Also, this paper by Lewis et al (2011) seems to be more focused on practical implementation of Vuong's tests.
Model selection is an open-ended task - I would not be surprised if some other answer proposes a completely different approach.
Comment: so I only need to test for Zt=0 and Qt=0 right?There is no need to test for Xt=0? "The two models self-discriminate with respect to a single regressor" What does this mean? Can you check if I'm right for the second part?
Comment: In the context of the enhanced specification containing all three variables, you should also verify that $X_t remains statistically significant. About the  sentence: it should be "self-select", I changed it. About the "second part": Ιf in the enhanced model, all variables remain statistically significant, and things like estimator variance, or adjusted R-square do not get worse, I would go with it rather than the reduced models. All three variables can be reasoned to affect FTSE returns, and going form two to three regressors does not really create issues of parsimony.
Comment: But given the available information I won't be able to determine if the enchanced model is better. The question does state that I must reach a conclusion with the available information. @Alecos
Comment: Indeed, but what is the "available information"? The estimated coefficients, the SE, and the R-squared only?
Comment: Yes, available information is everything given above. There is nothing else. The only thing possible are t-tests and F-tests. Is that right? @Alecos
Comment: To perform an F-test, you need the _unadjusted_ $R^2$ (the coefficient of determination) to express the F-statistic. To obtain the coefficient of determination through that adjusted $R^2$, which is what you are given, you need to know the size of the sample. Is the sample size part of the "available information"? If yes, you can also perform F-test, and not only t-tests. If for both models the test rejects the null of zero coefficient, you should find the p-values of the statistic in order to be able to say something discriminating
Comment: Well the questions says it's monthly returns and from Feb 1995 to Nov 2012, so there are 214 observations. If I carry out an F-test, is it worth carrying out t-tests?
Comment: Yes it is, for a more complete picture.


Why doesn't Rspec like my routes?

Question: In my application I have the concept of a topic.  A user can watch a topic.  I've created both a WatchedController and a TopicController to handle this.  Here's my routes:
resources :topics, :only => [:show, :index] do
  resource :watched, :only => [:create, :destroy]
end

Here's my failing sample Rspec test:
it "failing test" do
  post :create, :topic_id => @topic.id
end

When I run the spec, Rspec complains:
Failure/Error: post 'create', :topic_id => @topic.id
ActionController::RoutingError:
   No route matches {:topic_id=>1, :controller=>"watched", :action=>"create"}

I've wasted an hour on this problem and I still can't figure out why it's not working.  Any help?
Answer: Run rake routes at the command line to see what routes you have succeeded in defining, then either fix your route definitions to make sense or fix the code in your spec so it matches one of the routes in that list.
Comment: I didn't know that command existed.  It turned out watched was being pluralized to watcheds.  Thanks for the help.

Why doesn't Rspec like my routes?


If $A$ and $B$ are lines in $\mathbf{CP}^3$ and $P$ is a point in $\mathbf{P}^3$, is there a line incident to $A,B,P$?

Question: I want to figure out the following: If $A$ and $B$ are lines in $\mathbf{CP}^3$ and $P$ is a point in $\mathbf{P}^3$ then is it always possible to find a line in $\mathbf{CP}^3$ which meets all of $A,B,P$? I feel that it should be yes, but I fail to translate the problem into a problem in $\mathbf{C}^4$, where I could perhaps use some dimension argument (I mean some linear algebra, the same way as one proves that every two lines in $\mathbf{CP}^2$ intersect). Any ideas?
Answer: Consider that the point $P$ does not lie of the first line $l_1$. Then there is a plane passing through this line and the point. This plane then must intersect the second line $l_2$ in at least one point. Now the line through $P$ and the point of intersection is your desired line.
Answer: Hint. The set of lines in $\mathbf P^3$ which go through your point $P$ is a $\mathbf P^2$. Can you translate the condition «passes through some point of $A$» in terms of that $\mathbf P^2$?
Comment: It is safe to assume that $P$ is not on $A$ for this, as otherwise the problem is trivial.
Answer: For these questions, it is often better to try to think directly, in a geometric way, in $\mathbb P^3$, rather than to translate into a question about $\mathbb C^4$.
In this case, a useful way to think is to consider the span of $A$ and $P$,
which is a plane in $\mathbb P^3$. By definition this is the union of all the lines joining $P$ to a point of $A$, so any line passing through $P$ and meeting $A$ lies in this plane.  In particular, the line you are looking for, if it exists, lies in this plane.
If the line you are looking for exists, then its intersection with $B$ lies both on $B$ and in this plane.  So the next step is to think about how $B$ meets this plane.  Once you've gotten this clear, you should be able to see whether or not you can construct the line you want.
(Note: Mariano's answer gives a more abstract formulation of this approach,
while Rene's answer follows exactly this approach, but is terser.)


Observer - Observable model around an email receiving logic

Question: I have an email receiving logic written (class name: Mail_Receive_Logic). This class downloads all my unread emails into a messages array.
I also have classes EmailStatPrinter and EmailStatRecorder classes. EmailStatPrinter has a writeToConsole method which prints the array onto screen. EmailStatRecorder has a writeToFile method which writes the array in a text file.
I want to implement this logic in an observer-observable manner.
public class Mail_Receive_Logic extends Observable{
public class EmailStatPrinter implements Observer{
public class EmailStatRecorder implements Observer{

My main method looks like this
public static void main(String[] args) throws ClassNotFoundException {

    // observable
    Mail_Receive_Logic receiveMail = new Mail_Receive_Logic(usernameReceiving, passwordReceiving, fileReceivePath);

    // observer
    EmailStatPrinter writeToConsole = new EmailStatPrinter();
    EmailStatRecorder writeToFile = new EmailStatRecorder();

    receiveMail.addObserver(writeToConsole);
    receiveMail.addObserver(writeToFile);

    Thread receive = new Thread (new Runnable() {
        public void run() {
            try {
                while (true) {
                    receiveMail.receive();
                }
            } catch (InterruptedException e) {}
        }
    });

    receive.start();

How should I write the update method of each observer classes such that they take in messages array and print it on console and text file. i.e I want to know what to write in following code segment for each observer classes.
@Override
    public void update(Observable o, Object arg) {
        // TODO Auto-generated method stub

    }

Please also explain me that why is there 2 variables Observable o and Object arg?
Answer: You have to call Observable.notifyObservers() and supply the arg. It can be anything you like. In your case I think you would want something like:
try {
    while (true) {
        receiveMail.receive();
        receiveMail.notifyObservers(messages);
    }
}
catch (InterruptedException e) {
}

Then in your Observers you would receive the messages as:
@Override
public void update(Observable o, Object arg) {
    // The first arg is the thing you were observing (receiveMail in this case)
    // The second arg is the messages that was passed by the receiveMail instance
}

Comment: In my Mail_Receive_Logic class (i.e the class belongs to receiveMail object), I have wrote a line `notifyObservers(messages);`. Does it wrong? or do I have to write like you did in main class?
Comment: I think you want call `receiveMail.notifyObservers(messages)` right after `receiveMail.receive()` as I am assuming that's how you get your `messages` object, and thus that's where you will have visibility to pass the messages to your observers.


help with regex - extracting text

Question: Suppose I have some text files (f1.txt, f2.txt, ...) that looks something like
@article {paper1,
author = {some author},
title = {some {T}itle} ,
journal = {journal},
volume = {16},
number = {4},
publisher = {John Wiley & Sons, Ltd.},
issn = {some number},
url = {some url},
doi = {some number},
pages = {1},
year = {1997},
}

I want to extract the content of title and store it in a bash variable (call it $title), that is, "some {T}itle" in the example.  Notice that there may be curly braces in the first set of braces.  Also, there might not be white space around "=", and there may be more white spaces before "title".
Thanks so much.  I just need a working example of how to extract this and I can extract the other stuff.
Answer: Give this a try:
title=$(sed -n '/^[[:blank:]]*title[[:blank:]]*=[[:blank:]]*{/ {s///; s/}[^}]*$//p}' inputfile)

Explanation:

/^[[:blank:]]*title[[:blank:]]*=[[:blank:]]*{/ { - If a line matches this regex

s/// - delete the matched portion
s/}[^}]*$//p - delete the last closing curly brace and every character that's not a closing curly brace until the end of the line and print

} - end if
Comment: +1 After I changed my script to take care of the case where a comma could exist in the value I got exactly your script.  I've told the OP to accept yours but I think you should wrap it `title=$(sed ...)` to completely meet his requirements.
Answer: title=$(sed -n '/title *=/{s/^[^{]*{\([^,]*\),.*$/\1/;s/} *$//p}' ./f1.txt)

/title *=/: Only act upon lines which have the word 'title' followed by a '=' after an arbitrary number of spaces
s/^[^{]*{\([^,]*\),.*$/\1/: From the beginning of the line look for the first '{' character. From that point save everything you find until you hit a comma ','. Replace the entire line with everything you saved
s/} *$//p: strip off the trailing brace '}' along with any spaces and print the result.
title=$(sed -n ... ): save the result of the above 3 steps in the bash variable named title
Comment: Thanks.  However, what if there are commas in what I want to capture?  Then it doesn't work because of the "first comma" logic.  How can we incorporate the flexibility of additional commas?  Thanks
Comment: @Vinh you should accept Dennis' answer because after I changed my script to take care of the case where a comma could be in the name, I got exactly his script.  The only thing you need to do is wrap his script in `$()` like mine is.
Answer: There are definitely more elegant ways, but at 2:40AM:
title=`cat test | grep "^\s*title\s*=\s*" | sed 's/^\s*title\s*=\s*{?//' | sed 's/}?\s*,\s*$//'`

Grep for the line that interests us, strip everything up to and including the opening curly, then strip everything from the last curly to the end of the line

help with regex - extracting text


How to do sum with where clause group by with date?

Question: I want to sum up where clause. I don't know how to show off_duration in the next column.
SELECT CAST([RecordDateTime] AS DATE) AS DATE
    ,SUM(CAST([units] AS FLOAT)) AS Units
    ,sum(cast(duration AS INT) / 60) on_duration
FROM [Energies]
WHERE duration_mode = 'ON'
GROUP BY CAST([RecordDateTime] AS DATE)
ORDER BY CAST([RecordDateTime] AS DATE) DESC

op
 Datet      units   on_duration
-------------------------------
2020-01-17  3.53    758
2020-01-16  7.66    973
2020-01-15  15.12   1806
2020-01-13  10.4    500

ex op
date        units   on_duration    off_duration
-----------------------------------------------
2020-01-17  3.53    758            28
2020-01-16  7.66    973             9
2020-01-15  15.12   1806           96
2020-01-13  10.4    500            95

sample data
duration_mode   duration    RecordDateTime             units
-------------------------------------------------------------
ON              187     2020-01-07 20:18:33.9744232     0.19
ON              187     2020-01-07 20:19:03.1554359     0.19
OFF             10      2020-01-07 20:22:13.5283932     0.00
ON              187     2020-01-07 20:24:39.0510166     0.19

Comment: `sum(case when duration_mode = 'OFF' then duration end)` Remove the filter to get both rows in the group.
Comment: I don't understand your logic here. Explain your results. What does `off_duration` represent, and how is it calculated (from your sample data). I *thought* it might be to do with rows where the value of `duration_mode` is `'OFF'`, however, you filter those out in the `WHERE`; so it can't be that.
Answer: I think that you are looking for conditional aggregation:
SELECT
    CAST([RecordDateTime] AS DATE) as Date,
    SUM(CAST([units] as float)) as Units,
    SUM(CASE WHEN duration_mode = 'ON' THEN CAST(duration as int)/60 END) on_duration,
    SUM(CASE WHEN duration_mode = 'OFF' THEN CAST(duration as int)/60 END) off_duration
FROM [Energies]
GROUP BY CAST([RecordDateTime] AS DATE)
ORDER BY CAST([RecordDateTime] AS DATE) desc

Comment: can u please tell me , how to do trim upto 2 decimal,7.87999999999999 exp op - 7.87 in the units.it should be another question but for time being.
Comment: @krishnamohan: use `round(..., 2)`.


Find a limit $\lim_{x \to - \infty} \left(\frac{4^{x+2}- 2\cdot3^{-x}}{4^{-x}+2\cdot3^{x+1}}\right)$

Question: I am to find the limit of
$$\lim_{x \to - \infty} \left(\frac{4^{x+2}- 2\cdot3^{-x}}{4^{-x}+2\cdot3^{x+1}}\right)$$
so I used:
$$\lim_{x \to -\infty} = \lim_{x \to \infty}f(-x)$$
but I just can't solve it to the end...
Please show me all steps, or at least most of them, so I'll know how to solve it. Thank you.
This question was posted on: Find $\lim_{x \to - \infty} \left(\frac{4^{x+2}- 2\cdot3^{-x}}{4^{-x}+2\cdot3^{x+1}}\right)$, and got 3 answers, but I still don't know how should I solve it, because when I try to solve it (with help of those 3 answers) I get :
$$0−12/0$$ every time and that goes to minus infinity...
Comment: I think answer is 0 because وDenominator is greater than numerator
Comment: which part of the solution do you not understand ?
Comment: Well, there were only hints and neither I understood and I tried, believe me.
Comment: I just need to know how to get that minus infinity to plus infinity, so what you get when you convert this limit so it goes to plus infinity and with what I need to multiply or divide that limit in order to solve it to the end. That's all.
Answer: Numerator: $4^{x+2}=\varepsilon_1(x) \to 0$ is always positive.
Denominator: $2 \cdot 3^{x+1} = \varepsilon_2(x) \to 0$ is also always positive and we can choose an arbitrary constant $c_1$ such that
$$
\lim_{x \to -\infty} \frac{4^{x+2} - 2 \cdot 3^{-x}}{4^{-x} +2 \cdot 3^{x+1}}> \lim_{x \to -\infty}\frac{-2 \cdot 3^{-x}}{c_1 4^{-x}}=0
$$
Using the same logic for the upper bound, we get
$$
\lim_{x \to -\infty} \frac{4^{x+2} - 2 \cdot 3^{-x}}{4^{-x} +2 \cdot 3^{x+1}}< \lim_{x \to -\infty}\frac{-2 c_2 \cdot 3^{-x}}{4^{-x}}=0
$$
By squeeze lemma, the limit is 0.
Answer: First Method
For $x\to -\infty$, $4^{x+2}\sim 0$ and $3^{x+1}\sim 0$ so
$$
\frac{4^{x+2}-2\cdot 3^{-x}}{4^{-x}+2\cdot3^{x+1}}\sim \frac{-2\cdot3^{-x}}{4^{-x}}=-2\left(\frac{4}{3}\right)^{x}\to 0\qquad\text{for}\; x\to-\infty
$$
Second Method
$$
\frac{4^{x+2}-2\cdot 3^{-x}}{4^{-x}+2\cdot3^{x+1}}=\frac{3^{-x}}{4^{-x}}\frac{(3\cdot4)^x4^{2}-2}{1+2\cdot 3(3\cdot 4)^x}=\left(\frac{4}{3}\right)^{x}\frac{-2+8\cdot12^x}{1+6\cdot 12^x}\to0 \qquad\text{for}\; x\to-\infty
$$
Answer: $$\left(\frac{4^{x+2}- 2\cdot3^{-x}}{4^{-x}+2\cdot3^{x+1}}\right)$$
$$\left(\frac{4^{x}*16- 2\cdot3^{-x}}{4^{-x}+2\cdot3\cdot3^{x}}\right)$$
$$\left(\frac{4^{x}*16- 2\cdot\frac{1}{3^x}}{\frac{1}{4^x}+2\cdot3\cdot3^{x}}\right)$$
$$\left(\frac{16\cdot4^{x}\cdot3^x- 2 }{3^x}\right)\cdot\left(\frac{4^x}{1+6\cdot3^x\cdot4^x}\right)$$
$$\left(\frac{16\cdot4^{x}\cdot3^x- 2 }{1+6\cdot3^x\cdot4^x}\right)\cdot\left(\frac{4^x}{3^x}\right)$$
now when you put $x \rightarrow -\infty$ the $4^x/3^x$ tends to $0$ and the limit tends to $0$
As for $$\left(\frac{16\cdot4^{x}\cdot3^x- 2 }{1+6\cdot3^x\cdot4^x}\right)$$ the limit is finite value which is ultimately gonna multipled to $0$
Comment: How can be $4^{x}/8$ the same as $4^{x+2}$. I thought $4^{x+2}=4^{x}*16$.
Comment: Oh sorry . I will correct it !


Chessboard filled with numbers, having same sum always if exactly $1$ number is taken from each row and each column

Question: Consider the squares of an $8\times 8$ chessboard filled with the numbers $1$ to $64.$ If we choose $8$ squares with the property that there is exactly one from each row and exactly one from each coloumnwise, and add up the numbers in the chosen squares, show that the sum obtained is always $260.$
Numbers filled the squares as in the figure below.
https://1drv.ms/w/s!AsgThsrHfCMVhns1x3plDZjTFI5O
Comment: The accepted answer in @RobertZ's link does prove that. Are you asking something you did not understand in that answer?
Comment: Nah question is to show it will be 260 always
Comment: +1 to the *style* of OP's reactive comment.  Although I disagree with the underlying math analysis inherent in his reactive comment, I must show appreciation for his syntax.
Comment: @user2661923 Not a good way to handle it. Questions should be formulated out in the body, they should not be dependent on a link. A "good style" whatever this means here, does not compensate that.
Comment: @Peter I'm not sure what you are saying here.  If you are indicating a disagreement with the OP's reaction : I have found that if you attempt to criticize the OP directly, they will often "dig their heels in".  If instead, you go for "light satire" the OP may not be so defensive.
Answer: just think of how the number will be chosen
if you fix the row first, see the number you take will be of form {the $8$'th column number $-a$} where $a$=$0,1,2,3....7$
so the sum will be $(8-x_1$)+$(16-x_2$)$+(24-x_3$)+$(32-x_4$)+$(40-x_5$)+$(48-x_6$)+$(56-x_7$)+$(64-x_8$)
now see all $x_i$'s are distinct and take values from {$0,1,2,3,4,5,6,7$}
then the sum is $260$
In fact, if you think for $n×n$ chessboard the answer will be $n(n^2+1)/2$
you can read this article here
https://www.awesomemath.org/wp-pdf-files/math-reflections/mr-2020-05/mr_5_2020_counting_with_aid_of_cyclotomic_polynomials.pdf
Comment: can I ask ,are you solving 10+2?


Real vector bundles on $S^{7}$

Question: Is it true  that  $\pi_{6}(O(n))=0$ for all n?
Equivalently,  are all real  bundles on $S^{7}$ trivial?
Comment: Just some comments that you are no doubt aware of, but than can be useful to some:
1. Because $SO(n)$ is the connected component of $O(n)$ containing the identity, $\pi_k SO(n) = \pi_k O(n)$.
2. Because of Bott periodicity theorem, $\pi_6 SO(n) = 0$ forall $n \geq 8$.
3. $SO(1)$ is trivial, $SO(2)$ is a circle: no $\pi_6$ there.
Conclusion: the question boils down to: "is $\pi_6 SO(n)$ nontrivial for some $n \in \{3,4,5,6,7\}$?"
Answer: Because of the covering space $SU(2) \to SO(3)$ given by the quaternions (cf. this document, for example), we know that the universal cover of $SO(3)$ is homeomorphic to $SU(2) \simeq S^3$.
This implies that $\pi_6 SO(3) \simeq \pi_6 S^3 \simeq \mathbb Z/12$ (according to that table).
So there is a nontrivial rank 3 bundle on $S^7$.

Once the few remarks I made in my comment are clear, this article by Michel Kervaire gives the definitive answer (p. 162):
$$\pi_6 SO(3) \simeq \mathbb Z/12\qquad \pi_6 SO(4) \simeq (\mathbb Z/12)^2\qquad \pi_6 SO(5) = \pi_6 SO(6) = \pi_6 SO(7) = 0.$$
Note that $\pi_6SO(4)$ also comes from the quaternions: indeed, they also give the universal cover $SU(2) \times SU(2) \to SO(4)$, so $\pi_6(SO(4)) \simeq (\pi_6(S^3))^2 \simeq (\mathbb Z/12)^2$.
Comment: Many  thanks  for  your interesting answer and  useful comment.

Real vector bundles on $S^{7}$


Alternate to field_get_items() ?? CCK fields reconstriction?

Question: After a long quest, finally I got the solution for splitting $content in Drupal 7 in node.tpl.php.
Previously i was using i was using :-
print $node->field_name['und'][0]['value'];

Which was giving a warning :-
Notice: Undefined offset: 0 in include() (line 24 of C:\xampp\htdocs\drup\sites\all\themes\myCustomTheme\node.tpl.php)

Now I am using the function below which works fine with no errors
$output= field_get_items('node', $node, 'field_name');
$output = $output[0]['safe_value'];
print $output;

Now the problem is I have more than 50 fields on the page , I don't think it would be efficient to call field_get_items('node', $node, 'field_name') function 50 times.
Whats the alternate ?
OR should I strict to the field_get_items function
OR should I strict to the print $node->field_name['und'][0]['value']; which I heard is not good to use in Drupal 7 as und is undefined. This though solves my problem but gives irritating warnings.
Answer: First off, 'und' is the value of the LANGUAGE_NONE constant, which is used for the content not associated to a particular language. Not all the fields have a value for that, and if you are trying to access it, you get a PHP warning.
For example, for taxonomy terms, the array structure could be one of the following.

tags

other taxonomy terms

If you need to get the field value that would be used for the currently used language, you can just use field_get_items(). Its code cannot be optimized, as it needs to determinate the language used for the field.
function field_get_items($entity_type, $entity, $field_name, $langcode = NULL) {
  $langcode = field_language($entity_type, $entity, $field_name, $langcode);
  return isset($entity->{$field_name}[$langcode]) ? $entity->{$field_name}[$langcode] : FALSE;
}

Even the code for field_language() cannot be much optimized. You could avoid multiple calls to field_valid_language(), but that is the only thing you can do.
If you have your own code to decide which language should be used for the fields, you can access the field data directly, without using field_get_items().
Answer: You can install the EntityAPI module and use a meta-data wrapper to ease accessing Node fields. I believe it loads the object once and just references the field data when you want it.
You can also try the field_extract module.
I believe Drupal's entity_load() and default Drupal object caching makes your concern of calling field_get_items(), or the entity API or field_extract a non-issue as the node is retrieved once and then that cached node data is re-used for the lifetime of the page.
Give both a looksee; your mileage may vary.


Confidence interval for poisson distributed data

Question: I'd like to calculate the confidence interval for a variable with a lower bound and I can't seem to figure out how to do so. I've seen several similar posts, but none answered my question, at least not directly. I use a toy example here for simplicity.
To be concrete lets consider the distribution (I chose these numbers myself so it kind of looks like a Poisson distribution),
<code>{1,2,3,5,1,2,2,3,7,2,3,4,1,5,7,6,4,1,2,2,3,9,2,1,2,2,3}
</code>
which is shown below,

If I calculate the standard deviation and mean for this distribution I get 3.1 and 2.4 respectively, but since the distribution isn't a normal distribution this doesn't really give me the confidence interval. How would I go about calculating the confidence interval properly?
Comment: Another option is to use bootstrapping to get a confidence interval, then you don't have to make assumptions or rely on Normal theory.
Comment: http://ms.mcmaster.ca/peter/s743/poissonalpha.html
Comment: @whuber: Thanks! I am now able to reproduce the tables presented in the link. I think the link answers the question, "For a given bin in the histogram, between which interval do we expect the number of counts to lie 95% of the time?" Is there a similar way to answer the question, "In which interval will 95% of the measurements lie?" (i.e. not for a specific bin)
Comment: (1) Are you sure you mean *confidence interval*, rather than say a tolerance interval or a prediction interval, or some other interval? (2) If the answer to (1) is 'yes, definitely a confidence interval', then a confidence interval for what, precisely?
Comment: @whuber: Thanks for the link. Can you clarify what is meant by the function qchisq? I tried to find the equivalent function is Mathematica and the closest I found was ChiSquareDistribution, but that doesn't seem to be it.
Comment: @Glen_b: I'd like to make a statement such as "the mean is $3.1 ^{+\sigma_+}_{-\sigma_-}$" (or equivalently that I am 68% sure the mean is between $3.1-\sigma_-$ and $3.1+\sigma_+$), and I want to calculate $\sigma_+$ and $\sigma_-$. I think that at least in the physics world this is called a confidence interval.
Comment: The *Mathematica* equivalent of `R`'s [`qchisq(p,df)`](http://www.inside-r.org/r-doc/stats/pchisq) is [`InverseCDF[ChiSquareDistribution[df], p]`](http://reference.wolfram.com/language/ref/ChiSquareDistribution.html).  (Click on either expression to link to its help page on the Web.)
Answer: This answer is based on the clarification offered in comments:

I'd like to make a statement such as ... "I am 68% sure the mean is between $3.1−σ_−$ and $3.1+σ_+$", and I want to calculate $σ_+$ and $σ_−$.
I think that at least in the physics world this is called a confidence interval.

Let's take it as given that in response to my question "confidence interval for what?" you responded that want a confidence interval for the mean (and as made clear, not some other interval for values from the distribution).
There's one issue to clear up first - "I am 68% sure the mean is between" isn't really the usual interpretation placed on a confidence interval. Rather, it's that if you repeated the procedure that generated the interval many times, 68% of such intervals would contain the parameter.
Now to address the confidence interval for the mean.
I agree with your calculation of mean and sd of the data:
<code>> x=c(1,2,3,5,1,2,2,3,7,2,3,4,1,5,7,6,4,1,2,2,3,9,2,1,2,2,3)
> mean(x);sd(x)
[1] 3.148148
[1] 2.106833
</code>
However, the mean doesn't have the same sd as the population the data was drawn from.
The standard error of the mean is $\sigma/\sqrt{n}$. We could estimate that from the sample sd (though if the data were truly Poisson, this isn't the most efficient method):
<code>> sd(x)/sqrt(length(x))
[1] 0.4054603
</code>
If we assumed that the sample mean was approximately normally distributed (but did not take advantage of the possible Poisson assumption for the original data), and assumed that $\sigma=s$ (invoking Slutsky, in effect) then an approximate 68% interval for the mean would be $3.15\pm 0.41$.
However, the sample isn't really large enough for Slutsky. A better interval would take account of the uncertainty in $\hat \sigma$, which is to say, a 68% t$_{26}$-interval for the mean would be
$3.15\pm 1.013843\times 0.41$
which is just a fraction wider.
Now, as for whether the sample size is large enough to apply the normal theory CI we just used, that depends on your criteria. Simulations at similar Poisson means (in particular, ones deliberately chosen to be somewhat smaller than the observed one) at this sample size suggest that using a t-interval will work quite well for similar Poisson rates and 27 observations or more.
If we take account of the fact that the data are (supposedly) Poisson, we can get a more efficient estimate of the standard deviation and an interval for $\mu$, but if there's any risk the Poisson assumption could be wrong - a chance of overdispersion caused by some homogeneity of Poisson parameters, say - then the t-interval would probably be better.
Nevertheless, we should consider that specific question - "how to get a confidence interval for the population mean of Poisson variables" -- but this more specific question has already been answered here on CV - for example, see the fine answers here.
Comment: (1) Corresponding quantiles of the observed data *DOES NOT* give a 68% interval for the mean! This was the whole point of the question I first asked in comments - don't calculate an interval for the wrong thing! (2) that's nonparametric; why specify anything about the distribution? (3) You mean 16th and 84th, in any case.
Comment: Thank you very much for response. It really helps. As a follow-up question, if I wanted to know the answer to the question "In what interval will 68% of measurements lie?", what would I need to do? I assume if it were normally distributed then we just use sd(x), but what can we do if its a highly skewed distribution?
Comment: Never mind, I just realized for this I can just take the 14th and 86th percentile. Thanks again!
Comment: Yes, sorry I misspoke. I agree that it doesn't give the 68% interval for the mean. As well as the confidence interval of the mean I also wanted something to describe the distribution because in my particular case its interesting how spread the datapoints are (not just how precise we know the mean). It doesn't necessarily need to be formal so I figured I'd use the 16th and 84th percentile points. Do you know of a better solution?
Comment: It depends on what aspect of the distribution you want to describe. There's nothing wrong with quantiles if they capture what you're interested in. A better solution would require more information about what you need to be able to say.
Answer: It was mentioned in the comments on the original post but here it is more explicitly.  The bootstrap is simple to use and has a ton of nice asymptotic theory behind it, e.g. Shao and Tu, 1995.  Here's some R code which does what I think you want:
<code>the_data = c(1,2,3,5,1,2,2,3,7,2,3,4,1,5,7,6,4,1,2,2,3,9,2,1,2,2,3)
n_resamples = 1000
n_data = length(the_data)
bootstrap_mean = NULL
for(ii in 1:n_resamples){
  bootstrap_sample = the_data[sample(1:n_data, size = n_data, replace = T)]
  bootstrap_mean = c(bootstrap_mean, mean(bootstrap_sample))
}
plot(density(bootstrap_mean), main = "")
## 68% bootstrap confidence interval
lower_bound = 0.16
upper_bound = 0.84
quantile(bootstrap_mean, probs = c(lower_bound, upper_bound))
</code>
For one run I get:
<code>     16%      84%
2.740741 3.592593
</code>
Comment: Thanks, I will definitely apply this technique, as my real data is highly skewed. I'm unfamiliar with R code so I'll likely implement it into Mathematica. Can you explain what the "for(ii in 1:n_resamples){
  bootstrap_sample = the_data[sample(1:n_data, size = n_data, replace = T)]
  bootstrap_mean = c(bootstrap_mean, mean(bootstrap_sample))
}" segment is doing? The rest I think is self explanatory.
Comment: The "grand idea" behind the bootstrap is that you create a bunch (at least 1000) pseudo-datasets, calculate your statistic of interest, and utilize that statistic's empirical distribution over all the pseudo-datasets.  The pseudo-datasets are, importantly, drawn from the real dataset **with replacement** and so that the pseudo-dataset has the **same number** of observations as the real dataset.
Answer: For skewed distributions the confidence interval is tricky. One way to proceed is by having equal quantiles from tails. So, for instance, if you wish to have 95% confidence interval, you'd get 2.5% and 97.5% quantiles.
Your comment about $\pm\sigma$ being 68% CI in physics is only true when you assume normal-ish distribution. The Poisson distribution is quite not like normal, it's asymmetric(skewed) and has a lower bound as you noted. If you really want 68%, then get 15% and 84% quantiles.


Problema ao incluir chave estrangeira no Mysql

Question: Alguém pode me ajudar? ao tentar adicionar a foreign key, é apresentada a seguinte mensagem: "Error code 1215. Cannot add foreign key constraint".
<code>/*create table Processo*/

create table Processo(
Numero_processo varchar(15) not null,
Acao varchar(40),
Vara varchar(30),
Servidor_expeditor varchar(30),
Matricula varchar(10) unique,
Email varchar(40),
Primary Key (Numero_processo)
);

/*criando tabela Cartorio*/

create table Cartorio(

Matricula varchar(10) not null unique,
Servidor_solicitante varchar(30) not null,
Secao varchar(30),
Telefone varchar(15),
Email varchar(40),
Ocupacao varchar(30),
Fk_Numero_processo int,
primary key (Matricula),
foreign key (Fk_Numero_processo) references processo (Numero_processo)
);
</code>
Comment: Numero_processo x numero_processo , em Banco de Dadis faça tudo em "uppercase".
Comment: Mesmo assim continua dando erro...
Comment: Edita a pergunta com o novo script.
Answer: A chave primaria e a estrangeira têm de ser do mesmo tipo (na tabela "processo" está varchar e a estrangeira como int). Alias, recomendo que faça o contrário: o campo chave primaria de cartorio deve ser chave estrangeira da tabela processo.
Se for usar "Matricula" tire o "unique".
Outra obs: sua tabela processo não tem autor, réu, tipo de ação, nome dos advogados... até + e boa sorte. :-)
Answer: No seu caso, a tabela Processo tem a coluna Numero_processo como chave primária, mas na tabela Cartorio, a chave estrangeira Fk_Numero_processo é do tipo int, enquanto Numero_processo na tabela Processo é do tipo varchar(15).
Além disso, a declaração de Fk_Numero_processo precisa estar no mesmo tipo de dados da coluna referenciada. Portanto, você deve corrigir a tabela Cartorio para resolver esses problemas. Aqui está a versão corrigida, considerando que o Numero_processo será um campo que poderá armazenar caracteres alterei para ficar como varchar em ambas tabelas. Caso seja somente numeros deve ser alterado para int.
<code>/*create table Processo*/

create table Processo(
    Numero_processo varchar(15) not null,
    Acao varchar(40),
    Vara varchar(30),
    Servidor_expeditor varchar(30),
    Matricula varchar(10) unique,
    Email varchar(40),
    Primary Key (Numero_processo)
);

/*criando tabela Cartorio*/

create table Cartorio(
    Matricula varchar(10) not null unique,
    Servidor_solicitante varchar(30) not null,
    Secao varchar(30),
    Telefone varchar(15),
    Email varchar(40),
    Ocupacao varchar(30),
    Fk_Numero_processo varchar(15), -- Alterado para varchar(15)
    primary key (Matricula),
    foreign key (Fk_Numero_processo) references Processo (Numero_processo)
);
</code>


Tastypie detail_uri_name with forignkey's attribute

Question: I have a extended the User Django class with my own user class in django:
class MyUser(models.Model):
     user = models.OneToOneField(User, on_delete=models.CASCADE, primary_key=True, related_name='my_user')
     theme = models.IntegerField(default=0)
     tags = TaggableManager()

And a corresponding tastypie resource, that I want to use detail_uri_name with the username from the Django user class. Although, I do not know how.
class MyUserResource(ModelResource):
    class Meta:
        queryset = User.objects.all()
        allowed_methods = ['get']
        detail_uri_name = 'user_username'# ????

Error I have is: 'MyUser' object has no attribute 'user__username'
How do I access the username from MyUser as an attribute?
The line is:
 detail_uri_name = 'user_username'# ????

From the shell I can do MyUser.objects.all()[0].user.username to get the username of the associated django class.
Answer: You cannot use other class attribute as a detail_uri_name.
But, in theory:
class MyUserResource(ModelResource):
    class Meta:
        queryset = User.objects.all().select_related('user')
        allowed_methods = ['get']
        detail_uri_name = 'user__username'

    def get_bundle_detail_data(self, bundle):
        attrs = self._meta.detail_uri_name.split('__')
        return getattr(getattr(bundle.obj, attrs[0]), attrs[1])

Comment: This seems like a great idea, and I read the docs and it should work. But when I overload `get_bundle_detail_data` it does not actually get called. I think I am doing something wrong somewhere else, but if you have any insight I would appreciate it.


Find the value of: $\frac{\sqrt{45} + \sqrt{18}} {\sqrt{7+2\sqrt{10}}}$

Question:

Find the value of:$$\frac{\sqrt{45} + \sqrt{18}} {\sqrt{7+2\sqrt{10}}}$$

I need help solving this question. Every reply is appreciated.
Thanks!
Answer: We have:

$$\frac{\sqrt{45} + \sqrt{18}} {\sqrt{7+2\sqrt{10}}}$$

Then we will square the fraction:
$$ = \sqrt{\frac{(\sqrt{45}+\sqrt{18})^2} {(\sqrt{7+2\sqrt{10}})^2}}$$

Finish the squares:
$$=\sqrt{\frac{45+18\sqrt{10} + 18} {7+2\sqrt{10}}}$$

Simplify:
$$=\sqrt{\frac{63+18\sqrt{10}} {7+2\sqrt{10}}}$$

Factor:
$$\sqrt{\frac{9(7+2\sqrt{10})} {7+2\sqrt{10}}}$$

Crossing Out:
$$\sqrt{9}$$

Which is equal to:

$$\sqrt{9} = 3$$
Answer: Hint:
$$\frac{\sqrt{45} + \sqrt{18}} {\sqrt{7+2\sqrt{10}}} = \frac{3 \sqrt{5}+3\sqrt{2}}{\sqrt{\left(\sqrt{2}\right)^2+\left(\sqrt{5}\right)^2+2\,\sqrt{2}\sqrt{5}}}$$
Comment: Wish the downvoter had left a comment why.
Comment: Nice Hint.  (+1).  And who are these nefarious down voters?
Comment: @dxiv \ Don't worry about it, man. There are people in this world who pride themselves on being walking, talking toothaches.


Initialization in the tableaux method for first order logic

Question: This I believe is a question that can be answered in a one-line comment, but finals are just around the corner and I want to be sure about everything.
So suppose I want to check the validity of
$$\exists x(P(x)\wedge Q(x))\models \exists xP(x)\wedge \exists xQ(x)$$
What I've been doing is to negate the implication of the two, that is
$$\neg \left( \exists x(P(x)\wedge Q(x)) \rightarrow \exists xP(x)\wedge \exists xQ(x)\right)$$
and then try to arrive to a contradiction using the rules of the tableaux method. But I've questioned myself whether this is true, that is

$\psi \models \varphi \Leftrightarrow \psi \rightarrow \varphi$?

Thank you in advance.
Comment: @JeanMarie So sorry! I've edited the post accordingly. Thanks!
Comment: Related post: [truth-trees-in-predicate-logic](https://math.stackexchange.com/questions/2271103/truth-trees-in-predicate-logic): start  the tableau with the premise and the negation of the conclusion.
Answer: Technically, it's:

$\psi \models \varphi \Leftrightarrow \vDash \psi \rightarrow \varphi$

But given your proof by Contradiction method, what you really use is:

$\psi \models \varphi \Leftrightarrow \neg (\psi \rightarrow \varphi) \vDash \bot$

But yes, the method you use works just fine!


Problem with integrating sine function

Question: I have a problem solving an integration. This is my approach:
\begin{align*}
u &= \frac{1}{T}\cdot\intop_{0}^{T}(U_{0}+\hat{u}\cdot \sin(\omega t))dt\qquad\text{with }\omega=\frac{2\pi}{T}\\
u &= \frac{1}{T}\cdot\intop_{0}^{T}(U_{0}+\hat{u}\cdot \sin(\frac{2\pi}{T}\cdot t))dt\\
u &= \frac{1}{T}\cdot\left[U_{0}t-\hat{u}\cdot\frac{T}{2\pi}\cdot\cos\frac{2\pi}{T}\cdot t\right]_{0}^{T}\\
u &= \frac{U_{0}T}{T}-\hat{u}\cdot\frac{T}{2\pi\cdot T}\cdot\cos\frac{2\pi\cdot T}{T}\\
u &= U_{0}-\hat{u}\cdot\frac{1}{2\pi}\cdot\cos2\pi\\
u &= U_{0}-\frac{\hat{u}}{2\pi}.
\end{align*}
But the solution should be ${u = U_{0}}$. Where is my mistake?
Answer: It's $\cos 0=1$, not $\cos 0=0$.
Comment: You are missing the term that comes from inserting the lower limit $t=0$ into the antiderivative. That term will cancel the one coming from $t=T$, since $\cos 2\pi$ and $\cos 0$ are both 1.
Comment: Oh dear! I didn't see that. Thanks!
Comment: What do you mean? I only have $\cos(2\pi)=1$

Problem with integrating sine function


preserve format of node body

Question: for output display I used http://foundation.zurb.com/docs/components/accordion.html
I have the following code:

About Us

<?php foreach ($view->result as $key => $value) {?>
<img class="media-object" src="<?php print $GLOBALS['base_path']."sites/default/files/about/thumbnails/".$value->_field_data['nid']['entity']->field_about_us_image['und'][0]['filename'];?>" alt="">

<?php echo $value->node_title;?>

    <a href="#pane<?php print $key;?>">
    <?php print $value->_field_data['nid']['entity']->body['und'][0]['summary'];?>...
    <div id="pane<?php print $key;?>" class="content">
     <?php print $value->_field_data['nid']['entity']->body['und'][0]['value'];?>

  <?php } ?>

I want to preserve the format HTML  of
<?php print $value->_field_data['nid']['entity']->body['und'][0]['summary'];?>

and
<?php print $value->_field_data['nid']['entity']->body['und'][0]['value'];?>

How Can I achieve this with Drupal 7?
Comment: Could you explain better? These variables should have the entire value of the field, or not?
Comment: It is best if you give as much information as possible. For example, where is this code of yours? Is it in some views template? If so, which template?
Answer: Are you talking about the spacing in the html itself? So the code looks visibly nested? Such as:

         something

If that's the case, you might want to try a WYSIWYG and add WYSIWYG CodeMagic

preserve format of node body


apache2 and VirtualHost with foreign characters and more than one Alias

Question: I have a simple apache2 setup with two VirtualHosts. One is a domain name with "normal" characters, and one has foreign characters (with punycode).
The problem is that I don't know how to add "www" as an alias with the punycode domain. With the "normal" one it works fine.
Here's the "normal" host:
ServerName aev25.no
ServerAlias www.aev25.no

I can access the correct site with both the name and the alias.
Here's the "foreign"/punycode host:
ServerName xn--hvin-gra.no
ServerAlias høvin.no

Now, since I haven't got an Alias with "www", it naturally won't work like that. But if I try adding another alias, like this:
ServerName xn--hvin-gra.no
ServerAlias høvin.no
ServerAlias www.høvin.no

or this:
ServerName xn--hvin-gra.no
ServerAlias høvin.no www.høvin.no

it doesn't work. It just opens the default apache2 site. So it seems that this is not the correct way to do it, even though apache2 doesn't complain.
I also have a setup for the domain aev25.no, where the ServerAlias is www.aev25.no. That works fine, which should show that the problem is not related to www.*.:
ServerName aev25.no
ServerAlias wwwaev25.no

(works fine)
So, how do I add another Alias to the domain? Or should I do this another way?
NB: The apache2 version is 2.2.22.
Comment: Have you got your dns configured for the alias? Does www.høvin.no resolve to the server ip address?
Comment: Yes, the dns is correct. I just edited that into the post, since I forgot to mention it. høvin.no opens the correct site, while www.høvin.no just opens the default apache site ("default" from sites-available/enabled).
Comment: Can you rename `default-site` file to something like `z-default`, to make sure it is processed at last, and see if it helps?
Comment: Not sure if I know what you mean. Do you mean rename /etc/apache2/sites-enabled/000-default? If so, I tried that during the troubleshooting, and it didn't have any effect on anything. The way this version of apache2 works - possibly all newer versions? - is that I try accessing a sub domain or alias for a site that doesn't exist, it defaults to the default page. What I know is that høvin.no works, as the correct site is opened. But any extra Aliases - www.høvin.no, test.høvin.no etc - just opens the default page.
Comment: I also forgot to mention: I have a similar setup for the domain aev25.no, where www.aev25.no is the ServerAlias. That works fine, so the problem is not related to www.* I'd say.
Answer: I can't test it but I have a vague recollection of reading somewhere that you need to use punycode in both the ServerName and ServerAlias directives. I would try
ServerAlias    www.xn--hvin-gra.no

I got the www.xn--hvin-gra.no by entering www.høvin.no into a random punycode converter.

I did a  simple google search and this is correct apache serveralias punycode
Comment: This is correct; the reason is that translation from UTF-8 to punycode happens in the client, not in apache. You would have to use the same conversion in e.g. nginx. See https://www.w3.org/International/articles/idn-and-iri/ for links to the relevant RFCs.
Comment: You're right. That was the only thing I didn't try yet because I read that you should NOT use punycode in the ServerAlias.