email sent from my server shows up as spam

Question: I know this question has been asked and answered a couple of times, but I can't find any reference to my particular problem. I've set up Postfix and Dovecot on my Debian 7 VPS and I can send mail, but it shows up in my inbox at gmail as spam. The domain I'm sending from runs a web app that uses an input "site contact" email address, and I've put one of my gmail email addresses there. Now, when I send to my other gmail account, it shows in spam, and the reasoning google gives is because while the from field shows the "originating" email address as mine, it knows that it didn't in fact come from gmail and thinks that there is spoofing going on.
From what I've read, I need to set up a specific SPF record in my DNS to validate my server to not show up as a spam sender. I've added a TXT record for @ with v=spf1 a:fluidvector.com include:_spf.google.com ~all, but my messages still go directly into the spam folder. Also, I've read that the Return-Path could be an issue. In my case, the Return-Path shows as the user account that Apache2 runs under. Do I need to change this to reflect my gmail sender account?
I've run some SPF testing and the output claims the SPF is good:
SPF record lookup and validation for: fluidvector.com
SPF records are published in DNS as TXT records.

The TXT records found for your domain are:
v=spf1 a:fluidvector.com ~all

Checking to see if there is a valid SPF record.

Found v=spf1 record for fluidvector.com:
v=spf1 a:fluidvector.com include:_spf.google.com ~all

evaluating...
SPF record passed validation test with pySPF (Python SPF library)!

(the above result was given right after I added the include:_spf.google.com portion to my SPF record)
How can I keep messages sent from my server using a gmail account from showing up as spam in another gmail account? Is there possibly something wrong with my Postfix configuration?
Here is the raw email that gets put into the spam folder:
Delivered-To: firstname.lastname@example.com
Received: by 10.70.48.208 with SMTP id o16csp66094pdn;
        Thu, 10 Jul 2014 13:25:21 -0700 (PDT)
X-Received: by 10.224.135.2 with SMTP id l2mr86728784qat.37.1405023921164;
        Thu, 10 Jul 2014 13:25:21 -0700 (PDT)
Return-Path: <firstname.lastname@example.com>
Received: from fluidvector.com ([126.96.36.199])
        by mx.google.com with ESMTPS id h39si273921qgd.93.2014.07.10.13.25.20
        for <firstname.lastname@example.com>
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 10 Jul 2014 13:25:21 -0700 (PDT)
Received-SPF: pass (google.com: domain of email@example.com designates 188.8.131.52 as permitted sender) client-ip=126.96.36.199;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of email@example.com designates 188.8.131.52 as permitted sender) email@example.com;
       dmarc=fail (p=NONE dis=NONE) header.from=gmail.com
Received: from localhost (localhost.localdomain [127.0.0.1])
    by fluidvector.com (Postfix) with ESMTP id F0AF8914009F
    for <firstname.lastname@example.com>; Thu, 10 Jul 2014 20:25:18 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at
Received: from fluidvector.com ([127.0.0.1])
    by localhost (fluidvector.com [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id VUT1YtvYAYKz for <email@example.com>;
    Thu, 10 Jul 2014 20:25:14 +0000 (UTC)
Received: by fluidvector.com (Postfix, from userid 33)
    id EA69B91400A2; Thu, 10 Jul 2014 20:25:13 +0000 (UTC)
To: =?UTF-8?B?dGVzdA==?= <email@example.com>
Subject: =?UTF-8?B?V2VsY29tZSB0byAiQmVhbnNXb3J0aHki?=
X-PHP-Originating-Script: 1000:messenger.php
From: <email@example.com>
Reply-To: <email@example.com>
Sender: <firstname.lastname@example.com>
MIME-Version: 1.0
Message-ID: <email@example.com>
Date: Thu, 10 Jul 2014 16:25:13 -0400
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: phPULP
X-MimeOLE: phPULP
X-phPULP-Origin: phpulp://beansworthy.fluidvector.com
X-AntiAbuse: Site servername - =?UTF-8?B?YmVhbnN3b3J0aHkuZmx1aWR2ZWN0b3IuY29t?=
X-AntiAbuse: User_id - 1
X-AntiAbuse: Username - =?UTF-8?B?QW5vbnltb3Vz?=
X-AntiAbuse: User IP - 18.104.22.168

I should also add that ultimately I'm going to try and set up Postfix to use all 5 of the domains I host to send mail. Does this mean that I need to set individual "sender" addresses for each domain to avoid being marked as a spammer?
Any light that anyone could shed on this would be greatly appreciated.
-EDIT-:
I also notice that the mail seems to be bounced around on my server a couple of times before it's delivered. Is this normal?
Comment: I see 'dmarc=fail` so setting up your DMARC records should help. It will also let you get summary reports from the mailbox provider on spam counts etc if you should want them. Basically DMARC is a combination of SPF and DKIM (domain keys identified mail).
Answer: You cannot authorize some domain to send mail from gmail addresses, only Google can do that. If everyone could authorize their own server to send some other domain's email, then SPF would serve no purpose. Spammers would just authorize themselves. You need to use a source email address whose credentials you control.
Comment: So I should use a sending address such as `firstname.lastname@example.com`?
Comment: @chaoskreator Yes. Since you control fluidvector.com's SPF records.
Comment: Google does authorize all addresses with their `~all` SPF policy.  However, there are freemailer conditions in some spam checks which penalize failure to use Googles servers.
Answer: I see several issues:

PTR record returns shoplucro2.smtpilimitado.info which does not lookup to 184.108.40.206.  It should likely return mail.fluidvector.com.  This is a strong indication of Spam.
Envelope address and From: address don't match.  If you aren't resending email, they should match.  If you don't want auto-replies (as I would expect for a website) consider using an address like "email@example.com".  You can use a Reply-To: header if you want people to be able to reply.
From the received headers, it looks like you are virus filtering the outgoing email.  While this is a good idea, you are re-injecting the filtered mail, rather than bouncing it before accepting it.  This can result in back-scatter Spam.
Reply-To: and Sender: headers both have the same data as the From: address.  While I haven't verified these indicate Spam, they are intended to be used when the From: address does not specify the sender or reply-to address.  I would expect your use to indicate Spam.

Your SPF policy is extremely open, so anyone is allowed to send for your domain.  (This will happen anyway because you include the Google policy.)
Other than the above issues, I would not expect your use of a Gmail address in the From: header to cause your email to be classified as Spam.  The published Gmail SPF policy would allow this use.
It is likely you will need to get your IP address provider to update your PTR record.
EDIT: The Sender: field is intended for cases where someone (an assistant) is sending email on behalf of someone else (the From: address).  In this case the Sender: and envelope addresses should be the same and the From: address different.
The Reply-To: field is indented for cases where replies should go to a different address than the From: address sending the email.  There may be rare cases where it is appropriate to all three fields with different values for each.
Either the Sender: and Reply-To: semantics apply for many web-sites, but in most case the code is the message author, so it should be the From: address.
Sites that allow users to send other links or other information would then put the originating user's address in the Reply-To: address.  Responses to such email are likely not of interest to the site, so use a noreply address which is sent to the bit bucket.  You may want to track noreply volumes and reasons.  You should have active abuse and postmaster addresses so you can be notified of abuse and configuration issues.
Comment: I did notice that reverse DNS lookups on my IP were returning `smtpilimatado.info` and have opened a ticket with my VPS provider. Can you explain the "envelope address"? I thought that the `From:` and the `Sender:` fields were supposedly the envelope and from fields. Can you also explain how to avoid re-injecting the scanned email? Also, if I remove the `Sender:` field and rely on something like `email@example.com` for the `Reply-To:` wouldn't that send up a spam flag?
Comment: @chaoskreator I've updated with notes on the header fields.  You may want to review the RFCs particularly http://tools.ietf.org/html/rfc5322#section-3.6.2


How is this equation about expected value true?

Question: $X$ is a stochastic variable. We have $E(X) = \mu$ and $Var(X) = \sigma^2$
Then $E(X^2) = \mu^2 + \sigma^2$
We have data $X_1, X_2, X_3 , \ldots,  X_n$ from that distribution.
Then we have: $$\sum_{i=1}^n (X_i - \bar{X})^2 = \sum_{i=1}^n X_i^2-n\left( \bar{X}\right)^2$$
And we have: $$E[\sum_{i=1}^n X_i^2]= \sum_{i=1}^n E[X_i^2]$$
But why is this equation below true?
$$\sum_{i=1}^n E(X_i^2)=\sum_{i=1}^n E(X^2)$$
What is the proof of that?
Comment: @KitterCatter No, it's correct. Here, $\bar{X}$ is the *sample* mean $\bar{X} = \sum_i X_i / n$.
Answer: This is due to the line,

We have data $X_1, X_2, \ldots, X_n$ from that distribution.

Hence, $X_1^2, \ldots, X_n^2$ must share the same distribution and hence the same expectation too.
An alternative view can also be $X_1, \ldots , X_n$ has the same variance and expectation. Since $E[X^2]=Var(X)+(E(X))^2$, they must share the same expectation when they are squared too.


Pullback of Poincaré dual

Question: I'm currently taking a course on characteristic classes. In proving the uniqueness of the Chern classes we used the following.
Let $h_n \in H^2_{dR}(\mathbb{C}P^n)$ be the Poincaré dual of $\mathbb{C}P^{n-1}$ as a submanifold of $\mathbb{C}P^n$. Let $i$ denote the standard injection of $\mathbb{C}P^1$ into $\mathbb{C}P^n$. Then $\int_{\mathbb{C}P^1}i^*h_n=1$.
In this case I'm not quite sure how I'd go about relating the Poincare dual to it's integral under the pullback. I know that this might follow from the axioms of the Chern classes but I was wondering if there is a more direct argument that I'm missing.
Answer: This is a direct computation. The Kähler form for $\Bbb CP^n$ (which generates the integral cohomology) can be easily integrated over a linear $\Bbb CP^1$ to get $1$. On the other hand, Poincaré duality tells you that the Poincaré dual of a hyperplane (i.e., $\Bbb CP^{n-1}$) must integrate over a $\Bbb CP^1$ to give the intersection number of that $\Bbb CP^1$ with the hyperplane; this intersection number is patently $1$. Thus, the Poincaré dual is in fact the Kähler form.
Comment: Do we know these two forms are the same as one of them generates the integral cohomology and they both integrate to be 1?
Comment: Same cohomology class, yes.

Pullback of Poincaré dual


How to redirect to top level domain and not relative value

Question: Under admin/config/system/site-information, the startpage is configured as /home. So when you go to www.domain.com, the content of /home is displayed. However, when a form is submitted, it automatically redirects to www.domain.com/home. Is there any way to prevent this?
Comment: Any form using ***_form1_submit($form, $form_state) { .....drupal_set_message(...) on the startpage itself

So, in the content of the node /home, there is a form, when submitting that form, it does not redirect to domain.com, but redirects to domain.com/home
Answer: I resolved it as follows:
in the form submit handler, I simply added:
<code>$form_state['redirect'] = '<front>';
</code>
Hope this helps someone...
Comment: I love finding solutions to problems I didn't know I had. Nice one :)
Comment: Note [form_state redirect does not work](https://www.drupal.org/node/1074616) on custom forms in D7. You have to use a submit handler to accomplish the redirect.


Proofs: Induction on Handshakes

Question: Here is the problem:
Suppose $n$ people are at a party, and some number of them shake hands. At the end of the party, each guest $G_i$, $1 \leq i \leq n$ shares that they shook hands $x_i$ times. Assume there were a total of $h \geq 0$ handshakes at the party. Use induction on $h$ to prove that:
$x_i + \cdots + x_n = 2h$
I am a little confused about the best way to solve this problem. Performing induction on $h$, means that I will assign my base case for $h=1$. The base case then suggests:
$x_i + \cdots + x_n = 2h$
$x_i + \cdots + x_n = 2$
I am I correct to assume that this means that for my base case for $1$ handshake, that there are 2 ($x_1$ + $x_2$) people who shook hands?
Based on this assumption, my next step would be to assume $h=k, \ \forall \ k  \in \mathbb{Z} $. Performing induction on $h$, I would then let $h=(k+1)$ and I want to show that the left side of the equation is equivalent to $2(k+1)$:
$x_i + \cdots + x_n + 2 = 2(k+1)$
^This is the step I'm not really certain of. Since I know that when I add one additional hand shake, that means I am adding $2$ people to the situation... so, would adding $2$ to the left side be a fair way to show this induction? If I do this, then I easily come up with:
$2k + 2 = 2(k+1)$
$ 2k + 2 = 2k + 2$
Which is a true statement, so then I would have (hopefully...) proved this by induction.
What do you guys think? I have traditionally only performed induction on more straight forward math-y problems, so I'm not really sure if I am headed down the right rabbit hole on this one. I am currently in an Introduction to Proofs course at my University. This is a homework problem- I am obviously not looking for someone to do my homework for me (since that wouldn't bode well for my next exam...) but I just want to seek out some advice for my approach to this problem.
Thanks for looking!
Comment: Looks like some basic graph theoretic lemma. Don't really need induction to be proved btw.
Comment: I agree, but the assignment requires the use of induction. Prof earned her PhD in graph theory, so Yay...
Answer: This is the right idea. Your base case is correct, and your argument is the right one, modulo some technical points:

You don't want to assume $h=k$ $\forall k \in \mathbb{Z}$. That doesn't really make sense and isn't what you are really doing. You need to assume the claim holds for $1\leq h \leq k$, i.e. that $x_1 + \cdots + x_n = 2h$ when there are $h$ handshakes for all $1 \leq h \leq k$. The point of induction is to show that this holds for $h=k+1$, i.e.
$$x_1 + \cdots + x_n = 2(k+1)$$
when there are $k+1$ handshakes.
For clarity you might say, for the inductive step, to add a handshake, two people must shake hands with each other. Say person 1 and person 2 are this new handshake. Then we consider the sum
$$(x_1 + 1) + (x_2 + 1) + x_3 + \cdots + x_n.$$
This is obviously $2+x_1 + \cdots + x_n = 2 + 2k$ by the inductive hypothesis, and $2+2k = 2(k+1)$ as you have observed.
Comment: Great, thank you very much. Once I figured out the +2 thing it almost seemed too easy, so I thought I was doing it incorrectly or making an assumption that wasn't allowed. Appreciate the technical help, as well. Thank you!
Answer: Assume that the handshakes occur sequentially: At time $1$ there is a handshake, then there is a handshake at time $2$, and so on. Let us suppose that when $h=k$, then $x_1+x_2+\cdots +x_n=2k$. Now at time $k+1$, a new handshake takes place, say between person $i$ and person $j$. Then $x_i$ is incremented by $1$, and so is $x_j$, and therefore $x_1+x_2+\cdots+x_n$ is incremented by $2$.

Proofs: Induction on Handshakes


Functional optimization: maximize a double integral where the functional appears twice

Question: Please help me solve the following optimization problem. Suppose that you have to choose a function $U: [0,1]\mapsto [0,1],$ which must be nondecreasing ($U'\geq 0$) to maximize the following integral:
\begin{align}
L[U] & =\int_0^1\int_y^1  (x-y) f(x)f(y)(1-G[U(y)-U(x)])  dx dy\nonumber \\
  & \quad + \int_0^1\int_0^y  (y-x) f(x)f(y)G[U(y)-U(x)] dx dy,
\end{align}
where the $f$ is a probability density function and $G$ is a cumulative density function. Both are known, generic, with no particular assumptions. Just assume  $[0,1]$ is in their support.
The problem is that $U$ appears twice inside the integral, so I can't (or I don't know how to) solve this with calculus of variations or optimal control.
Does anyone know how to solve this? Or if this can be solved at all (under what assumptions)? Or maybe a reference? Please point me in the right direction.
Comment: You might have a look on: [Functional Derivative](http://math.stackexchange.com/a/1356581/79762)
Answer: You need to differentiate with respect to each occurrence of $U$ and add up all the contributions. For the first term, differentiating the inner integral with respect to $U(y)$ yields
$$
\int_y^1(y-x)f(x)f(y)g[U(y)-U(x)]\,\mathrm dx\;.
$$
To differentiate with respect to the other occurrence, first swap the order of integration and interchange $x$ and $y$,
\begin{align}
&\int_0^1\int_y^1(x-y)f(x)f(y)(1-G[U(y)-U(x)])\,\mathrm dx\,\mathrm dy\\
=&\int_0^1\int_0^x(x-y)f(x)f(y)(1-G[U(y)-U(x)])\,\mathrm dy\,\mathrm dx\\
=&\int_0^1\int_0^y(y-x)f(y)f(x)(1-G[U(x)-U(y)])\,\mathrm dx\,\mathrm dy\;,
\end{align}
and then differentiate the inner integral with respect to $U(y)$:
$$
\int_0^y(y-x)f(x)f(y)g[U(x)-U(y)]\,\mathrm dx\;.
$$
Since $U$ is nondecreasing, the two contributions can be combined into
$$
\int_0^1(y-x)f(x)f(y)g\left[-\left|U(y)-U(x)\right|\right]\,\mathrm dx\;.
$$
Proceeding the same way for the second terms yields a contribution
$$
\int_0^1(y-x)f(x)f(y)g\left[\left|U(y)-U(x)\right|\right]\,\mathrm dx\;.
$$
Thus the condition for stationarity of $L[U]$ is
$$
\int_0^1(y-x)f(x)f(y)\left(g\left[\left|U(y)-U(x)\right|\right]+g\left[-\left|U(y)-U(x)\right|\right]\right)\,\mathrm dx=0
$$
for all $y\in[0,1]$.


So I'm moving from shared hosting to a VPS. What do I need?

Question: I have apache2, mysql, and vsftpd running. Are there any other tools I can use to make the transition easier? Any free control panels similar to cpanel?
Comment: How many sites do you host? What platform are you using?
Comment: I host 2 low traffic sites. I'm using the latest version of Ubuntu Server.
Answer: I loathe using a control panel, because while they make the simple stuff easy, they make the hard stuff damn well near impossible, which severely hampers your ability to do things down the road -- and you never know when you're going to hit a massive control panel-inspired road block (it's not as though the authors are going to say "here's where we suck"), and moving away from a control panel later ends up being a lot more costly and troublesome than just avoiding them to begin with.
Learn some basic sysadmin skills and you'll be a lot better off in the long run.
Comment: OP and others might dismiss this comment but anyone who has had the misfortune of inheriting a server with a control panel config knows the truth of how difficult it can become to admin your server when you want to do something the panel doesn't consider "proper".
Comment: You've got a control panel that's quicker than "ssh server adduser fred"?  That's verging on the magical.
Comment: I work at a hosting company, managing customer boxes with control panels on them (whether it's one of the commercial ones, or the open source ones) is several times more difficult than managing customer boxes that have just been configured by hand.
Comment: The only thing I use a control panel for is quickly making users and groups and modifying some start/stop process at boot.
Answer: You'll probably want an MTA as well - I'd recommend Postfix.  You can also install Postfix Admin (http://postfixadmin.sourceforge.net/) if you'd like a web interface to it.
Make sure you've got a firewall running as well.  APF (http: //www.rfxn.com/projects/advanced-policy-firewall/) is a reasonably nice iptables configuration manager that's quite straight forward to configure.
As for making the migration process easier, just follow these simple steps.

Lower the TTL on your domain to 300 seconds
Create your Apache config for your VirtualHost
Create your database(s) and user(s)
Migrate your static content, databases and cron jobs (but leave them disabled for now)
Add appropriate entries to your hosts file so that you can test the site before going live
Check & fix all aspects of the site
Put up an "under maintenance" page on your old server
Take a final database dump on your old server and import it on your new server
Change your DNS records to point to your new IP and adjust TTL back
Enable your cron jobs.
Answer: Don't forget to install php (or the scripting language of your choice).
phpMyAdmin is useful for MySQL administration.
You may also want to install open-ssl for generating SSL certificates and/or certificate requests (assuming you want secure connections).
If you want to send mail from your scripts you'll need something like PostFix. If you want to receive and serve mail as well, you'll also need a pop3 and/or IMAP server (Cyrus, Dovecot, Courier IMAP).
You may also want to checkout SliceHost's Article Section. It's fairly comprehensive and aimed at new server admins.
I agree with the comments above- control panels are very limiting. In the long run, you'll be better off editing the config files by hand- it's usually pretty straight-forward.
Answer: If you are looking for control panels to manage your server then consider Webmin or ISPConfig both free and open source. Like others recommended it would not hurt to configure by hand, in the end it's your choice.
Answer: http://www.howtoforge.com/perfect-server-debian-squeeze-with-bind-and-dovecot-ispconfig-3
This will help you to setup an ISPconfig server.


Xlr output in guitar processor

Question: Could you point be out why XLR output is so important?
I'm thinking to buy bass processor Zoom B3 or Boss gt-1b and first one has this output onboard.
Thanks everyone for your help!
Answer: XLR output is useful when connecting directly to mixing console or audio interface with XLR cable (rather than connecting to a guitar amplifier with instrument cable).
XLR connection is differential. The main advantage is that the ground can be disconnected (e.g. using switch on ZOOM unit) which helps to fight ground loop noise. Also XLR cable is more resistant to interference and typically will introduce less noise.
Comment: "XLR connection is differential" – While this happens to be true for this particular device, there is no guarantee that this is always the case. XLR simply says something about the physical dimensions and layout of the pins … you can transport anything across those pins. Heck, I have devices that use XLR for the power adapter!
Comment: @JörgWMittag Good point. I was referring to this specific use only.
Answer: The physical format of the plug (XLR) is irrelevant. What is relevant is that the output is

balanced and
line-level

An XLR plug can be used for balanced or unbalanced signals and it can be line-level, instrument-level, or mic-level, or even high-voltage speaker-level. It doesn't even need to be analog audio at all, it could also be AES3 digital audio.
On the other hand, a balanced and line-level output can be TRS or XLR.
The physical format of the output doesn't tell you anything.
The output of the Zoom B3 is a line-level, balanced output. That is what is important, not the fact that it is physically an XLR output (although that is nice, too, since it is the de-facto standard for transporting analog audio signals in professional environments).
Balanced connections use differential signaling, where the original signal is transmitted over one wire, and the equal but opposite signal is transmitted over another wire. The two wires are kept in close proximity to each other (typically, they are twisted together into a twisted pair). Now, when external noise is introduced into the cable, the noise will be roughly the same on both wires.
At the receiving end, we receive on the two wires:

the positive signal +S and the noise N
the negative signal -S and the noise N

And we take the difference of the two wires:
  +S + N - (-S + N)
= +S + N + +S - N
= +S + +S + N - N
= 2S + N - N
= 2S

in other words: since we take the difference between the two wires but the noise introduced is roughly the same on each wire, the noise cancels out.
This allows transporting analog audio signals over relatively large distances with very little noise.
This differential signaling is used in many other applications, for example in Ethernet. In fact, because of the high demands of Gigabit Ethernet, Cat5e cables are typically much higher quality than standard audio cables, and because of the much bigger IT market compared to the pro audio market, they are also much cheaper (cents per meter for Cat5e compared to tens of cents per meter for microphone cables). And they contain 4 twisted pairs, so you can actually transport 4 audio signals on a single Cat5e cable for cheaper and with higher quality than you can through a microphone cable, and with a connector that has roughly the same footprint as a single XLR. (This is a nice trick to keep in mind when building small multicores.)
Comment: You would also want the cat5e to be shielded, right? Also, would network cable has the same mechanical durability as audio cables?
Comment: Yes, at least shielding for the whole cable, i.e. S/xTP, F/xTP, or SF/xTP. I don't think shielding for the pairs (x/FTP) is necessary. The durability, like with audio cables, depends. There's different kinds of cables for different kinds of applications. But even 50m ruggedized S/UTP Cat5e with "obviously Neutrik etherCON but we're not allowed to call 'em that because it's a Chinese clone" connectors is less than $2/m which is less than twice that of a comparable microphone cable of the same company and brand and about 2/3 of a 4-channel multicore cable *without connectors*.
Comment: Now the only mystery left is why it is significant that a guitar processor has  line-level, balanced output.
Comment: And btw, I'd claim that in practice the physical format of the plug is insignificant only if the users of the device carry a set of connectors and soldering tools and are willing to take the time to build adapters.

Xlr output in guitar processor


What is the loss function for C - Support Vector Classification?

Question: In article LIBSVM: A Library for Support Vector Machines it is written that C-SVC uses the loss function
$$ \frac{1}{2}w^Tw+C\sum\limits_{i=1}^l\xi_i$$
I know what is $w^Tw$.
But what is $\xi_i$?  I know that it is somehow connected with misclassifications, but how it is calculated exactly?
P.S. I don't use any non-linear kernels.
Answer: $\xi_i$ are the slack variables. They are typically nonzero when the 2-class data is non-separable. We are trying the minimize the slack as much as possible (by minimizing their sum, since they are non-negative) along with maximizing the margin ($w^Tw$) term.
Exact calculation: Well, if the convex program has been solved to optimality without any optimization error, then yes, they are calculated exactly.
Comment: Yes, it is the hinge loss. The hinge loss has been removed from the objective and made into a bunch of constraints (their number equalt to the number of examples, $l$ in your notation). In particular, $\max[0,1-y_iw^Tx_i]$ is the loss on example $i$.
Comment: I know, that they are typically nonzero when the 2-class data is non-separable.
I know, that we are trying to minimize their sum.
But I don't know, what is the loss function they are calculated with. Is it a step function or hinge loss or something else?
Comment: @Theja can you explain how wTw term is same as maximum margin?


How to specify the mandatory courses in this training managment web applicaiton?

Question: I am developing a training management web-based system using ASP.NET. This application will display the training record of each employee to the managment. In our company, we have three different types of courses and some courses of each type are mandatory coruses. My database design is as following:
       Employee Table: Username, Name, Job, DivisionID
       Division Table: DivisionID, DivisionName
         Course Table: CourseID, CourseName, GroupID
          Group Table: GroupID, GroupName.
Employee_Course Table: Username, CourseID

(The first attribute is the primary key in each table except for the last table which consists of two foreign keys)
I could be able to retrieve all the information and show them in a nice GridView. The problem now is how to determine the mandatory courses. I don't know what I should do. Should I change the above design of the database? Should I show these courses in different categories? Also, how will the admin determine whether this is a mandatory course or not?
BTY, later on I will need to come up with a query that shows how many employees finish the mandatory courses, so design should be applicable to this issue.
Comment: Are mandatory courses required for all employees, or do different jobs, divisions etc... have different required courses?
Comment: Are you actually asking how the admin will decide if a course is mandatory?
Comment: How to implement this in the system with the database.
Comment: Add another column in table "Course Table" name it as ISmandatory  and set it to 0 for Notmandatory  and 1 for mandatory
Answer: If there are courses which are mandatory and some which are not then apparently you need to store this classification in the database. If a mandatory course is always mandatory for every employee then you could add a IsMandatory column to the course table otherwise you might have to come up with some other table which stores the relationship of which course is mandatory for which employee (or group of employees).
Also if you have three types of courses then it would make sense to store the type for each course as well.


Expected value of number of heads in n flips

Question: We flip a coin n times and calculate the number of heads. for the obtained number of heads of the previous n tosses we will toss the coin again. what is the expected value of heads in this tossing process?
Comment: What have you tried? Where have you gotten stuck?
Comment: How we can calculate this expected value after tossing for the obtained number of heads of the previous n tosses?
Answer: Let $H_1$ be the number of head in the first $n$ tosses, and $H_2$ be the number of head in the second. We are looking to compute $E(T)$, where $T=H_1 + H_2$. Note that $E
(T)=E(H_1)+E(H_2)$. $E(H_1)$ is simply $n*0.5 = n/2$. To compute $H_2$, we will use the law of total expectation,
\begin{align}
E(H_2) = E(E(H_2|H_1)) &=\sum_{i=0}^n E(H_2|H_1=i)P(H_1=i) \\
&= \sum_{i=0}^n i/2 \binom{n}{i}(0.5)^n \\
&= (0.5)^{n+1}n2^{n-1}\\
&= n/4
\end{align}
Thus, $E(T)=n/2+n/4 = 3n/4$
Comment: +1 More concise: $\mathbb E[\mathbb E[H_2\mid H_1]]=\mathbb E\frac12H_1=\frac14n$


How difficult is this piece of music (electric guitar)?

Question: Listening to this piece of music (from 2:12), "Running from Evil" (by Bobby Prince), and being a total novice, I was wondering how difficult it is to play on the guitar.
Another version that seems also amazingly difficult.
How would you rate the level of difficulty, say from 1 (easy) to 5 (top level)?
Note: this is not an opinion-based question. I'm simply amazed at the pace of the music, observing the fingers moving at the speed of light, or almost... ; seen from my level (= zero) that sounds hyper-difficult, but I'd like to hear what the expert has to say...
Comment: Would editing question and asking to rate the piece against a grade make it non opinion based? @Dom
Comment: Hi e2-e4 and welcome to the site. Unfortunately, this is an opinionated question as there is no universal way to judge difficulty so any answer is going to be based on the user's perception of the piece making every answer equally valid.
Comment: The question would be better if you asked for a list of prerequisites and skills that are needed to play that example. It would be interesting to see how people divide areas to learn, for example right-hand vs left-hand, sight reading, playing to a click, picking, hand synchronization, hand independence etc. What kind of a difficulty hierarchy is there in each area, and are there dependencies between skills. Maybe that would still be too broad, but a skilled Stack Exchange expert might be able to construct a proper question. ;)
Answer: 185 BPM, alternate picking between 2 strings nonstop. So many notes.
I think it is 3. Feels like the beginning of Advanced level
Comment: Wow! What would you say is 5? Van Halen? Slash? Gilmour?...
Comment: @e2-e4 your example isn't particularly fast, and it seems to be a pre-written sequence, which can be learned through mechanical repetition. Things get more advanced when you make it up it on the spot as you're playing! Check out stuff like this https://www.youtube.com/watch?v=dloRBA3Z1W4&t=132s
Comment: "there are simply too many notes" :-)


Does it make sense to "cluster" when you use a regression discontinuity?

Question: One of the breakthroughs of econometrics over the past two decades has been to employ "clustering" to take into account the correlation of error terms across observations. For instance, if you're evaluating the effect of an educational intervention where you have data on individual students but you suspect that teachers implemented the intervention differently, it is common to analyze the data in a way that recognizes that there are common effects at the "class" level. A common correction is to use clustering.
When you run a regression discontinuity, do you similarly have to take into account that your observations may be clustered? If so, how is the estimator implemented differently?
Comment: Clustering first gained notoriety with the so-called "Moulton critique" discussed here: http://www.sciencedirect.com/science/article/pii/0304407686900217. A more recent example of the danger of not-clustering was presented  in: http://www.nber.org/papers/w8841. You can read about RD here: http://en.wikipedia.org/wiki/Regression_discontinuity_design
Comment: Could you give some references? It is a first time I hear about this breakthrough. Also what do you mean by regression discontinuity?
Comment: @lejohn, yes I think too, but this is exactly the term  OP used. If I understand correctly it is simply robust standard errors for panel data type models.
Comment: -1 This question does not provide enough information to allow for a constructive answer.
Answer: Whether you take into account clustering or not only affects the standard errors of your estimates. In a situation like yours, I would not focus too much on the standard errors. It is much more important that you can justify the use of the regression discontinuity framework, and you have to be able to show that it allows recovering the parameter(s) you are interested in. Once you are sure about this you can go a step further.
This next step would be the following. If you are concerned about these class effects, you should have a look at multilevel regression models. It should be possible to implement a regression discontinuity strategy within a multilevel model.
Comment: rescued from a deleted answer
Comment: Agreed; if you are worried about the efficiency of your coefficient estimates, using "clustered standard errors" might be fruitful.  But as mentioned, this is a separate (and minor!) issue compared with your regression discontinuity experimental design.
Answer: Is it a regular linear regression model or not?
In Quantitative finance we have the same problem. we use a GARCH model to estimate volatility, however volatility does not seem to be normally distributed and seems to cluster. for this we use a switching garch model. this "switching" model differentiates and switches between different clustering regions.
you could look into this switching it might be usefull...


the page 1 was request but the document has only 0 pages

Question: So I am trying to generate a report that may contain many pages depending upon the user input and data.
Similar error was summarized here but its not seen to be the actual solution of the problem.
My function:
public void generateReport()
{
  final String path = "/sdcard/testreport.pdf";
  try
  {
  Document document = new Document();
  writer = PdfWriter.getInstance(document, new FileOutputStream(FILE));
  document.open();
  // many lines code related to report generation
  writer.setOpenAction(action);
  status = GetReport(document);
  document.close(); //RunTimeException here.
  }
  catch (Exception e)
  {
    e.printStackTrace();
  }

But I am getting RunTimeException as-
Log
04-16 19:34:01.928: W/System.err(14248): java.lang.RuntimeException: The page 1 was requested but the document has only 0 pages.
04-16 19:34:01.929: W/System.err(14248):    at com.itextpdf.text.pdf.PdfWriter.close(PdfWriter.java:1208)
04-16 19:34:01.929: W/System.err(14248):    at com.itextpdf.text.pdf.PdfDocument.close(PdfDocument.java:776)
04-16 19:34:01.929: W/System.err(14248):    at com.itextpdf.text.Document.close(Document.java:420)
04-16 19:34:01.929: W/System.err(14248):    at com.wassap.main.ReportsActivity.generateReport(ReportsActivity.java:541)

I am not getting the clue as of how to close the document only if to know in advance if the page is there.
Comment: It isn't just to read but also to write. Also, the function contains many lines of code but I only mentioned the relevant lines that are causing the actual problem as mentioned in the question comment itself in the code.
Comment: No as for PdfReader(). You imagine as I am opening the report, writing into it and then closing the report. Then getting exception on close().
Comment: Why are you using a PDF writer to open a file that you apparently want to read?  Should you be using PdfReader() instead?
Answer: This question is old, but an answer may help others coming across it later.
I had this same error. In my case, the PDF I was creating only had 1 page, but the PdfAction I created was calling PdfAction.GotoLocalPage(2, pdfDest, writer). Since there was no page 2, the "page requested" error was thrown.
If your error is that there are 0 pages, I wonder if you are experiencing some error that is causing your PDF code to not run and therefore not create any pages in the generated PDF.
One other thought is that I don't see a document.NewPage() call in the sample code you included. Maybe there are multiple ways to create new pages, but that's the call I make to manually add new pages. If you don't have any of those calls, maybe that's why there are no pages in the PDF?


What's the maximum number of games at the playoff phase in NBA?

Question: I was checking out the playoff bracket the other day and I realised that a significant number of the games (in the first round) went up to the 7th game. Considering that there are 15 series in total x 7 matches the theoretical maximum is 105 matches. (In parallel with that, the theoretical minimum would be 15 x 4 = 60 matches).
Doing a quick calculation, I noticed that so far 72 games were played with 3 match-ups remaining.
So re-iterating my questions:

In what year were the most number of NBA playoff games played?
How many total games were played during that year's playoffs?

EDIT: Now that the 2014 playoffs are over, the total number of games has turned out to be 89 (if I am not mistaken).
Comment: I don't know -- but it's almost definitely since 2003. There have been 16 qualifying teams for the playoffs since 1984 but from '84 to '02 the first round was a best three-out-of-five series. Since 2003, the first round, like the second, third, and fourth, has been a best four-out-of-seven series. This change would create the possibility for up to an extra sixteen games.
Answer: 2014 and 2006 both had 89 games played in them, based on a cursory scrape of Wikipedia (which is much harder than it should be, btw; some significant editing is needed for consistency!).  Those are the two largest number of games played; however, pre 2003 of course the number of games in the first round was fewer, so it's not likely to surpass this.
If you weight for number of games in the playoffs, 1988 there were 80 games played out of a max of 89, for 89.9% of possible games played (2014 had 85.7% of possible games played, for reference).  1976 had 50/55 games played, for even higher 90.9%, and every single series went to either 6 or 7 games (plus the two 3 game series which went to 3 games in both).
1962 and 1963 look like they went to the highest overall, 29 of 31 games, for 93.5%.
I'll post a table of the whole thing, if I can find a better source to scrape than Wikipedia (which has a lot of minor formatting errors making it impossible to do efficiently).


Spring Security still returns HTTP 403 when HTTP request is created from remote host

Question: Situation
This is my Spring security config and it works when I create request from localhost and application runs at localhost too.
  @Override
  protected void configure(HttpSecurity http) throws Exception {

    http.sessionManagement()
        .sessionCreationPolicy(SessionCreationPolicy.STATELESS)
        .and()
        .authorizeRequests()
        .antMatchers("/client/**")
        .hasRole(HttpClientType.CLIENT.name())
        .antMatchers("/**")
        .hasRole(HttpClientType.USER.name())
        .anyRequest()
        .authenticated()
        .and()
        .exceptionHandling()
        .authenticationEntryPoint(restAuthenticationEntryPoint)
        .accessDeniedHandler(restAuthorizationEntryPoint)
        .and()
        .addFilterBefore(jwtTokenFilter, BasicAuthenticationFilter.class)
        .csrf().disable().cors().disable();

  }

Problem
When I deploy my application to production server, and create request from remote host, every time I get HTTP 403.
P.S. I thought the problem will be in CORS and CSRF but it doesn't work even I disable it.
Comment: Try making the request from Postman or cUrl. If it is CORS the Postman request should succeed.
Comment: I am using Postman and `Authentication` is successful.
Comment: Then I'm pretty sure it's CORS issue. Spring Security is pretty non-intuitive when using `.cors().disable()`. The `cors().disable()` does not disable cors security filter at all, but only applies the default cors configurations
Comment: I don't think CORS would return a 403. Try removing `.antMatchers("/**").hasRole(...)` and change to `.anyRequest().hasRole(...)` instead of `.anyRequest().authenticated()`.
Comment: Are you using an embedded servlet container? Or deploying on an external web container like Tomcat?
Comment: Look at the error via `wget --content-on-error`?
Comment: I'm also wondering what you're deploying the application to, and how it manages the app. Are you deploying using Tomcat with a context path? (e.g. *myremotehost.domain.com/myapp/client/abc*) or a Spring Boot app that runs at the root of a virtual host? (e.g. *myremotehost.domain.com/client/abc*)
Comment: Are u using contextpath in both the condition?
Answer: The cors().disable() does not disable cors security filter at all, but only applies the default cors configurations.
@Override
protected void configure(HttpSecurity http) throws Exception {
    http
    ...
    .cors().configurationSource(corsConfigurationSource())
    ...
}

@Bean
public CorsConfigurationSource corsConfigurationSource() {
    final CorsConfiguration configuration = new CorsConfiguration();
    configuration.setAllowedOrigins(List.of(CorsConfiguration.ALL));
    configuration.setAllowedMethods(List.of(CorsConfiguration.ALL));
    configuration.setAllowedHeaders(List.of(CorsConfiguration.ALL));
    configuration.setAllowCredentials(true);

   // ideally CorsConfiguration.ALL should not be used in production

    configuration.addExposedHeader(HttpHeaders.AUTHORIZATION); // The headers you expose in response

    final UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
    source.registerCorsConfiguration("/**", configuration);

    return source;
}

Comment: No, it didn't help.
Answer: Are you using @EnableWebSecurity in the class to enable mvc security settings?. Test with .sessionRegistry(sessionRegistry())
@Bean
public SessionRegistry sessionRegistry() {
    return new SessionRegistryImpl();
}


How do you stop a WebBrowser object in WPF?

Question: How do you stop a WebBrowser object in WPF?
The stop method is not an option because that is only available in Windows Forms.
WebBrowser.Stop Method (System.Windows.Forms)
And Dispose does not seem to be the answer either.
And, now that I mention it. Is there a way of easily finding a corresponding method in WPF that exists in a Windows Form?
Comment: Did you ever come up with a good solution to this?
Answer: Try to navigate to "about:blank" or
use two WebBrowsers, first visible, second hidden. When you stops first just hide it and show empty second
Comment: I am not using the webbrowser class for its visible properties to the user.  I am using it to navagate several urls in a row.  This can have a wierd effect where the browser starts sending back load completed messages and I just want it stopped.  Doing a swapping tric between two webbrowser classes will not be good enough.
Comment: I agree. just standard solutions do not always contain all that you may need, and sometimes even ugly solutions help out


Bochner Integral: Axioms

Question: Disclaimer
It is meant to record. See: Answer own Question
It is written as question. Have fun! :)
Reference
It is taken from the original paper: S. Bochner, Integration
It is related to: Bochner Integral: Integrability
Definition
Given a measure space $\Omega$ and a Banach space $E$.
Consider Bochner measurable functions $S_n\to F$.
Suppose Bochner integrability as $\int\|F\|\mathrm{d}\mu<\infty$.
Denote the abstract Bochner integral by $\int F\mathrm{d}\mu$.
Define the Bochner integral axiomatically by:
$$\text{(L)}\quad\int(F+G)\mathrm{d}\mu=\int F\mathrm{d}\mu+\int F\mathrm{d}\mu,\,\int\lambda F\mathrm{d}\mu=\lambda\int F\mathrm{d}\mu$$
$$\text{(N)}:\quad\left\|\int F\mathrm{d}\mu\right\|\leq\int\|F\|\mathrm{d}\mu$$
$$\text{(DC)}:\quad\|F_n\|\leq h:\quad F_n\to F\implies\int F\mathrm{d}\mu\to\int F\mathrm{d}\mu\quad\left(\int h\mathrm{d}\mu<\infty\right)$$
And especially for finite measures:
$$\text{(C)}:\quad\int C\mathrm{d}\mu=C\mu(\Omega)$$
Construction
This forces the integral for simple functions to become:
$$S=\sum_{k=1}^KS_k\chi(A_k):\quad\int S\mathrm{d}\mu=\sum_{k=1}^KS_k\mu(A_k)$$
Now, for Bochner measurable functions Bochner integrability should determine the integral, too:
$$S_n\to F\implies\int S_n\mathrm{d}\mu\to\int F\mathrm{d}\mu$$
But how to apply dominated convergence therefore:
$$\|F-S_n\|\to0\implies\|S_n\|\leq\|F\|$$
(I got it now; in any way, you are warmly encouraged to do give an answer, too!!)
Discardure
Introduce the restricted integral through $\int_A F\mathrm{d}\mu:=\int\chi_AF\mathrm{d}\mu$.
There is another axiom which seems not independent of the others:
$$\text{(S)}:\quad\int_A F\mathrm{d}\mu=\sum_{k\in\mathbb{N}}\int_{A_k}F\mathrm{d}\mu\quad\left(A=\bigsqcup_{k\in\mathbb{N}}A_k\right)$$
(Is it really independent?)
Comment: Is the reference really Bochner's original paper?
Comment: Isn't the axiom on disjoint sums redundant? (S)
Comment: Dude, you don't have to ask yourself questions in the comments also.
Comment: @Michael: Haha ^^ no these are really questions.
Answer: One can choose an appropriate sequence:
$$\|F-S_n\|\downarrow0$$
This way one obtains a dominant:
$$\|F-S_n\|\leq\|F-S_0\|:\quad\int\|F-S_0\|\mathrm{d}\mu\leq\int\|F\|\mathrm{d}\mu+\int\|S_0\|\mathrm{d}\mu<\infty+\infty$$
Thus by dominated convergence and linearity:
$$\int F\mathrm{d}\mu-\int S_n\mathrm{d}\mu=\int(F-S_n)\mathrm{d}\mu\to0$$
(For more details see: Bochner Integral: Integrability)

Bochner Integral: Axioms


HTML Email Responsive Table Arrangment

Question: I am trying to make a group of tables show up on mobile like it does on desktop.
This is what I am trying to do, but I can't figure it out:

Here is my code:

Comment: I don't have time for a detailed analysis, but what jumps out at me is that you have a bunch of "ghost" code to force desktop behavior on Outlook. If you want to force desktop behavior everywhere, you need to remove the MSO conditional code and let the HTML rule everywhere.
Answer: Your example above contains hybrid and responsive code to make the email change layouts from desktop to mobile. Sounds like you want to remove that.

Remove the following code:
Responsive Classes: Remove instances of class="mcnImage" from and  or  tag that you don't want to be full width on mobile.
Ghost Table Markup for Outlook: remove instances of , including what's inside those two tags.

Once you remove the code that makes this part of the email responsive, your layout should go back to the desktop view even on small screens.

HTML Email Responsive Table Arrangment


Regular expression from the hostname in the proxy_pass uri in Nginx

Question: I want to do something like:
server_name   ~^(www\.)?(?<subdomain>[^.]+).example.org$;

location / {
    proxy_pass      "http://example.org:8080/$subdomain";
}

Answer: Untested, but give this a shot:
server_name   ~^(www\.)?[^.]+.example.org$;

if ($host ~* ^(www\.)?([^.]+).example.org$) {
    set $subdomain $2;
}

location / {
    proxy_pass http://example.org:8080/$subdomain;
}

Answer: Tested and working:
server_name "~(.*).dummy.local";

location / {
    echo $1;
}

When requesting foo.dummy.local, it will display foo.
curl -vvv foo.dummy.local
* Rebuilt URL to: foo.dummy.local/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to foo.dummy.local (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: foo.dummy.local
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.10.3
< Date: Thu, 20 Sep 2018 09:15:54 GMT
< Content-Type: application/octet-stream
< Transfer-Encoding: chunked
< Connection: keep-alive
<
foo
* Curl_http_done: called premature == 0
* Connection #0 to host foo.dummy.local left intact


How can I remove the effect of one independent variable so I can standardize and compare the values of my dependent variable?

Question: I have a table of television viewership data with each row being one series and the columns being various data about that series, e.g. name, time the series is on TV, length of an episode, how many households watched the show, and the average percent of the show that was watched among people who did watch it.
There is a clear relationship between episode length and average percent views, with longer shows having lower average percent views and shorter shows having higher. I have included a screenshot of these plotted against each other.

My question is, I want to compare average percent viewed between episodes without the effect of episode length to create an index with 1 being average and values above or below indicating better or worse to a standardized degree.
My currents ideas are either...
1) Group series by episode time, find the average "average percent watched" for each group and index on that.
2) Create a linear regression of average percent views and episode length and compare individual episode APV scores to that line, being higher or lower.
How should I go about this? Are either of those ideas statistically sound? Is there something else I should be doing? I am using R to work with this data, if that is relevant.
I apologize if this is too vague, I'm struggling a bit with this and it has been some time since I have worked with this kind of data.
Comment: Stratification, idea 1, is a good place to start. The thing is episode length is correlated with other characteristics of series. The reason we do regression using multiple series characteristics is to isolate the unique effect of episode length. Before going on, the question then is: why are you trying to extract episode length from the data in the first place?
Comment: The thought is that the two primary components of "percentage watched" are length of episode and viewer engagement. By moving the effect of episode length, I can compare engagement between shows of different length. Does that make sense?
Comment: *removing. Also, would plotting a line of "averages" and comparing differences of points with that line not be more accurate than binning and comparing to bin average? My worry with the bins is that shows with run times in between say, 30 mins and 60 mins, may get an unfairly low index depending on which bin they're in.
Comment: @Donovan192 do you have any variables that measure viewer engagement? If not, a simple linear regression will allow you to separate the effect of episode length from everything else, but you will not be able to see the components of "everything else".
Comment: @candamir That's fine with me, that's what I'm aiming for! Is a linear regression appropriate here or a different type? The chart above looks a bit curved to me. And is it the residuals I'm interested in, then, to see how far a point is from the line that accounts for the effect of run time on percentage viewed? Again it's been a while since I've done regression, so I hope this isn't too vague.
Comment: If you want to compare engagement between shows of different lengths, you can simply run a multiple regression with both variables, engagement and length, as your predictors. This is what regression coefficients allow you to quantify. You can also transform length since its relationship is not a linear one with the outcome, maybe log(length) will work.
Answer: Idea nr. 2 does not achieve what you want. Simply speaking, regressing APVs on show length tells you what the impact of the latter is on the former across the entire sample - which, if I understand you correctly, is precisely what you don't want.
In case you have other variables in your dataset, then a multiple linear regression might help you. By regressing APVs on show length AND (as an example) the budget of the show AND (as an example) the age of the show, you will be able to isolate the effects from the latter three variables on APVs: the estimated coefficient for the budget of the show will tell you the impact of the budget on APVs while holding the other two explanatory variables constant.
You can even divide the shows into qualitative groups by creating dummy variables (e.g. a variable western that is 1 for all Western shows and 0 otherwise, etc.). Let me know if this is of interest and I can elaborate further. (In particular, don't forget to use cross terms when including dummy variables among your regressors.)


Variance vs Standard Deviation vs SE vs Var(beta hat)

Question: How I understand it:
Sum of squares is:
$$\newcommand{\ybar}{\bar y} (y_i - \ybar)^2$$
Variance is:
$$\frac{(y_i - \ybar)^2}{n}$$
When variance is from a sample
$$\frac{(y_i - \ybar)^2}{n -1}$$
Standard deviation is square root of the variance
$$\sqrt{\frac{(y_i - \ybar)^2}{n}}$$
Sample standard deviation is square root of the sample variance
$$\sqrt{\frac{(y_i - \ybar)^2}{n -1}}$$
Is this correct?
Now standard error is where I start to get a bit confused
$SE = \frac{s}{\sqrt{n}}$ where $s$ is the sample standard deviation which is $\sqrt{\frac{(y_i - \ybar)^2}{n -1}}$
So why is standard error the standard deviation divided by square root $n$?
Is $s$ as I have defined it?
My next question is for multiple regressions it talks about finding the variance (beta-hat). I don't even know what that is talking about as there are multiple parameters I am guessing that it is using beta-hat as a substitute for all parameters in the model?
I don't understand the following:

It says that the estimate is the "variance of the unknown errors"
multiplied by the identity matrix

It says that it is equal to $\frac{e'e}{n-p} = \frac{SE}{n-p}$
So I assume that this is some sort of derivation of the variance formula for multiple regression parameters.
Is the $p$ a replacement of the "1" in the other sample formula? What is $Se$ in the above case?
What is "variance of the unknown errors"?
When I check the SSE for my model by using the following R commands:
<code>SSE <-sum(resid(model1)^2)
n   <-length (resid(model1))
p   <- length(coef(model1))

SSE/(n-p)
</code>
So this is saying that it is the sum of square error divided by $n-p$ , which is closest to $s^2$ or the variance but the formula is slightly different, so what exactly is going on here?
Comment: also ... Isn't SSE/n-p just the MSE ?
Comment: Where are the $\Sigma$'s ?
Comment: I don't know how to do sums but I assumed they would be assumed, lol.
Answer: I do not have the time to give a detailed answer, but since no one has helped you so far, I will give some hints.
Is this correct? and Is s as I have defined it?
Your formulas seem to be fine, they just lack the summations, as noted by @Gilles.
So why is standard error the standard deviation divided by square root N?
Standard error is useful for significance testing. It is used in $t$-tests. Without the division by $\sqrt{N}$ the $t$-tests would not work. You should be able to find more information in econometrics textbooks.
I am guessing that it is using beta-hat as a substitute for all parameters in the model?
That could very well be the case. $\hat{\beta}$ would be a parameter vector rather than a single parameter. Its variance would be a matrix rather than a single number.
"variance of the unknown errors" <--- don't know what that means
In a linear regression model of the form $y=\beta_0+\beta_1 x_1+\dotsb+\beta_K x_K+\varepsilon$, variance of the unknown errors would be the variance of $\varepsilon$, denoted $\sigma^2$; it is a single number (not a matrix). It can be estimated as the sample variance of the model residuals, denoted $\hat{\sigma}^2$.
However, the variance of the linear regression parameter vector $\hat{\beta}$ is $\text{Var}(\hat{\beta}|X)=\sigma^2 (X^T X)^{-1}$, which is not the variance of the unknown errors multiplied by the indentity matrix.
A note on what you did not ask: if you want to really understant what is going on in the OLS estimation applied on a linear regression model, you should take your time and study the subject carefully and with patience. Just getting your questions answered like here may not be useful in the long run.


Javascript - Problema em função que adiciona textbox

Question: Na minha página em PHP e Javascript, criei um espaço que contém duas Textbox e uma label clicável pra adicionar mais Textbox, se necessário, ao div numPart:

<input name="part" id="" class="validate[required]"></input>

<input name="part" id="" class="validate[required]"></input>

ADICIONAR PARTICIPANTE

E a função que adiciona mais textbox é a seguinte.
var x=1;
$("#addPart").click(function(){
   var d = document.getElementById('numPart');
   d.innerHTML += "
<input name='part' id='' class='validate[required]'></input>
";
});

Contudo, se eu tiver preenchido alguma das Textbox existentes e clicar pra adicionar mais uma, o texto existente que eu preenchi some.
Como eu posso fazer pra manter os textos escritos nas Textbox existentes a medida que eu for clicando pra adicionar mais ao formulário?
Answer: Neste caso deves utilizar a função .append visto que cada vez que fazias o +=, para ele era como se estivesses a dar um html completamente novo.

var x=1;
$("#addPart").click(function(){
   $("#numPart").append("
<input name='part' id='' class='validate[required]'></input>
");
});

<input name="part" id="" class="validate[required]"></input>

<input name="part" id="" class="validate[required]"></input>

ADICIONAR PARTICIPANTE
Answer: Copiar sempre os conteudos do ultimo input:
$("#addPart").on('click', function() {
     var newField = $('.inputMGM').last().clone();
    $(this).before(newField)
});

Exemplo
Comment: Ótima ideia usar o `last()`. +1
Comment: Obrigado @PedroCamaraJunior
Answer: O innerHTML substutui todo o conteúdo, para adicionar mais conteúdo no final utilize o append.
Agora dois pontos no código existente:

Você está utilizando javascript e jQuery em um mesmo código, se usar jQuery, use o máximo que puder
Não existe <input></input> a tag é fechada nela mesma - <input type="text" />.

Veja o exemplo com as melhorias apontadas.

$("#addPart").on('click', function() {
  var html = "
<input name='part' id='' class='validate[required]' />
";
  $('#numPart').append(html);
});

    <input name="part" id="" class="validate[required]" />

    <input name="part" id="" class="validate[required]" />

ADICIONAR PARTICIPANTE


What is a good way to think of Factor Groups?

Question: I'm having a hard time thinking about factor groups. I just don't understand what notation like $\mathbb{Z}_{60}/\langle 12 \rangle$ means. Furthermore, when asked about giving the order $26 + \langle 12 \rangle$ in $\mathbb{Z}_{60}/\langle 12 \rangle$, I don't know how to picture it.
I've seen the solution that $26 + \langle 12 \rangle = 2 + \langle 12 \rangle$. The cyclic subgroup of $2 + \langle 12 \rangle$ is $\{\langle 12 \rangle, 2 + \langle 12 \rangle, 4 + \langle 12 \rangle, 6 + \langle 12 \rangle, 8 + \langle 12 \rangle, 10 + \langle 12 \rangle\},$ hence we have an order of 6, but I don't see the relation between that and $\mathbb{Z}_{60}/\langle 12 \rangle$. I can't seem to visualize it.
If I could see another solution with $\mathbb{Z}_{60}/\langle 12 \rangle$ formulaically broken down into its components, I think I would have a much easier time with all of this.
I understand $\mathbb{Z}_{60}$ is quite large, so you could use a much simpler case. Thank you very much.
Comment: Your second equality on that line is wrong. On one side you have a subgroup, and on the other you have just one element (ie, one coset).
Comment: You mean the $2 + \langle 12 \rangle$?
Comment: Yes, that is one coset, ie one element in the quotient group.
Comment: I would suggest that you read Abstract Algebra by Dummit and Foote. The chapter on Homomorphisms explains it really well, with pictures.
Comment: @TobiasKildetoft, I thought $26 + \langle 12 \rangle$ is equivalent to $2 + \langle 12 \rangle$ because $26 \equiv 2 \mod 12$.
Comment: Yes, that equality is correct. It is the next one in the line that is wrong.
Comment: Ah, yes. Edited.
Comment: [A somewhat related question](http://math.stackexchange.com/q/69050/11619).
Answer: You should always think of quotients as being equivalence relations.  If $\sim$ is an equivalence relation on $A$, then $A/\sim$ is the set of equivalence classes.  You can visualize this by imagining that you have just taken $A$, but somehow identified (or, to use a visual word, glued) $a$ and $b$ whenever $a\sim b$.
What you are doing here is strengthening equality, or forcing some things to equal some other things.  For example, maybe I'm working with the set $\{x,y\}$, where $x$ represents my brother and $y$ represents my roommate.  But then, maybe I move in with my brother—now this is the wrong set to use, and I should use $\{x,y\}/\sim$, where $x\sim y$, because I now want to think of my brother and my roommate as two different people.
A more complicated example would be the group $\mathbb{Z}/24\mathbb{Z}$.  The equivalence relation here is $a\sim b$ whenever $a-b$ is divisible by $24$.  You can think of this as the set of hours in a day, because there's always a next hour, and always a previous hour, but we want to think of hours separated by $24$-hour increments as being "the same".
With a factor group (or, the more modern term "quotient group"), you have a group $G$ and a subgroup $H$. You want to know what happens when you force every element of $H$ to be equal to every other element, but still require the group operation to $\color{red}{(missing text)}$.  (e.g. above, we set $0=24=48=\ldots$, which forced $1+0=1+24=1+48=\ldots$).  We usually assume that $H$ is normal for technical reasons (if we glue together the elements of $H$, it turns out that we are forced to also glue together the elements of the smallest normal group containing $H$).
Then, with this little technical condition that $H$ is normal, we are just asking for $G/\sim$, where the equivalence relation is $x\sim y$ whenever $xy^{-1} \in H$.  We glue together elements in such a way that everything in $H$ will be the identity of our new group.  The even integers above are an example; another good one is the projection map $(x,y)\mapsto x$ from $\mathbb{R}^2\to\mathbb{R}$ as abelian groups—we are forcing every pair $(0,y)$ to be equal to $(0,0)$, which also forces $(x,y)$ to be equal to $(x,0)$ for every $x$.  If you think about it, this is a demonstration that $\mathbb{R}^2 /\mathbb{R} \cong \mathbb{R}$ as abelian groups.  More generally, we could describe the projection of a vector space $V$ onto a subspace $W$ as $V \to V/W^\perp \cong W$.
The quotient of a more complicated group is harder to visualize.  But the basic ideas are all here.  When you have a piece $H$ of a group $G$ that you want to ignore, the quotient group is almost always the way to do it.  You are saying "This part ($H$) is unimportant, but let's see what happens when we get rid of it."  The result of this thinking is $G/H$.
Comment: What do you mean when you say we assume $H$ is normal for technical reasons?
Comment: @JackM Essentially, if $H$ isn't a normal subgroup then just directly 'gluing together' elements that differ by a member of $H$ won't generate a clean subgroup.  (Technical explanation: because the cosets of H will have non-trivial intersection; you can find elements $g_1, g_2\in G$ such that the sets $g_1H = \{g_1h: h\in H\}$ and $g_2H$ have some but not all elements in common.)
Answer: Here's some intuition on the meaning of a factor group.
Reference: http://www.math.toronto.edu/lgoldmak/301F09/LS10-27.pdf

Reference: http://cdsmith.wordpress.com/2009/04/11/needing-intuition-in-maths-one-example/
Most people learning abstract algebra, as far as I can tell, have no idea why homomorphisms and factor groups are sensible things to think about.  They quickly come to understand the idea of a group, and enough varied examples are usually given that they can see how the idea of a group applies to a number of things.  They quickly come to terms with subgroups, though the idea looks rather trivial to them.  Then you get to homomorphisms and factor groups; at this point, most classes run out of intuition and just jump in for some unmotivated mathematical constructions.
I'm not quite sure why this is, honestly.  Anyone with the slightest modicum of mathematical curiosity probably has thoughtn about factor groups since they were seven or eight years old.  In the context of integers and addition, most children realize on their own (whether it's taught to them or not) that the sum of two even numbers, or of two odd numbers, is even, while the sum of an even number and an odd number is odd.  This is, of course, a factor group.  Students who are presented with the mathematical definition of a factor group should first have, in their set of mental tools, this simple intuitive definition:
Factor Group: For any group (G,*), a factor group is a group that is obtained by being sufficiently sleep-deprived (or perhaps drunk, depending on the university) that one can't tell the difference between some members of the original group, and then trying to write down a group table.
Of course, one then goes on to point out that sometimes this works, but sometimes it doesn't.  If one looks at the integers and only sees "even" or "odd", then it works.  If one looks at the integers and only sees "negative" or "non-negative", then it doesn't work, since the sum of a negative number and a positive number could be either negative or positive.  It then becomes natural to ask when it works, and when it doesn't.  This provides a justification, then, for nailing down the abstract definitions, defining normal subgroups, proving that the factor group is well-defined when modding out a normal subgroup, and so on.  First, though, the student needs to be convinced that these are natural things to think about.
Speaking of defining normal subgroups, it is really inexcusable how many students have never even noticed the close relationship between normal subgroups and commutativity.  Sure, everyone knows that all subgroups of an abelian group are normal; but this seems to be treated as a sort of occasionally useful curiosity.  Few students are even exposed to the simple fact that normality of subgroups is inextricably entwined in the degree to which the subgroup commutes with the surrounding group.
Reference: http://scienceblogs.com/goodmath/2007/12/27/the-meaning-of-division-quotie/
A quotient group is a shining
example of the beauty of abstract algebra. We've abstracted away from numbers to these
crazy group things, and one reward is that we can see what division really means. It's
more than just a simple bit of arithmetic: division is a way of describing a fundamental
structural relationship that pervades mathematics.
So what is division all about?
Suppose you want to divide 50 by 5. What you're really doing is saying
you've got a collection of 50 indistinguishable things, and you want to break it into a 5
indistinguishable collections. Since you started with 50 indistinguishable things, that means
that you'll end up with 5 sets of 10 things.
That's pretty simple, right? Now, suppose that we're not talking about simple numbers. Instead we
want to work in terms of groups. Can we take that basic concept of division, and find some meaningful way of applying it to groups?
Well, first, we need to somehow talk about division in a way that doesn't involve numbers. We start with a group – that is, a collection of objects with some kind of meaningful structure. What can we divide it by? A group with a similar structure – in fact, a group with the same structure: a subgroup But not just any subgroup: it's got to be a normal subgroup, because that's the kind of subgroup that properly preserves the structure of the group.
So what happens when we divide a group by one of its normal subgroups? We partition the
group into a new group, where the elements of the new group are formed from subsets of the
elements of the original group. It's the same idea as simple integer division described up above, except that we want to preserve the group structure, so the result is going to be a group.
Comment: Beautiful explanation!
Answer: I suppose that $\mathbb Z_{60}/\langle 12 \rangle$ indicate the group obtained quotienting $\mathbb Z_{60}$ its subgroup generated by the element $12$: usually notation of the kind $\langle X \rangle$ means the smallest sub-structure (group, ring, module, ideal, etc) containing $X$, i.e. generated by $X$.
Generally when you have a group $G$ and a normal subgroup $H \lhd G$ the quotient group is the group having as support (or carrier) the set
$$G/H = \{gH \mid g \in G\}$$
where each $gH$ is a coset and is a subset of $G$ defined as
$$gH = \{ gh \mid h \in H\}$$
In case the group $G$ is abelian the is also used the notation $g+H$ to indicate the sets $gH$.
In the case you are considering the cosets
$$x + \langle 12 \rangle  = \{ x + 12*k \mid k \in \mathbb Z_{60}\}$$
more longly this set is the set
$$\{x + 12, x + 24, x + 36, x + 48, x + 60 = x \}$$
which has cardinality $5$.
Since every element $x + \langle 12 \rangle$ is an element of the group $\mathbb Z_{60}/\langle 12 \rangle$ it makes sense to ask what is the order of such element in the group: i.e. for is the smallest positive integer $n \in \mathbb N \setminus \{0\}$ such that $\sum_{i=1}^n (x + \langle 12\rangle)=n(x+ \langle 12 \rangle) = 0$.
It's easy doing the count observe that $26 + \langle 12\rangle = 2 + \langle 12 \rangle$ and that the smallest $n \in \mathbb N \setminus \{0\}$ such that
$$n(2 + \langle 12 \rangle) = 0 + \langle 12 \rangle = 12 + \langle 12 \rangle$$
is exactly $n=6$ which is the order of the element:
indeed we have that such elements are
$$1(2 + \langle 12 \rangle)=2 + \langle 12 \rangle$$
$$2(2 + \langle 12 \rangle)=4 + \langle 12 \rangle$$
$$3(2 + \langle 12 \rangle)=6 + \langle 12 \rangle$$
$$4(2 + \langle 12 \rangle)=8 + \langle 12 \rangle$$
$$5(2 + \langle 12 \rangle)=10 + \langle 12 \rangle$$
$$6(2 + \langle 12 \rangle)=12 + \langle 12 \rangle= \langle 12 \rangle$$
Hope this helps.
Comment: All the cosets have the same cardinality, regardless of $x$. And the order is $6$, not $5$.
Comment: Although I agree with @TobiasKildetoft remark, this does really help clear it up. So when searching for order, I am interested in essentially the steps it takes to get to 0, which would be the steps from 0 to 12 in this case, and we're stepping up by 2. So, we have 2, 4, 6, 8, 10, 12 -> order of 6?
Comment: @TobiasKildetoft yes sorry, thanks for pointing out.
Comment: apparently hunger can help in making mistakes :P, it should be correct now.
Comment: @David I have added the details of the proof that the order of $2+\langle 12 \rangle$ is $6$.


JavaScript: Existe uma forma de fazer um do-while testar a condição antes do fim do laço?

Question: Contexto: Estou desenvolvendo um RPG baseado em texto e na hora de fazer a mecânica da batalha estou tendo esse 'problema'.
Ex:
    function verificarEstado(pessoa){
     if (pessoa.estado != 'normal') {
      /* aqui terá um switch para cada efeito tipo envenenado, sangrando, etc, esses efeitos podem causar dano, então pode ser que um dos combatentes morra nessa etapa*/
      verificarVida(pessoa.vida);
    }

    function verificarVida(pessoa){
      if (pessoa.vida<=0){
        /* aqui vem a condição para parar o laço, pois se um dos combatentes morreu, é o fim da luta */
      }
    }

    function realizarAtaque(atacante, alvo){
     /* Não vou entrar em detalhes aqui porque não é relevante, o ponto é que eu não quero que essa etapa seja executada se um dos personagens tiver morrido no 'verificarEstado()  */
    }

    function iniciarBatalha(jogador, inimigo){
      let ambosVivos=true;
      let turno=1;

       do{

        verificarEstado(jogador);
        verificarEstado(inimigo);

        realizarAtaque(jogador, inimigo);
        realizarAtaque(inimigo, jogador);

       turno++;
       } while (ambosVivos);
      }
iniciarBatalha(jogador,inimigo) /* Chamada para o inicio da batalha    */

Comment: Qual problema você está tendo? Não ficou claro
Comment: Não entendi a duvida?
Comment: dentro do Do-while estou executando 4 funções, certo?

Quero saber se existe uma forma de, por exemplo, a primeira dessas funções já trocar a condição de verificação e as outras não ser executada. Nesse contexto, um personagem pode morrer na primeira dessas quatro funções, então não faria sentido ele tentar dar um ataque na função seguinte. Eu poderia colocar um if(ambosVivos) envolvendo as funções realizarAtaque(), mas não parece ser a melhor forma. Eu to querendo algo como o "continue" do laço For.
Answer:
JavaScript: Existe uma forma de fazer um do-while testar a condição
antes do fim do laço?

Respondendo diretamente o título da pergunta
Em qualquer estrutura de repetição em JS, você pode realizar um break para parar o loop, ou continue para iniciar a próxima iteração e interromper a atual. O continue neste caso somente irá para a próxima iteração se a condição do loop permanecer verdadeira.

function loop() {
  let i = 0
  do {
    if(i ==3) {
      i += 2
      alert("Continue irá pular para a próxima iteração")
      continue
    }

    if(i == 5) {
      alert("Parar o loop")
      break
    }
    alert("iteração normal")
    i++
  } while(i < 6)
}

loop()

um personagem pode morrer na primeira dessas quatro funções, então não
faria sentido ele tentar dar um ataque na função seguinte.

Sendo bem simples na solução de acordo com o que você nos mostrou, as funções que podem parar o loop devem retornar um booleano indicando se o loop deve ou não continuar. Caso não deva continuar, você pode realizar um break.


Error updating record You have an error in your SQL syntax

Question: I try to update my table but I give this error please help me
function updatePost($id , $title, $content, $date, $groups)
    {
        $connection = mysqli_connect(DataBaseManager::HOST, DataBaseManager::USER, DataBaseManager::PASSWORD, DataBaseManager::DATABASENAME);
        $sqlCommand = "UPDATE posts
          SET title = '$title', content = '$content', date = '$date' , groups = '$groups'
          WHERE id == 1";
        if ($connection->query($sqlCommand) === TRUE) {
            echo "Record updated successfully";
        } else {
            echo "Error updating record: " . $connection->error;
        }

        $connection->close();
    }

Answer: You need to treat your SQL query like a string and concat your variables accordingly:
"UPDATE posts SET title = '".$title."', content = '".$content."', date = '".$date."' , likes = '".$likes."', groups = '".$groups."' WHERE id = '".$id."'";

Comment: oh thank you its work but now if ($connection->query($sqlCommand) === TRUE) {
            echo "Record updated successfully";
        } else {
            echo "Error updating record: " . $connection->error;
        }      return nothing ... and table in data base dont update
Comment: As Dave mentioned above, be sure your WHERE clause only includes a single '='
Comment: Also, if our answers have been helpful, please upvote and/or accept.
Comment: I'm pretty sure that the WHERE clause with the double = is not going to work.
Answer: Your where clause has 2 equal signs. Change it to a single equal sign ... also, date is a reserved word so you'll need to wrap it in backticks.
$sqlCommand = "UPDATE posts
    SET title = '$title'     ,
         content = '$content',
        `date`   = '$date'   ,
         groups  = '$groups'
    WHERE id = 1";

Comment: thank you but now my if ($connection->query($sqlCommand) === TRUE) {
            echo "Record updated successfully";
        } else {
            echo "Error updating record: " . $connection->error;
        }    return nothing ... and table dont update... what do you think about that??


AgGrid custom filter not created until interaction

Question: I have a custom filter component on a column in my grid. I've noticed that the filter component is not constructed until I click on the filter icon in the column header. This means that the data is not filtered according to my default settings (e.g., filter out records where status == StatusEnum.Complete).
As a workaround, I've found that I can get a filter instance in the onGridReady event by calling api.getFilterInstance('status'), and this causes the filter component to be created and thereby apply default filtering.
This workaround seems a bit clunky. The filter variable is unused in the onGridReady event, which causes warnings in the IDE / build. Another developer may come along and delete this line of code, thinking it is unnecessary.
Is there a better way to force my custom filter to be instantiated when the grid is created? I'm using AgGrid 17.1 and Angular 4.4.
The grid is configured like so:
gridOptions: GridOptions = {
    enableFilter: true,
    onGridReady: (event) => {
        let filter = event.api.getFilterInstance("status"); // forces the filter component to be constructed
        let data = this.loadAsyncData();
        event.api.setRowData(data);
    },
    columnDefs: [
    ...
    {
        headerName: "Status",
        field: "status",
        filterFramework: MyCustomStatusFilterComponent,
        filterParams: {
            valueGetter: (obj) => { return obj.data.statusEnum; },
            hideCompleteByDefault: true,
            ...
        }
    },
    ....
    ]
}

I've set up an example that demonstrates the issue. Note the "hack" on line 63 of app.component.ts.
Comment: I'm sort of interested in looking at this, but you don't have a test case, which makes it difficult to help you. Can you set up a demo showing the problem? You can start [from this](https://plnkr.co/edit/oZ5hOkDw11nONZVNoLOo?p=preview) or any other example from the [documentation](https://www.ag-grid.com/documentation-main/documentation.php).
Comment: @thirtydot plunkr added. thanks!
Answer: First of all - it's two independent thing, custom filter and filter initialization.
I suppose you've mixed those two phases and trying to achieve an unexpected result.
Your custom filter shouldn't contain pre-defined logic cuz init will occur only with first touch of your filter, you have to divide your logic and then in onGridReady you can execute setModel with needed things.
or keep a hack as you've already mentioned
Comment: This approach looks like it will work for me. I'm still curious, though, why the filters are not initialized until they are first touched. Is this somewhere in the `ag-grid` documentation?
Comment: @djs: I can't see it explicitly in the docs, but it doesn't matter. If you look in the source, it's "initialise on demand", and it's almost certainly for performance reasons. Look at the very first feature they list: https://www.ag-grid.com/features-overview/ ("Outstanding Performance"). Think about it: the set filter has to go through every one of your [(set it to: 100,000) rows](https://www.ag-grid.com/example.php) to collect the unique values. If you have 22+ columns of this, the grid will lag on startup (especially on slow hardware).
Answer: Here You Use Custom Filtering,
If Your Row Not Bind with Value Then You Use Custom Filtering.
         {
            headerName: "Date",
            field: "date",

            //Custom  Filter Start

            filterValueGetter: (params: ICellRendererParams) =>
            {
              if (this.transactionIsEmpty(params.data))
              {
                const tx: Transaction = params.data;
                return moment(tx.date).format('DD-MM-YYYY');
              }
            },

            //Custom  Filter End

            cellStyle: { 'text-align': 'left' },
            minWidth: 250,
        }


Custom field widget form - Multiple values

Question: I've create a custom field form widget for an entity reference field. When only one value is allowed it shows a (filtered) select list, but I want to allow user to assign multiple reference values to that field.
In my use case I want it to be shown as checkboxes when multiple values are allowed, but by default it shows my checkboxes in a 'add-another' fieldset.
How can this be accomplished?
Below some code, $type is now a field setting and is manually set.
// Construct element
$widget = $element;
$widget['#delta'] = $delta;
$type = $settings['option_limit_type'];

... extra logic ...

switch ($type) {
      case 'select':
        $widget += array(
          '#type' => 'select',
          '#prefix' => '
',
          '#suffix' => '
',
          '#options' => $options,
          '#delta' => $delta,
          '#element_validate' => array('_entityreference_hierachy_option_limit_validate'),
          '#default_value' =>  $saved_target_id,
        );
        break;
      case 'checkboxes':
        $widget += array(
          '#type' => 'checkboxes',
          '#prefix' => '
',
          '#suffix' => '
',
          '#options' => $options,
          '#delta' => $delta,
          '#element_validate' => array('_entityreference_hierachy_option_limit_validate'),
          '#default_value' =>  $saved_target_id,
        );
        break;
    }
return $widget;

Answer: You can use drupal form api attributes like

 case 'select':
        $widget += array(
          '#type' => 'select',
          '#prefix' => '
',
          '#suffix' => '
',
          '#options' => $options,
          '#delta' => $delta,
          '#element_validate' => array('_entityreference_hierachy_option_limit_validate'),
          '#default_value' =>  $saved_target_id,
          '#multiple' => $multiple,
          '#size' => $multiple ? min(12,count($options)) : 0,
        );
        break;

https://api.drupal.org/api/drupal/developer!topics!forms_api_reference.html/7#multiple
Comment: It still shows the 'add-another' fieldset when multiple values are allowed in the field settings. I would prefer it changes based on this field setting.
Comment: It can be set with the behaviors in the hook_field_widget_info.
https://api.drupal.org/api/drupal/modules!field!field.api.php/function/hook_field_widget_info/7
Answer: In hook_field_widget_info() use 'FIELD_BEHAVIOR_CUSTOM' like this:
 'field_my_checkboxes' => array(
  'label' => t('Let the user select from a set of checkboxes.'),
  'field types' => array('field_myfield'),
  'behaviors' => array(
    'multiple values' => FIELD_BEHAVIOR_CUSTOM,
  ),

In this case 'Add another' link and fieldset won't be shown but you need to handle user's input manually. Example: https://www.drupal.org/node/1464770


O que é savedInstanceState?

Question: Sou novata no android e quero aprender mais então estou no github vendo uns projetos prontos e gostaria de saber o que é isso:
  @Override
    protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_tempo_agora_main);
    if (savedInstanceState == null) {
        getSupportFragmentManager().beginTransaction()
                .add(R.id.container, new PlaceholderFragment())
                .commit();
    }
}

Esse savedInstanceState o que isso faz? Aliás me explica o que esse método faz?
Comment: Já foi dada a resposta, mas complemento dizendo que ela serve para quando voce gira a tela...Sabe? Nesse momento, a activity é destruída e reconstruída toda do zero. Então, na hora que alguém gira a tela, voce programa pra salvar os dados que estão naquela tela e depois os poe novamente na tela, agora em outra orientação (landscape ou portrait)
Comment: Direto da fonte a explicação https://developer.android.com/reference/android/app/Activity.html#onSaveInstanceState(android.os.Bundle)
Comment: Meu inglês é uma bosta...não tem português não moço?
Answer: savedInstanceState é um parâmetro do método onCreate() que recebe um argumento do tipo Bundle.
Ele é usado pelo sistema para, ao recriar uma Activity, permitir restaurar o estado que ela tinha na altura em que foi destruída, por exemplo, devido ao usuário ter rodado o dispositivo.
Entenda-se que estado aqui apenas se refere ao conteúdo das views do layout da Activity, ou seja, o sistema apenas guarda automaticamente o estado do layout.
Qualquer outro tipo de informação que a Activity tenha será perdida, a não ser que seja explicitamente guardada nesse Bundle.
Para esse efeito a Activity disponibiliza o método onSaveInstanceState() que é chamado antes da Activity poder vir a ser destruída. O sistema passa a ele o objecto Bundle que mais tarde é recuperado no método onCreate() quando a Activity for recriada, permitindo que seja guardada e depois recuperada outro tipo de informação.
Exemplo da documentação onde se guarda o valor das variáveis mCurrentScore e mCurrentLevel:
static final String STATE_SCORE = "playerScore";
static final String STATE_LEVEL = "playerLevel";
...

@Override
public void onSaveInstanceState(Bundle savedInstanceState) {
    // Save the user's current game state
    savedInstanceState.putInt(STATE_SCORE, mCurrentScore);
    savedInstanceState.putInt(STATE_LEVEL, mCurrentLevel);

    // Always call the superclass so it can save the view hierarchy state
    super.onSaveInstanceState(savedInstanceState);
}

que depois são recuperadas no onCreate(), se a Activity tiver sido recriada:
...
@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState); // Always call the superclass first

    // Check whether we're recreating a previously destroyed instance
    if (savedInstanceState != null) {
        // Restore value of members from saved state
        mCurrentScore = savedInstanceState.getInt(STATE_SCORE);
        mCurrentLevel = savedInstanceState.getInt(STATE_LEVEL);
    } else {
        // Probably initialize members with default values for a new instance
    }
    ...
}

No exemplo de código que colocou na pergunta o savedInstanceState apenas está a ser utilizado para verificar se o onCreate() está a ser chamado devido a uma recriação da Activity ou não.
Se savedInstanceState == null é porque a Activity está a ser criada pela primeira vez, nesse caso é criado o PlaceholderFragment e colocado no R.id.container.

O que é savedInstanceState?


Div mostrando conteudo ao clicar em botão

Question: to com essa duvida ja pesquisei diversos códigos aqui porem nenhum funcionou, tenho um botão no centro da tela e ao clicar eu quero que mostra um teste especifico que está em uma div, porem o que eu escrevo na div fica na pagina inicial em baixo e o botão não faz nada, então o que eu queria, clicar no botão iniciar teste, e ele chamar essa div, ai dentro dessa div ter outro botão para seguir para o proximo teste e assim por diante .
segue o código
<!DOCTYPE HTML>

    <meta charset="utf-8">
    <title>TESTE DO FREIO</title>
    <link rel="stylesheet" href="css/estilos.css">

0

INICIAR TESTE

1. Pressione e solte o pedal de freio do estacionamento varias vezes para checar se a pressão diminui

© 2017 PokaYoke Team | Elaborado por Felipe Deolindo

js
$(document).ready(function () {

    var sensorValor = '';
    var testEnabled = false;
    var roundedValue = 0;

    $.ajaxSetup({ cache: false });

    setInterval(function () {
        $.get("IOCounter.htm", function (result) {
            $('#counter').text(result.trim());
            sensorValor = $('#counter').text();
        });v
    }, 100);

});

$(function(){
        $(".meubotao").click(function(e){
            e.preventDefault();
            el = $(this).data('element');
            $(el).toggle();
        });
    });

css
body{
    background-color: #373435;
}

#teste{
    width: 1920px;
    height: 1080px;
    background-image: url('../images/background.jpg');
    background-repeat: no-repeat;
}

.meubotao {
    -moz-box-shadow: 0px 1px 0px 0px #000000;
    -webkit-box-shadow: 0px 1px 0px 0px #000000;
    box-shadow: 0px 1px 0px 0px #000000;
    background:-webkit-gradient(linear, left top, left bottom, color-stop(0.05, #ffec64), color-stop(1, #ffab23));
    background:-moz-linear-gradient(top, #ffec64 5%, #ffab23 100%);
    background:-webkit-linear-gradient(top, #ffec64 5%, #ffab23 100%);
    background:-o-linear-gradient(top, #ffec64 5%, #ffab23 100%);
    background:-ms-linear-gradient(top, #ffec64 5%, #ffab23 100%);
    background:linear-gradient(to bottom, #ffec64 5%, #ffab23 100%);
    filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffec64', endColorstr='#ffab23',GradientType=0);
    background-color:#ffec64;
    -moz-border-radius:15px;
    -webkit-border-radius:15px;
    border-radius:15px;
    border:3px solid #000000;
    display:inline-block;
    cursor:pointer;
    color:#333333;
    font-family:Georgia;
    font-size:28px;
    font-weight:bold;
    padding:20px 50px;
    text-decoration:none;
    text-shadow:0px 1px 0px #ffffff;
    position:absolute;
        top: 500px;
        left: 700px;
}
.meubotao:hover {
    background:-webkit-gradient(linear, left top, left bottom, color-stop(0.05, #ffab23), color-stop(1, #ffec64));
    background:-moz-linear-gradient(top, #ffab23 5%, #ffec64 100%);
    background:-webkit-linear-gradient(top, #ffab23 5%, #ffec64 100%);
    background:-o-linear-gradient(top, #ffab23 5%, #ffec64 100%);
    background:-ms-linear-gradient(top, #ffab23 5%, #ffec64 100%);
    background:linear-gradient(to bottom, #ffab23 5%, #ffec64 100%);
    filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffab23', endColorstr='#ffec64',GradientType=0);
    background-color:#ffab23;
}
.meubotao:active {
        position:absolute;
        top: 500px;
        left: 700px;
}

.footer {
    position:absolute;
        top: 1080px;
        left: 800px;
}

.teste01 {
  visibility: hidden;
}

Comment: obs: tem um `v` no meio do seu código que com certeza tá causando um erro no load do seu JS, ali no final do setInterval, talvez só isso ja resolva tudo, veja se você errou ao digitar aqui ou esse erro existe na sua aplicação
Comment: Outra coisa, você tá carregando um arquivo `.htm` no get o que não faz muito sentido pra mim uma vez que um arquivo `.htm` não é dinâmico, então qual seria o motivo de você não incluir as instruções do teste direto na sua página ou então em um [tag:JSON]?
Comment: então paulo o arquivo não dinâmico , aquele counter se refere ao contador do teste, só funciona assim não sei porque mais envolve o CLP , ele manda as informações pra esse counter e depois manda pra mim, tirei o v porem deu em nada
Answer: *Resposta inicial porém não completa.
Pelo que entendi, você gostaria de um teste por etapas onde o botão passa para a próxima etapa, por isso criei um exemplo para que você possa utilizar, baseando-se e utilizando o seu código inicial. Sabemos que não irá retratar 100% do seu problema pois eu não tenho o acesso ao arquivo IOCounter.htm mas você pode adaptar o código para que o contador também funcione corretamente e então eu editarei posteriormente esta resposta para poder abranger de forma mais concreta a sua pergunta.
Disponibilizei aqui no jsFiddle o exemplo para que você possa utilizar.
Comment: Muito bom, muito obrigado mesmo, estava ficando doido já aqui kkkkkk
Comment: o iocounter.htm só tem a seguinte linha :="webdata".counter: ele pega as informações do webserver do PLC e joga na html pra mim
Comment: mas o counter é utilizado só para contar os segundos percorridos durante o teste? se for da pra fazer em javascript também
Comment: mais ou menos, ele pega a pressão em bar do caminhão, eu segui um tutorial do site da siemens pra fazer, nem tentei outra forma, você tem alguma em mente?
Comment: Paulo, nesse mesmo código que você me mandou, se quando eu clicar no botão, ele chamar outro, no caso um tipo proxima teste, porem outro botão, com outro estilo, onde eu modifico no seu codigo? eu troquei o segundo if pela outra id do botão que criei mas não rodou
Comment: é que eu utilizei o mesmo botão, só alterei o texto dele, mas se tu quer além de alterar o texto alterar o estilo vc pode utilizar o `.addClass('sua_classe_css')` no botão lá junto com onde ele muda o texto, e ai nessa `.sua_classe_css` no css vc altera o que for preciso no estilo
Comment: Cara desculpa eu sou meio leigo, to me virando mais ou menos com o que posso aqui eu adiciono o .addClass no lugar da ultima opção .html ?
Comment: e no css não tem que criar uma div pra essa class? ou posso só fazer exmplo .proximo lá que seria o outro botão e editar conforme quero?
Comment: veja eu adicionei uma cor vermelha pros "proxima etapa" https://jsfiddle.net/1xupaac4/1/
Comment: muito obrigado mesmo, salvou esse projeto, valeu mesmo cara
Comment: Vamos [continuar esta discussão no chat](http://chat.stackexchange.com/rooms/62404/discussion-between-felipe-deolindo-and-paulo-roberto).
Comment: Paulo vou precisar da sua ajuda novamente, no que você fez para mim, como posso destinguir os testes, exemplo o teste um vai aparecer quando iniciar o teste, porem ao clicar em proximo teste, mostra o 1 junto ao "teste 2", e assim por diante, como eu mostro e escondo isso?
e outra duvida rs, fiz um retangulo para mostrar a pressão dentro dele, mas ele aparece na pagina inicia, sabe me dizer como esconder ele de uma div especifica?


Apply function element wise to list of matrices

Question: I have a list of matrices/dataframes , for example:
mat = list(matrix(1:6,2,3),matrix(5:10,2,3),matrix(4:9,2,3))
mat
[[1]]
     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6

[[2]]
     [,1] [,2] [,3]
[1,]    5    7    9
[2,]    6    8   10

[[3]]
     [,1] [,2] [,3]
[1,]    4    6    8
[2,]    5    7    9

I want to get max elementwise from each matrix.
a= matrix(0,2,3)
for(i in 1:2){
  for(j in 1:3){
    a[i,j] = max(sapply(mat,function(x){x[i,j]}))
  }
}
a
     [,1] [,2] [,3]
[1,]    5    7    9
[2,]    6    8   10

This gives me the desired output but is there any way it can be done without for loop?
Answer: An option with Reduce and pmax:
Reduce(pmax, mat)
#     [,1] [,2] [,3]
#[1,]    5    7    9
#[2,]    6    8   10

Or, with purrr::reduce:
purrr::reduce(mat, pmax)
#     [,1] [,2] [,3]
#[1,]    5    7    9
#[2,]    6    8   10

Answer: apply(simplify2array(mat), c(1,2), max)
#     [,1] [,2] [,3]
#[1,]    5    7    9
#[2,]    6    8   10

Answer: We could use pmax
do.call(pmax, mat)

-output
     [,1] [,2] [,3]
[1,]    5    7    9
[2,]    6    8   10

Answer: a <- array(unlist(mat), dim=c(2,3,3))
apply(a, c(1,2), max)


Multilevel model with partially pooled variance

Question: I have a model with N data points and J groups, where I want to partially pool the means and the variances of the groups.
Within group j, for data $\{y_i\}$ I'm assuming
$y[i] \sim \mathcal{N}(\theta[j], \sigma_y[j])$
For $\theta$, I have
$\theta[j] \sim \mathcal{N}(\mu_\theta, \sigma_\theta)$ for j = 1, ..., J
I'm not sure what prior to use for the $\sigma_y$ parameter.  I've tried inverse gamma; i.e,
$\sigma_y^2[j]  \sim IG(a, b)$ for j = 1, ..., J
but my chains aren't converging for a toy data set I've created.  Even when I pass the "true" parameter values as the initial values for the chain, it still doesn't work.
How do people typically handle models like this?  This isn't working in stan or in pymc, so I'm thinking there's something wrong with my probability model.
Answer: The general idea of your model can be handled in Stan (and possibly in PyMC and other statistical inference packages). Here are a couple of suggestions:
Hyperparameters
We (i.e. Andrew) have been suggesting a half Cauchy prior.
Reparameterization
The non-centered parameterization should help this model. The posterior geometry of $theta$ and $\sigma_y$ will look like a funnel. This is hard to sample from. Reparameterizing the model should help. See Section 20.1 in the Stan 2.8.0 manual for details.
Note: this reparameterization is dependent on the shape of the posterior distribution. The posterior distribution changes with different data, so thinking that one way to write the model will be the best way for all data isn't true.