Thousands of visits a day from untraceable traffic to website - Serious issue

Question: At the end of January we noticed a spike in traffic to what JetPack stats says was home/archive page and what Google was classifying as going to /gaming/ which is an archive list in WordPress.
This started off as ~3,000 unique visitors and jumped up to 65,000 unique visitors in one day, again all to the "home" page. This happened over a course of a couple of weeks and we thought we were getting attacked.
The traffic then dropped off for a few days but then came back but came back as only about ~15,000 uniques a day and has been like that every day since. We came to the conclusion that something wasn't tracking right somewhere and this is legitimate traffic and brushed it off.
Now here comes the problem, Google AdSense has just disabled our account for "invalid clicks". We are trying to figure out where this traffic is coming from and stop it if it's not legitimate or figure out a way to track it correctly.
Specs for the site:
Dedicated server running CentOS 6 with nginx, php-fpm and MySQL. The site is built in WordPress and we use CloudFlare and W3 Total Cache. Analytics being used are Google Analytics, Quantcast, Alexa and Compete.
Any kind of help would be awesome.
UPDATE:
I'm finding more people with the same type of problem and there doesn't seem to be a solution.
http://netmeg.com/bot-attack/
http://stkywll.com/2012/03/02/annoying-cyborgs-attach-distort-analytics/
After looking at the access logs I noticed they were all CloudFlare IP's. I looked into that and found out CloudFlare acts as a proxy and there was a way to fix the logs in nginx. They are coming from many different ISP's in the US. They are going to /games/ or /gaming/ (/games/ redirects to /gaming/) and all seem to have the same user agent of Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0).
Comment: Do you have access logs?  The first step to finding where the requests come from is to look at the logs.
Comment: I suspect you'll get better answers on webmasters.SE.
Comment: Did you have a look at webserver logs?
Comment: Ok, after looking at the access logs I noticed they were all CloudFlare IP's. I looked into that and found out CloudFlare acts as a proxy and there was a way to fix the logs in nginx. So I will be monitoring the logs to see what the actual IP address is now.
Comment: I'm looking at the logs and they are coming from many different ISP's in the US. They are going to /games/ or /gaming/ (/games/ redirects to /gaming/) and all seem to have the same user agent of `Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)`. What should I be looking for?
Comment: Do you get any refer(r)er URLs?
Comment: @Skaperen no referrers. I've update the post with links of people having the same issue.
Comment: My next guess is this is playing with Google to make clients look (more) like human run when they are still bots doing bad ad clicks elsewhere.
Answer: Sounds like your site may be compromised, esp. if your AdSense has been suspended. Register for Google's Webmaster Tools and see if it reports any malware. Run the site through Sucuri's Sitecheck, too. Install the WordPress Exploit Scanner plugin for a quick check, then consider diffing your site against a clean copy of v3.4.1 and doing the same for your plugins and themes (including the twentyten & twentyeleven directories, which are a common hideout for malware files). I see a lot of sites getting owned via insecure plugins with third-party libraries like timthumb, so it's definitely a vector to be aware of.
Put some sort of protection in place for /wp-login.php, all the WordPress sites I admin get regular automated brute-force attempts against them, so if any of your users' passwords are weak you can be vulnerable there. I tend to protect this location in my nginx configs by limiting IP ranges for logins (if the clients permit), rate-limiting with limit_req_zone and/or installing plugins such as login-lockdown, Duo's Two-factor Auth or Google Authenticator for WordPress. I've seen hacks where the spammer will get an admin login to a blog and then alter the content, rather than trying to own the box, often with the objective of utilising a site's Pagerank to get a linkback boost.
If you've ruled out any sort of hack of your site and you're still getting malicious visitors deliberately trying to subvert the clicks on your ads, something that screens and bans them before they have a chance to will ameliorate matters. The Bad Behaviour plugin will do this and may be suitable (though it's rather agressive, so keep an eye on its ban list and whitelist generously where applicable) or, if you want to do it at a lower level, take a look at naxsi for nginx or fail2ban.
Comment: Bad Behavior is my code. As far as I know it's very conservative in what it blocks. Though you can make it more aggressive by enabling strict mode.
Comment: @Stef Pause, no malware reported from GWT, Sucuri shows good. I run WordPress exploit scanner every week. I'm also connected with VaultPress. I've update the post with links of people having the same issue though.
Comment: @MichaelHampton, I have Bad Behavior installed but not active. Reason was a while ago there was conflict with Google +1 buttons. It looks like that's been resolved so I just reactivated it.
Answer: Very simple. Simply open up index.php in the root directory and add this to the beginning:
$useragent = $_SERVER['HTTP_USER_AGENT'];

if ($useragent == "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)")
    {
    header("Location: http://www.google.com/");
    exit();
    }

Since all these are coming from different IP addresses, it isn't practical, realistic, or time-efficient to block the IP addresses. But since they all have the same user agent, you can just block that instead. This simply redirects anyone with that user agent to Google. The only possible downside I can think of is that if any legitimate visitor has this same user agent, they will also be redirected to Google. To solve this, instead of redirecting to Google, you can make them enter in a CAPTCHA or something.
Thanks @Oerd for adding the essential exit/die statement.
Comment: An `exit()` or `die()` should go after your `header(...)` line. You *need* to make sure that code below won't execute if request has been redirected. You cannot be sure that these "attacking" bots will behave according to HTTP standards.
Answer: Yup I've had this problem for quite some time with a different user agent.
We just learned to live with the traffic ... if you want to keep the bad page views out of your google analytics or otherwise suspend certain services like adsense for them (we certainly did) you can find out more about doing that here:
http://stkywll.com/2012/04/27/annoying-robots-a-solution-for-google-analytics/
There is basically no good server side solution to this -- the solution above is based on click and mouse-move detection in the browser, and so far that's the only good way to tell good traffic of this sort from bad.
Good luck y'all!


how to show table data as columns in tsql

Question: I have 2 tables in sql-server 2005 express DB
1. ItemSize Table
CREATE TABLE [dbo].[ItemSize](
[ID] [int] IDENTITY(1,1) NOT NULL,
[iSize] [int] NULL
) ON [PRIMARY]

2. SalesLog Table
CREATE TABLE [dbo].[SalesLog](
[SalesID] [int] IDENTITY(1,1) NOT NULL,
[MemoNo] [int] NULL,
[ProductCode] [int] NULL,
[Quantity] [int] NULL,
[Price] [int] NULL,
[pGroup] [int] NULL,
[pName] [nvarchar](30) NULL,
[pSize] [int] NULL,
[BillDate] [nchar](10) NULL,
CONSTRAINT [PK_SalesLog] PRIMARY KEY CLUSTERED
(
[SalesID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

ItemSize table contains different size of drink like
100ml
200ml
300ml
400ml
500ml
I want  ItemSize table columns (iSize) data as column like
ItemName   100ml    200ml    300ml   400ml   500ml
 Drink A    10       20        5       4      30
 Drink B    20       10        15      35     40

and data will be arranged from Sales table according to size and Product Name as shown above.
I have similar query to do so but its not dynamic. i mean if i add new size to ItemSize table then it will not be avilable to report. So i need solution for this.
Currently I am using this query to fetch sales according to item size
select i.gName,
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=0 and
pGroup=i.gCode),0) as '0ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=180 and
pGroup=i.gCode),0) as '180ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=375 and
pGroup=i.gCode),0) as '375ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=500 and
pGroup=i.gCode),0) as '500ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=650 and
pGroup=i.gCode),0) as '650ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=750 and
pGroup=i.gCode),0) as '750ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and pSize=1000 and
pGroup=i.gCode),0) as '1000ml',
isnull((select
sum(Quantity)from saleslog where BillDate='01-06-2010' and
pSize=2000 and pGroup=i.gCode),0) as '2000ml'
from saleslog as s
inner join ItemGroup as i on s.pGroup=i.gCode
where BillDate='01-06-2010'
group by i.gCode, i.gName

Answer: Have you checked out the PIVOT feature?  You don't mention which SQL version you're using, but it's available in 2008, and does pretty much what you're after.
Also, your code seems awfully inefficient.  You're doing selects inside selects fetching back data you already have access to.  Maybe I'm having a hard time identifying with your data set, but it seems like a need for a case statement in your current situation, rather than lots of selects.  I believe the following works:
select
  i.gName,
  case i.pSize when 0 then sum(Quantity) else 0 end as [0ml],
  case i.pSize when 180 then sum(Quantity) else 0 end as [180ml],
  case i.pSize when 375 then sum(Quantity) else 0 end as [375ml],
  { rest of case samples }
from
  saleslog as s
    inner join ItemGroup as i on s.pGroup = i.gGroup
where
  s.BillDate = '01-06-2010'
group by
  i.gName

I'm also a little confused on your question.  You mention you have two tables, one has id, and iSize, but I don't see iSize, so I'm assuming pSize(?).  Also, your sales table doesn't match up with your query, ie there is no quantity amount in your sales table, your create mentions [sales], but your actual query uses [salesdata].
Maybe some more clarifying?
Comment: hey Jon Angliss i have edited my question as there were mistake. Actually i want to show ItemSize table column data(i.e. Size of drinks) as Column Header in my report and Sales quantity according to Drink Group(gName) in rows.
Comment: Your table structures still don't match your question, so I'm not sure where you're getting iSize from when your question doesn't use it, and you join on iGroup, but there is no iGroup field in the ItemGroup table.  But, if the case statement above doesn't work for you, look at the PIVOT option in TSQL, it does exactly what you're after.
Comment: iSize Column is defined in ItemTable table, see its structure in above question
Comment: And iGroup is where?  The answer is PIVOT if you can use it, or the CASE statement if you cannot.

SELECT * FROM
  (
    SELECT
      i.gName,
      case i.pSize
        when 0 then '0ml'
        when 180 then '180ml'
        when 375 then '375ml'
      end as [Size],
      Quantity
    FROM
      saleslog as s
        inner join ItemGroup as i on s.pGroup = i.gGroup
    WHERE
      s.BillDate = '01-06-2010'
  ) SalesInfo
PIVOT
  (
    SUM(Quantity)
    FOR [Size] IN (
      [0ml],[180ml],[375ml]
    )
) AS PivotTable ORDER BY [gName]

I'm sure you can figure it out matching your fields.


"yum groupinstall" i686 packages on x64?

Question: I'm working on enabling x64 host build workers to build i686 RPMs, now that AWS EC2 (which we use to run build workers) is deprecating i386 nodes like m1.medium in favour of x64-only instance types like m3.medium.
I could roll a custom AMI that runs a 32-bit userpsace on a 64-bit kernel, but this seems like a good opportunity to switch to using multiarch for builds.
I currently install the core dev tools and package build dependencies with
yum install yum-utils
yum groupinstall "Development Tools"
yum-builddep postgresql

... but I can't figure out how to do the multiarch equivalent. For individual packages you just:
yum install glibc-devel.i686

for example, but this won't work for groupinstall or builddep.
Ideas?
Comment: Why aren't you using koji, or at least mock? It sounds like you tried to reinvent the wheel.
Comment: @MichaelHampton I I could +100 a comment, I would. Thankyou very much for the pointer at mock; incredible life-saver, and needs a "see also" in the rpmbuild man page. I may well land up using koji for all the RPM distros targeted too, and just deal with using sbuild or something separate for deb distros. (I wish I could just lock 'em all in one room and let the last package format standing leave).
Comment: BTW, `multilib_policy=all` [per this answer](http://serverfault.com/a/512791/102814) works for "groupinstall" but not `yum-builddep`.
Comment: @MichaelHampton We're building packages for many different distros, and running a lot of tests as well as doing packaging. Distro-specific tools are great when you're building lots of packages for that distro / package format. Much less so when you're doing cross-distro releases of a few packages. As for mock: I was completely unaware of it. It looks very useful.
Comment: They will handle at least RHEL/CentOS and Fedora. If you're trying to build for SuSE, no guarantees.
Comment: @MichaelHampton I'll check it out. SuSE isn't a target of interest. It won't help us with the package building for Debian/Ubuntu, OS X, Windows, etc (which is why I've been using Jenkins to control jobs) and doesn't seem to support EC2 dynamic build worker provisioning etc - but mock looks useful, at least.
Answer: As you mentioned in your comment, you can use multilib_policy=all for groupinstall, and use setarch command for yum-builddep like this:
setarch i686 yum-builddep postgresql

Comment: Ah, that's the trick. Thankyou.


How can I call a Rust function from assembly?

Question: I am looking for like C calling via function call [function]. I am linking asm and Rust in one binary via ld.
Answer: If you know how to call a C function from assembly (I don't, offhand), then this will help you get a Rust function that can be called like a C function:
<code>// foo.rs
#[no_mangle]
pub extern fn increment(a: i32) -> i32 {
    a + 1
}
</code>
Compile it as a dynamic or static lib:
<code>$ rustc --crate-type dylib foo.rs
$ nm libfoo.dylib  | grep increment
0000000000000c70 t __ZN9increment10__rust_abiE
0000000000000c30 T _increment
</code>
Now you have a library that can be called like a C library. You should read the entire FFI Guide to better understand the tradeoffs and pitfalls of calling Rust from outside of Rust.


Show that the image of a cube is almost a cube

Question:
Let $C_r = \left \{ x \in \mathbb{R}^n : |x^i| < r \forall 1 \leq i \leq n \right \}$
and $ g \in C^1(U, \mathbb{R}^n)$ for some open $\left \{ 0 \right \} \subset U$ s.t $dg(\vec{0}) = I, g(\vec{0})=\vec{0}$. and let us choose some $0 < \varepsilon < 1$.
Show that there exists $\delta > 0$ s.t $\forall r < \delta, C_{(1-\varepsilon)r} \subset g(C_{r}) \subset C_{(1+\varepsilon)r}$

I couldn't think of a better title, edits are welcome.
This is what I tried:
$g(0+\triangle x) - g(0) = dg(0)(\triangle x) +o(\triangle x) \implies g(x) = x+o(x) \implies \frac{||g(x) - x||}{||x||} \xrightarrow[x \to 0]{} 0$
So we can choose some $\delta > 0$ s.t if $||x|| < \delta$ then $||g(x)-x|| < ||x||_{\infty} \varepsilon$
Now $x \in C_r \implies |x^i|<r \implies |g(x)^i| < r+ ||x||_{\infty}\varepsilon \leq r+r \varepsilon \implies g(C_{r}) \subset C_{(1+\varepsilon)r}$
But i'm not sure how to show $C_{(1-\varepsilon)r} \subset g(C_{r})$.
Hints appreciated.
Also, does what I did so far seem correct?
Comment: I now notice I made a mistake as i need $|g(x)^i| < (1+\varepsilon)r$ and I showed something else.
Comment: Remark: Either work with a more suitable norm or use the usual norm and balls (with cubes inscribed and superscribed).
Comment: @TedShifrin
Thanks. I fixed the mistake, still looking for help on the other part of the question.
Comment: How did you prove the open mapping part of the Inverse Function Theorem?
Comment: I tried looking at the proof now but I'm not sure how this helps...
Answer: Step 1: Definition of multivariate derivative
By definition, $dg(x)$ is the unique matrix which satisfies:
$$
\lim_{h\rightarrow 0}\frac{||g(x+h)-(g(x)+dg(x) h)||_2}{||h||_2}=0
$$
In our case, this simplifies to:
$$
\lim_{h\rightarrow 0}\frac{||g(h)-h||_2}{||h||_2}=0
$$
Step 2: Working with cubes
But since we are working with cubes, not Euclidean balls, we would perfer the "cube-norm" (aka the sup-norm): $||x||_\infty:=\max_{i\in[n]}|x_i|$.
It still works, using the following inequalities (easy to show):
$$
||x||_\infty\leq ||x||_2\leq \sqrt{n}||x||_\infty
$$
We can write:
$$
0=\lim_{h\rightarrow 0}\frac{||g(h)-h||_2}{||h||_2}\geq \lim_{h\rightarrow 0}\frac{||g(h)-h||_\infty}{\sqrt{n}||h||_\infty}\geq 0
$$
So, it follows that the limit is zero, also for the $||\cdot||_\infty$-norm.
Finally, this means that for any $\varepsilon>0$, there exists a $\delta>0$, such that for all $||h||_\infty<\delta$ (in other words, for all $h\in C_\delta$),
$$
\frac{||g(h)-h||_\infty}{||h||_\infty}<\epsilon
$$
Step 3: Triangular inequality
Using the triangular inequality, $||h||_\infty\leq ||g(h)-h||_\infty+||g(h)||_\infty$.
Therefore:
$$
\frac{||h||_\infty}{||h||_\infty}-\frac{||g(h)||_\infty}{||h||_\infty}<\epsilon
$$
Rearranging:
$$
||h||_\infty(1-\epsilon)< ||g(h)||_\infty
$$
Using instead the triangular inequality $||g(h)||_\infty\leq ||g(h)-h||_\infty+||h||_\infty$, we get:
$$
\frac{||g(h)||_\infty}{||h||_\infty}-\frac{||h||_\infty}{||h||_\infty}<\epsilon
$$
Which means:
$$
||g(h)||_\infty\leq ||h||_\infty(1+\epsilon)
$$
To summarize:
$$
||h||_\infty(1-\epsilon)\leq ||g(h)||_\infty\leq ||h||_\infty(1+\epsilon)
$$
This already implies that $g(C_\delta)\subset C_{\delta(1+\epsilon)}$.
However, it is not enough to conclude that $g(C_\delta)$ contains all of $g(C_{\delta(1-\epsilon)})$.
Step 3: Inverse function theorem
By the inverse function theorem, there is some open set $V$ containing $0$, such that $g^{-1}$ exists on $V$ and is continuous.
Henceforth we will assume $C_{\delta(1+\epsilon)}\subset V$ (if not, we make $\delta>0$ smaller until this is true).
Moreover, the inverse function theorem tells us that $dg^{-1}(0)=I^{-1}=I$.
Repeat steps 1-2, to obtain:
$$
||g^{-1}(y)-y||_\infty\leq \epsilon||y||_\infty
$$
This holds for all $||y||_\infty<\gamma$ for some small $\gamma$, we may take $\gamma=\delta$ without loss of generality.
Step 4: Completing the proof
Take any $x\in C_{\delta(1-\epsilon)}$. We show that $x\in g(C_{\delta})$.
Since $g$ is invertible on $C_{\delta}$, this is the same as requiring that $y\in  C_{\delta}$ whenever $g^{-1}(y)\in C_{\delta(1-\epsilon)}$.
Finally:
$$
||y||_\infty \leq ||g^{-1}(y)||_\infty+||g^{-1}(y)-y||_\infty\leq \delta(1-\epsilon)+\epsilon||y||_\infty
$$
Rearranging, we have $||y||_\infty<\delta$. The proof is complete.


How to find indication of a Validation error (required="true") while doing ajax command

Question: I have a form inside a dialog which I close by clicking on commandbutton with ajax ,
like this
<h:commandButton value="Add" action="#{myBean.addSomething(false)}"
    id="add_something_id" >
    <f:ajax render="@form someTable" execute="@form"
        onevent="closeAddNewSomethingDialogIfSucceeded"></f:ajax>
</h:commandButton>

and here is the js code for closing the dialog
    function closeAddNewSomethingDialogIfSucceeded(data) {
        if(data.status === 'success') {
            $("#dialog_id").dialog("close");
        }
    }

No problems till here...
Now I changed some of the dialog form fields into required="true" and now I want to prevent the closing of the dialog of i got validation errors...
But the ajax data.status still reaches its success state , and I can't figure out what indication of validation failure I can hook on...
any ideas?
Thanks to BalusC answer I did the following:
in JSF , added :
    <h:panelGroup id="global_flag_validation_failed_render">
        <h:outputText id="global_flag_validation_failed" value="true"
            rendered="#{facesContext.validationFailed}"/>
    </h:panelGroup>

the f:ajax was changed into
<f:ajax render="@form someTable global_flag_validation_failed_render"

and in js added the following check
if(data.status === 'success') {
    if($("#global_flag_validation_failed").length === 0){
         $("#dialog_id").dialog("close");
    }
}

Answer: Not specifically for required="true", but you can check by #{facesContext.validationFailed} if validation has failed in general. If you combine this with  checking if the button in question is pressed by #{not empty param[buttonClientId]}, then you can put it together in the rendered attribute of the <h:outputScript> as follows:
<h:commandButton id="add_something_id" binding="#{add}" value="Add" action="#{myBean.addSomething(false)}">
    <f:ajax execute="@form" render="@form someTable" />
</h:commandButton>
<h:outputScript rendered="#{not empty param[add.clientId] and not facesContext.validationFailed}">
    $("#dialog_id").dialog("close");
</h:outputScript>

(note that you need to make sure that the script is also re-rendered by f:ajax)
A bit hacky, but it's not possible to handle it in the onevent function as the standard JSF implementation doesn't provide any information about the validation status in the ajax response.
If you happen to use RichFaces, then you could just use EL in the oncomplete attribute of the <a4j:xxx> command button/link. They are namely evaluated on a per-request basis instead of on a per-view basis as in standard JSF and PrimeFaces:
<a4j:commandButton ... oncomplete="if (#{!facesContext.validationFailed}) $('#dialog_id').dialog('close')" />

Or if you happen to use PrimeFaces, then you could take advantage of the fact that PrimeFaces extends the ajax response with an additional args.validationFailed attribute which is injected straight in the JavaScript scope of the oncomplete attribute:

(note that & is been used instead of &, because & is a special character in XML/XHTML)
Or you could use the PrimeFaces' RequestContext API in the bean's action method to programmatically execute JavaScript in the rendered view.
RequestContext.getCurrentInstance().execute("$('#dialog_id').dialog('close')");

No conditional checks are necessary as the bean's action method won't be invoked anyway when the validation has failed.
Comment: Thank! eventually I used relied on the #{facesContext.validationFailed} (I wasn't aware of it) I used it in a render condition of simple outputext... and later on In the js code i checked for that outputtext existence using the jquery .length ...
Comment: @BalusC Does facesContext.validationFailed applies for conversion failures too ? Is it true if conversion is failed ?
Comment: You're welcome. For an overview of implicit EL objects in JSF, see also [this](http://balusc.blogspot.com/2011/09/communication-in-jsf-20.html#ImplicitELObjects).
Comment: @Bren Yes. According to the Java EE 6 FacesContext documentation, "validationFailed() Sets a flag which indicates that a conversion or validation error occurred while processing the inputs." (http://docs.oracle.com/javaee/6/api/javax/faces/context/FacesContext.html#validationFailed())
Comment: Note that you don't have to use &&... You can instead use "and"...
Comment: @Andrew You're confusing JS with EL.
Comment: @BalusC You are correct. I must have used EL in a different attribute.
Answer: I think you should take a look at PrimeFaces' RequestContext. This would help you trigger client-side code on the server side.
Answer: Two things
1) Checking for an error in the 'onevent' function
Surely you have a message tag for the mandatory field?
<h:message id="m-my-field-id" for="my-field-id" errorClass="error-class"/>

So you can check for the error-class something like
var message = $('#m-my-field-id');

if(message.hasClass('error-class'){
    //do this
}
else{
  //do that
}

2) The DOM isn't up to date on success
Yes, I can see the message on the page in Firefox, yet jQuery tells me it is not there.
I have found that using the smallest possible timeout is sufficient to fix this
setTimeout(
    function(){
    setErrorStyle(source.attr('id'));
    },
   1
);

Answer: @BalusC
in your example code the clientId of the button is not set as a param because it is a AJAX request. So
not empty param[add.clientId] is always false.

But this works:
param['javax.faces.source'] eq add.clientId

(tested with jsf-impl-2.2.12.redhat-1)
regards


Firefox sec_error_unknown_issuer after renewing root CA certificate

Question: For our corporate network we use an internal CA, which directly signs some intranet server certificates. Now the root certificate is about to expire. I tried to renew it with the following OpenSSL command:
openssl req -new -x509 -days 123 -key root.key -out root.crt
I deleted the old certificate from my PC and installed the new one. Windows/IE, Chrome, and Opera seem to be fine with it, but Firefox won't accept it. Trying to access an intranet server gives me the sec_error_unknown_issuer error, stating that no chain is provided. I googled my fingers to the bone, but the only answer I could find was that that the intermediate certificate is missing from the server config. But in my case there is no intermediate CA!! Why can't Firefox link the server certificate to the new root? All the other browsers seem to be able to and even openssl verify says everything is OK. Any ideas?
Answer: I figured it out! The problem was my openssl.cnf. The old certificate was created with string_mask = utf8only. This line was missing from my current file, resulting in a default value of PrintableString, T61String, BMPString. I did not use any non-ASCII characters, but it seems it was still enough to irritate Firefox.
Comment: Well done! That sounds like a difficult issue to track down. You should probably accept this (your own) answer to let others know the question has a definitive solution -- and to help anyone else with this issue to find your answer.
Answer:
I deleted the old certificate from my PC and installed the new one.

Firefox uses its own separate certificate store. At the risk of stating the obvious (for those of us who regularly use Firefox), did you add the new CA certificate to Firefox itself?
Comment: @user2323470 What happens when you open the CA certificate in Firefox?
Comment: Yes, I did install/replace it twice, in Firefox and in Windows via *Install Certificate* and restarted Firefox/the PC--just as I did when I installed the original certificate years ago.


Skip function if it takes too long

Question: In Java I have a function that processes a text file in a certain way. However, if it takes too much time the process will most likely be useless (whatever the reason is) for that text file and I would like to skip it. Furthermore, if the process takes too long, it also uses too much memory. I've tried to solve it this way, but it doesn't work:
for (int i = 0; i<docs.size(); i++){
    try{
        docs.get(i).getAnaphora();
        }
    catch (Exception e){
        System.err.println(e);
    }
 }

where docs is just a List of files in a directory. Usually I have to manually stop the code because it is 'stuck' at a particular file (depending on the contents of that file).
Is there a way of measuring time for that function call and tell Java to skip the file the function takes more than, let's say, 10 seconds?
EDIT
After scraping a few different answers together I came up with this solution which works fine. Perhaps someone else can use the idea as well.
First create a class that implements Runable (this way you can pass in arguments to the Thread if needed):
public class CustomRunnable implements Runnable {

    Object argument;

    public CustomRunnable (Object argument){
        this.argument = argument;
}

    @Override
    public void run() {
        argument.doFunction();
    }
}

Then use this code in the main class to monitor the time of a function (argument.doFunction()) and exit if it takes to long:
Thread thread;
for (int i = 0; i<someObjectList.size(); i++){
    thread = new Thread(new CustomRunnable(someObjectList.get(i)));
    thread.start();
    long endTimeMillis = System.currentTimeMillis() + 20000;
    while (thread.isAlive()) {
        if (System.currentTimeMillis() > endTimeMillis) {
        thread.stop();
        break;
    }
    try {
        System.out.println("\ttimer:"+(int)(endTimeMillis - System.currentTimeMillis())/1000+"s");
        thread.sleep(2000);
        }
    catch (InterruptedException t) {}
    }
}

I realise stop() is depcecated, but I haven't found any other way to stop and exit the thread when I want it to stop.
Comment: Take a look at this:

[http://stackoverflow.com/questions/8982420/time-out-method-in-java][1]

  [1]: http://stackoverflow.com/questions/8982420/time-out-method-in-java
Comment: This helped me achieve my solution, thanks :)
Answer: Wrap your code in a Runnable or Callable and submit it to a suitable Executor to execute it. One of the submit method takes a timeout period after which has passed the code is interrupted.
Comment: Are you able to give an example?
Comment: I've searched briefly for the interfaces and this seems like the right answer to the problem. I don't know how to do it yet however. I'll look in to it.
Answer: You can use System.nanoTime() inside run method as a condition.
For example,
long cur = System.nanoTime(); //records the time when start executing
...
double elapsedTime(System.nanoTime() - cur) / 1000000.0; // at the end

Answer: Detailed solution with Runnable (see previous thorbjorn's answer):
           ExecutorService executorService = Executors.newSingleThreadExecutor();

           executorService.execute(() ->doFunction()); //here is the Runnable (represented as simple lambda here)

           executorService.shutdown();

           try {
                    if (!executorService.awaitTermination(2000, TimeUnit.MILLISECONDS)) {
                        System.out.println("ERROR: Stopping going to next step");
                     }
            }catch(InterruptedException e){
                   e.printStackTrace();
            }

Skip function if it takes too long


Why can we write a uncontinual function continual?

Question: Let's consider $$f(x) = \frac{(x-1)(x-2)(x-3)}{x^2-3x+2}$$ with definition $D_{f} = \mathbb{R} \setminus \lbrace 1, 2 \rbrace$. This means we are allowed to set $x$ to every value of $\mathbb{R}$ except one and two.
However we can transform the function so that those get valid:
$$u_{0} = x^2-3x+2 = \frac{3}{2} \pm \sqrt{\frac{3}{2}-2} = \frac{3}{2} \pm \frac{1}{2}$$ we can factorize $f(x)$ as $$f(x) = \frac{(x-1)(x-2)(x-3)}{(x-1)(x-2)}$$ which yields in $$f(x) = x-3$$
This function is now defined for $f(1) = 1-3 = -2$ and $f(x) = 2-3 = -1$ how can this be?
Comment: The function still isn't defined for $x\in\{1,2\}$, but you have shown that $x-3$ is an analytic continuation to $\mathbb R$.
Answer: You cancel terms, which is fine as long as the cancelled terms are nonzero. That is, we have
$$ \frac{(x-1)(x-2)(x-3)}{x^2-3x+2}=x-3$$
provided $x\ne 1$ and $x\ne 2$. And inded for $x=1$ or $x=2$ the two sides differ: The left side is not even defined.
Answer: There is the function and there is the expression that gives the function. Those expression can be transformed.
Think of the function $\frac{x}{x}$ with domain $D\backslash \{0\}$.


OpenGL, Showing Text and getting values using C

Question: This has been my problem since I started using openGL.
What code am I going to use to show text and get value. I could not use printf and scanf and my only header file is glut.h.
Answer:
This has been my problem since I started using openGL.
  What code am I going to use to show text

Difficult subject, because OpenGL itself doesn't deal with text output. You can:

render text to an image and display that
create a texture atlas from the glyphs of a font, then render from that font texture
draw the font glyph outlines as geometry

If you Google "OpenGL font rendering" you'll get a large number of results of papers on the topic. Recent and old ones alike.

and get value.

Not with OpenGL. OpenGL is a drawing API. You send it points, lines and triangles, and it draws nice pictures for you. User input is outside the scope of OpenGL. That's on part of the GUI system. Most likely one of

Windows GDI
MacOS Cocoa
X11

Standard user input event processing applies. Usually one uses a toolkit like Qt, GTK or similar. Those toolkits deal with user input processing through their event mechanism.
Answer: http://linux.die.net/man/3/glutstrokestring
How about this?
<code>#include <openglut.h>
glutStrokeString(GLUT_STROKE_ROMAN, "I will draw this string at the origin of the model");
</code>
Comment: You need openglut or freeglut. Glut has no text drawing functions.


Can the jackknife be used for asymmetric confidence intervals?

Question: All of the jackknife methods (JN) I have seen (for example) use the JN to estimate standard errors and then use those estimates in standard normal-assumption symmetric intervals constructions ($ \hat{\mu} \pm z_{1 - \alpha} \cdot \hat{se} $),
Can the jackknife be used to estimate the sample statistic distribution directly ala the bootstrap? In other words, can the values returned from jackknife replication be used as inputs to percentile or BCa-type confidence interval functions to get asymmetric interval? Can Jackknife only be used to find standard errors?
Answer: Yes, it can. Example: https://influentialpoints.com/Training/log_normal_confidence_intervals.htm
Say you have lognormal-distributed data (this is right-skewed and positive), and you're trying to estimate the mean $D$. If you use a normal assumption, you could get a nonsensical CI which includes negative values. If you use a log-normal assumption, you would still calculate the standard error using the jacknife, but, the way you apply it to generate a CI is different. In the case in the above link, the jacknife SE plugs into a formula for a value called $C$, which is basically a log-scale confidence interval width.
Comment: It can give asymmetric confidence intervals when we replace the standard normal assumption, but can it also do the second part of the question *"Can the jackknife be used to estimate the sample statistic distribution directly ala the bootstrap?"*


Google IP sends a TCP (ACK, PSH) to my router. Known why?

Question: I am monitoring my traffic and noticed several TCP (ACK, PSH)  connection attempts on my router's input chain, which gets dropped by my firewall.
The log looks like that:
Dropping input: src-mac 9c:80:df:a0:8a:dd, proto TCP (ACK,PSH), 184.108.40.206:443 (google ip) ->192.168.1.2:40382 (my router IP), len 115

Obviously this is dropped because my last rule on my input chain is to drop packets.
I do not understand the TCP protocol well enough, so sorry if this is a bit naive, but why is the request directed to my router?
I have various devices that uses google services and probably third party software as well, but it is very confusing to me, why the packet is actually sent to the router and not nat'ed to a device in my network (which would be the forward chain, right?).
I have not yet noticed a degraded experience with my devices regarding google products. Software updates, push notifications etc all seem to work correctly.
Comment: It's probably the last ACK of a closed connection.
Comment: You could try increasing various netfilter tcp parameters ( `/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_*` ) and see if those spurious logs disappear (at the cost of possible extra memory usage)
Answer: It's clearly showing a packet received from a Google IP, with a source port of 443.
There is of course the possibility that someone does a man-in-the-middle attack with Google IP address, or even that someone from (or broken into) Google is sending you packets, yes, but this is very, very unlikely.
What's more likely is that your packet filter, and/or the conntrack subsystem (or a similar functionality on your router and/or on your ISP) already has seen packets from the packet flow between google:443 and yourip:40382 showing the connection to be closed already.
Typically connections initiated by you are not logged by the packet filters, but that's only true for established connections.
When the connections are closed down with a FIN from both sides or RST from either side, the connection is no longer considered ESTABLISHED.  Any packets delayed or resent by either side will then be considered new, and log-worthy by your packet filter, especially if the removed conntrack entry prevents de-nating the packet to its proper destination.
This is very common, there is probably no reason to worry at all, you can usually safely ignore those logged packets.
If you see any pattern, and want to investigate, you can keep a tcpdump running on your system logging all packets from similar IP addresses, then when you see one more similar log, you can stop the tcpdump, and filter for the local port to see if you indeed had a connection open just a few seconds before the event.
A sample tcpdump for the above would be something like:
tcpdump -ieth0 -s0 -w /var/tmp/allssl.cap port 443

Comment: I indeed have an accept rule for established and related packets on the input chain. However I also have a drop rule for invalid packets (out of order etc), which does not log anything. That's why I am confused why those packets pass through the firewall to the very last drop (which does also the logging). I might need to tcpdump indeed some traffic to dig into that. Thanks
Answer: The phenomenon is somewhat more complex. I did some research in the according logs and want to share my guess here for all whom wondering too.
The related log entries showing drops in the input chain with TCP flags ACK and PSH (push). This indicates that some service in the network is awaiting messages from the google servers (via. push). But wait - why the package is logged in the input chain? Thats rather easy to explain - if the router  (or as @chexum already mentioned - the conntrack) has no connection information about NATed/PATed traffic the package looks like traffic for the router and not for any devices behind it. So that explains why "google is sending traffic to the router".
But the question is why the router has no connection information for these packages.
And here i can only speculate for now: The drops are observed in times where most users are at the internet. My guess is that googles replies took to long (mybe because it prior push traffic lower than other) so that the connection times out. The timeouts on this particular router are about 10s for TCP packages and 5s for SYN. After some more time the conntrack will remove his memory about old "invalid" traffic and from that time on every traffic from this connection looks like input traffic.
Thats also one possible explanation why the invalid rules are not triggered.
I think @samuel should observe it a little more and see if any pattern is recognizable. But generally i sare @chexum oppinion that these drops are ignorable.


Twelfth man in cricket

Question: I heard a term called twelfth man in cricket.
W.K.T, there are 4 players apart from the playing 11.

Is it necessary to fix a particular player as 12th man before
starting the match?
What is the difference between the twelfth man and the other substitutes?
Answer: From Wikipedia:

Twelfth man
Traditionally, the first substitute player who fields when a member of the fielding side is injured.

So the twelfth man is one of those substitutes. The only difference is that he has priority to field than other substitutes.

Is it necessary to fix a particular player as 12th man before starting the match?

There it no laws for the twelfth player in laws of cricket regarding nomination of the 12th or substitute player. But in ICC's match playing conditions PDF, a captain shall nominate 4 substitutes along with playing XI. There is no specification of 12th man.
Comment: Actually, none of the four selected players has priority over the others. Any of them can be selected to field for any player and that selected substitute is then termed the twelfth man.

Twelfth man in cricket


How to predict gam random field markov model to grid?

Question: I am working on a similar problem as the example in the mgcv package.
I have created a similar model and would like to predict to grid level instead of the districts identified in the training data. Is this possible? If so, how?
I have created a prediction dataset as an example.
library(mgcv)
data(columb)
data(columb.polys)

b <- gam(crime ~ s(district,bs="mrf",xt=xt),data=columb,method="REML")
plot(b,scheme=1)

#Create prediction dataset

df <- data.frame(district=numeric(0),x=numeric(0),y= numeric(0)) #Create empty df to store x, y and IDs for each polygon

# Extract x and y coordinates from each polygon and assign district ID
for (i in 1:length(columb.polys)) {
  district <- i-1
  x <- columb.polys[[i]][,1]
  y <- columb.polys[[i]][,2]
  df <- rbind(df,cbind(district,x,y))
}

sp <- df %>%
  group_by(district) %>%
  do(poly=dplyr::select(., x, y) %>%Polygon()) %>%
  rowwise() %>%
  do(polys=Polygons(list(.$poly),.$district)) %>%
  {SpatialPolygons(.$polys)}

#Convert sp to sf object
sp_sf <- st_as_sf(sp, proj4string='+proj=utm +zone=48N +ellps=WGS84 +units=m')
sp_sf <- st_set_crs(sp_sf, '+proj=utm +zone=48N +ellps=WGS84 +units=m')

#Make grid
grid <- st_make_grid(sp_sf,
                           cellsize = 0.05,
                              what = "centers", crs = '+proj=utm +zone=48N +ellps=WGS84 +units=m')

#intersection of grid and Columbus, Ohio
pred_grid <- st_intersection(grid, st_buffer(sp_sf, dist=0))

pred_b <- predict(b, pred_data)

Error in data[[txt]] : subscript out of bounds

Thank you!
Answer: This isn't possible with the MRF, at least so far as I know. In the model you show, you only have the MRF smooth, so the conditional distribution of the response is modelled as estimated values for each district under the constraint that the nearby districts are similar to one another-ish. To use the model you need to predict for the districts, not some other data.
I'm assuming in this line
pred_b <- predict(b, pred_data)

you meant
pred_b <- predict(b, pred_grid)

If your real problem has a mixture of data on a grid and at a region level, you can predict on the grid, but you'll need to assign each grid location to a region so that you can pass in a column like District which is what feeds into the the MRF term.


$\tanh$ activation function and sparsity constraint

Question: According to LeCun's paper "effient backprop" [1] the $\tanh$ activation function should be preferred over the logistic activation function for the hidden units in neural networks.
For the $\tanh$ units an output of $a_i = -1$ is considered as inactive. But is correct especially for the sparsity constraints (e.g. sparse autoencoders)? An inactive unit should act like not present. But an unit with $a_i = -1$ contributes to the $\text{logit}$ $\sum_i (w_ij * a_i)$ extremely. In contrast a unit with output $a_i = 0$ would not contribute to the sum. But for output $a_i = 0$ the $\tanh$ unit is absolutely not in the saturated regime.
What output is considered as inactive if there is an sparsity constraint to force most of the units to be inactive?
Is there a paper about the "sparsity of activation" for $\tanh$ units?
[1] Y. LeCun, L. Bottou, G. Orr and K. Muller: Efficient BackProp, in Orr, G. and Muller K. (Eds), Neural Networks: Tricks of the trade, Springer, 1998
Comment: We use mathjax for math formatting. http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference More information
Answer: It does not really matter. Let's say you wanted to fix this problem by using $f(x) = \tanh(x) + 1$ as a transfer, which makes its co-domain $(0, 2)$ but does not change the gradient characteristics which make it superior to the logistic.
Consider a very small network with a single input, output, weight and bias: $y = w f(x) + b$. This network is equivalent to the network $y = w \tanh(x) + b + 1$. This shows that you could also use a different bias $b' = b + 1$ letting you arrive at $y = w \tanh(x) + b'$.
Long story short, the network will have the same output and the same gradients. The only difference would be if you regularised the bias with weight decay or such--but it is not a good idea to impose this kind of regularisation on the bias anyway.
What is much more interesting with respect to sparsity, is that the gradients for sparse units are near zero--this makes the output of the net not vary with the unit, albeit only linearly. This is also shown by the manifold tangent classifier network, where the units tend to become either 0 or 1--they try to saturate in order to make the network invariant to local distortions.


Complex clustering using generalized estimating equations

Question: I am studying judicial decision making. I have coded a large number of cases, each of which was heard by three judges randomly selected from a judge pool. I have coded whether each judge's vote was a conservative or liberal vote, and I have each judge's political affiliation. I want to see if a judge's party status predicts voting behavior. If each observation is a particular vote for a judge, then the outcome is binomial (liberal or conservative vote) and the primary predictors are that judge's political affiliation and one or more variables representing the number of conservatives out of the additional two judges.
I believe that my observations may be correlated in two ways: I have three observations for each case (because three judges vote in every case), and I have many observations for each judge. Observations for the same case may be correlated, as may be observations for each judge. I am not particularly interested in learning about any one judges in particular. What I really care about is: as a general matter, do conservative judges tend to vote more conservatively.
I think I want to use generalized estimating equations to assess the data so that I can account for these potential correlations, but I'm only finding examples for how to do this with software for one type of correlation. Do you agree that GEE is the right approach, do you think a different approach might be better, and how do I do this on R or a similar software?
Answer: This sort of two-way correlation has been studied by multiple independent people (including me).  Two useful published references are by Miglioretti and Heagerty and by Cameron, Gelbach and Miller.  I have a blog post here, including a link to R code. Cameron et al
The basic idea is that you get the same sort of sandwich estimator as with longitudinal/clustered data, but the middle of the sandwich is
$$\sum_{(i,j)\sim(i',j')} U_{ij}U_{i',j'}^T$$
where $(i,j)\sim (i',j')$ if the pairs share either index and $U$ is the estimating function.
In more generality, you add up $U_s$ and $U_t$ for pairs where you don't know a priori that the correlation is zero, and you get a consistent estimator as long as that's an asymptotically vanishing fraction of all pairs.
Comment: Thank you, I believe I should be able to take it from here!


Active Record Query for subset conditions

Question: In my application, staff users create patient records. The patient model has a HABTM relationship with an insurance model to record the insurance/s the patient is covered by.
Staff users create referral requests that belong to patients. Staff users also belong to universities, which in turn belong to markets. For example, staff user Jim might belong to Harvard University, and Harvard University in turn belongs to the market Boston.
Clinician users belong directly to a market.
Clinician users each have a clinician_profile, and their profiles have a HABTM relationship with the insurance model to record the insurances/ the clinician accepts. ClinicianProfile belongs to user where role: :clinician.
The roles are defined with an enum:
  enum role: { staff: 0, clinician: 1, admin: 2 }

A staff user creates a patient record, which flows directly into creating a referral request, which flows directly into dispatches/new which I intend to display a list of all of the clinicians that are in their market and accept at least one of the insurances that the patient is covered by. This is on the path to figuring out how to actually send the referral request and relevant patient details to each of these clinicians. I've commented below what I'm trying to do with each step and what is not working - any advice on how to do this would be greatly appreciated.
Dispatches Controller New Action:
def new
#This works
@patient = @referral_request.patient
#This works
@user = current_user

#This works - returns a list of clinician users who match the current user's university's market.
@market_matched_clinicians = User.clinician.where(market: @user.university.market)

#This doesn't work -My intention is for this to return a list of clinician profiles that share at least one insurance with the patient.
@matched_clinician_profiles = ClinicianProfile.where(insurances: [@patient.insurances])

#Once I have all of the matched clinician profiles, how can I use that to return the corresponding clinician users?
# @matched_clinicians = ???

@dispatch = @referral_request.dispatches.new
end

Models:
class User < ApplicationRecord
  include Clearance::User
  include StaffUser
  include ClinicianUser
  include AdminUser

module ClinicianUser
  extend ActiveSupport::Concern

  included do
    has_one :clinician_profile
    has_many :lists
    has_many :universities, through: :lists
    has_many :dispatches
    has_many :referral_requests, through: :dispatches
    after_create :create_clinician_profile
    belongs_to :market
    validates :market_id, presence: true, if: :clinician?
  end

class ClinicianProfile < ApplicationRecord
  belongs_to :user, -> { where role: :clinician }
  has_and_belongs_to_many :languages
  has_and_belongs_to_many :races
  has_and_belongs_to_many :insurances
  has_and_belongs_to_many :genders
end

class Patient < ApplicationRecord
  belongs_to :author, -> { where role: :staff }, class_name: 'User', foreign_key: 'user_id'
  has_and_belongs_to_many :genders
  has_and_belongs_to_many :concerns
  has_and_belongs_to_many :insurances
  has_and_belongs_to_many :races
  has_many :referral_requests
  belongs_to :staff_doctor, class_name: 'User', foreign_key: 'staff_doctor_id'

  validates :staff_doctor_id, presence: true

class ReferralRequest < ApplicationRecord
  belongs_to :user, -> { where role: :staff }
  belongs_to :patient
  has_many :dispatches
  has_many :clinicians, through: :dispatches
  has_and_belongs_to_many :languages
  has_and_belongs_to_many :races
  has_and_belongs_to_many :genders
  validates :user_id, presence: true

  enum status: { created: 0, sent: 1, shared: 2, closed_under_care: 3, closed_not_seeking_care: 4, closed_unresponsive: 5 }
end

Here is the view where I am trying to render these results:
<% provide(:title, "New Dispatch") %>

 These clinicians match your market and your patient's insurance

          Provider ID
          Name
          Provider Market
          Insurances Accepted
          Gender/s
          Race/Ethnicity
          Languages

    <% @users.each do |user| %>

        <%= user.id %>
        <%= user.name %>
        <%= user.market.name %>
        <%= user.clinician_profile.insurances.map(&:name).to_sentence %>
        <%= user.clinician_profile.genders.map(&:name).to_sentence %>
        <%= user.clinician_profile.races.map(&:name).to_sentence %>
        <%= user.clinician_profile.languages.map(&:name).to_sentence %>

    <% end %>

Answer: To get ClinicianProfiles that match at least one insurance:
@profiles = ClinicianProfile.joins(:insurances)
                            .where(insurances: { id: @patient.insurances })
                            .group('clinician_profiles.id')
                            .having("count(*) = 1")

You can then fetch the users by joining clinician_profiles:
@users = User.joins(:clinician_profile)
             .where(
               market: @user.university.market,
               clinician_profile: { id: @profiles }
             )

Comment: Thanks Max - when I try this i get an ActiveRecord Configuration error in Dispatches#new. Specifically it says "Can't join 'User' to association named 'clinician_profiles'; perhaps you misspelled it?". When I mess around in console, I can do something like c = User.clinician.first, then p = c.clinician_profile. p.user returns the clinician who owns the profile, but p.clinician returns "undefined method 'clinician'". Is this the problem? Do I need to fix my model associations?
Comment: Ah - should be `clinician_profile` in singular. Edited.
Comment: I also removed the double market that you edited out - in the @profiles block there is a misspelling of clinician in the .group line, but after changing those things, I am getting the error: SQLite3::SQLException: no such column: clinician_profile.id
Comment: It should be `'clinican_profiles.id'` - its the name of the table there and not the association so it needs to be plural.
Comment: do you mean clinician_profiles.id? vs. clinican_profiles.id
Comment: Yes. Didn't even see that typo. Anyways its the name of the table + '.id'.
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/156571/discussion-between-mike9182-and-max).
Answer: Do it in Rails way.
I assume you defined association with ClinicianProfile in Insurances Model. Add has_many :clinician_profiles, through: insurances to Patient and call @patient.clinician_profiles.
class Patient < ApplicationRecord
  belongs_to :author, -> { where role: :staff }, class_name: 'User', foreign_key: 'user_id'
  has_and_belongs_to_many :genders
  has_and_belongs_to_many :concerns
  has_and_belongs_to_many :insurances
  has_and_belongs_to_many :races
  has_many :clinician_profiles, through: insurances
  has_many :referral_requests
  belongs_to :staff_doctor, class_name: 'User', foreign_key: 'staff_doctor_id'

  validates :staff_doctor_id, presence: true

Comment: This does not answer the question. "My intention is for this to return a list of clinician profiles that share at least one insurance with the patient."
Comment: Yes this is exactly what you mean.
Comment: No - he does not intend to use insurances as a join table. You are not getting the domain model here.
Comment: Do you think it is possible without join table?


Duvida simples em relacao a vetores

Question:
Faça um programa que leia o valor dos elementos de um vetor de inteiros com 10 posições. Depois que os dados forem fornecidos o programa deverá:

a.    Mostrar o maior e o menor valor do vetor
b.    Mostrar quantos números pares e quantos números ímpares existem no vetor
c.    Mostrar a média aritmética dos valores do vetor
A resposta a que cheguei foi essa, mas não consigo rodar o programa por erro de sintaxe:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace WorkBJA
{
    // Vetor de inteiros com 10 posições.
    //     a. Mostrar o maior e o menor valor do vetor
    //     b. Mostrar quantos números pares e quantos números ímpares existem no
vetor
    //     c. Mostrar a média aritmética dos valores do vetor
    class Program
    {
        static void Main(string[] args)
        {
            int[] Vetor = new int[10];
            for (int i = 0; i < 10; i++)
            {
                Console.Write($"Elemento {i + 1} do Vetor: ");
                Vetor[i] = int.Parse(Console.ReadLine());
            }
            Console.WriteLine("\n");

            Console.WriteLine($"O Maior Valor do Vetor é {AchaMaior(Vetor)}");
            Console.WriteLine($"O Menor Valor do Vetor é {AchaMenor(Vetor)}");
            Console.WriteLine($"Existem {ContaPares(Vetor)} elementos Pares no
Vetor");
            Console.WriteLine($"Existem {ContaÍmpares(Vetor)} elementos Ímpares no
Vetor");
            Console.WriteLine($"A Média Aritmética dos elementos do vetor é
{AchaMédia(Vetor):F2}");
            Console.ReadKey();
        }
        static int AchaMaior(int[] V)
        {
            int Maior=V[0];
            for (int i = 1; i < V.Length; i++)
            {
                if (V[i] > Maior)
                    Maior = V[i];
            }
            return Maior;
        }
        static int AchaMenor(int[] V)
        {
            int Menor = V[0];
            for (int i = 1; i < V.Length; i++)
            {
                if (V[i] < Menor)
                    Menor = V[i];
            }
            return Menor;
        }
        static int ContaPares(int[] V)
        {
            int Pares = 0;
            for (int i = 0; i < V.Length; i++)
            {
                if (V[i] % 2 == 0)
                    Pares++;
            }
            return Pares;
        }
        static int ContaÍmpares(int[] V)
        {
            int Ímpares = 0;
            for (int i = 0; i < V.Length; i++)
            {
                if (V[i] % 2 != 0)
                    Ímpares++;
            }
            return Ímpares;
        }
        static double AchaMédia(int[] V)
        {
            int Soma=0;
            double Média;
            for (int i = 0; i < V.Length; i++)
            {
                Soma += V[i];
            }
            return (double)Soma / V.Length;
        }
    }
}

Comment: Coloque o erro que está ocorrendo para que possamos te ajudar
Comment: Qual é a sua dúvida? Lembrando que o [pt.so] é um site de perguntas e respostas e se não houver uma pergunta pontuando uma dúvida concreta a sua questão não poderá ser catalogada e portanto será fechada. Veja [ask], [Guia de sobrevivência do Stack Overflow em Português](https://pt.meta.stackoverflow.com/q/8045) e faça o nosso [tour].
Answer: Não utilize acentos em suas variáveis como em Ímpares e também na hora de dar o ConsoleWriteline(), utilize o + para concatenar sua string.
Exemplo:
ConsoleWriteline("O Maior Valor do Vetor é " + AchaMaior(Vetor));

Duvida simples em relacao a vetores


Does the dissonance of scale notes, to the tonic, follow a pattern of ascending fifths?

Question: Does the dissonance of scale notes, to the tonic, follow a pattern of ascending fifths?
For example start with a C major scale and lets track the position of the C note in ascending keys ... C G D A E B etc.
In key of C, C is at position 1.
In key of G, C is as position 5.
In key of D, C is at position 2.
In key of A, C is at position 6.
In key of E, C is at position 3.
In key of B, C is at position 7.
In key of F#, C is at position 4#.
(I am using 5# instead of 5b as we're ascending.)
The resulting pattern is 1-5-2-6-3-7-4#.
This means that the 4 must be on the other end ...
4-1-5-2-6-3-7 ... then
4#-1#-5#-2#-6#-3#-7# ... etc.
(And this pattern follows if we had descended.)
4b-1b-5b-2b-6b-3b-7b ... then
4-1-5-2-6-3-7        ... then
4#-1#-5#-2#-6#-3#-7# ... etc.
(In this manner one might say key signatures 'start' on F, the 4, the beginning of the grouping, and not C.)
Regardless, the question is, does the 4-1-5-2-6-3-7 also denote the dissonance of the scale tones relative to the I?
Is 4 (F) the most consonant to C?, then 5 (G), then 2(D) etc?
Comment: Related: [What is the list of intervals in order of dissonance](https://music.stackexchange.com/q/46509/21766)
Answer: No, the 4-1-5-2-6-3-7 does not denote the dissonance of the scale tones relative to I (at least not strictly).
While both 4 and 5 are quite consonant with 1, 6 is more consonant with 1 than 2 is, as 6 creates a 6th or a 3rd, while 2 creates a 2nd or a 7th.
Using your example, F (4) and G (5) may be among the most consonant with C (1), but A (6) and E (3) are both more consonant with C (1) than D (2) is.
Comment: How about against F?
Comment: @RandyZeitman - The dissonance curve, complete with hump, still applies. G is less consonant with F than the note after G, D, is.
Answer: It's difficult talking about the circle of fifths without a cirlce so I made one...

Coincidentally, I was thinking about one of the concepts in this question only a few days ago: if you want to generate the major scale from the circle of fifths, you need to go back one step on the circle from a starting position tonic to get the ^4 scale degree.
Of course you can also say that if you just go up seven steps from a starting tonic position you will end on a #^4 and have the Lydian mode. (Look into The Lydian Chromatic Concept of Tonal Organization by George Russell. Apparently he thought this particular rooting of a segment of the circle of fifths was the way to organize tonal music.)
You can take this idea further and step back further on the circle to get all the modes. This is the chart I made for myself to illustrate the idea...

The conclusion I came to was any series of seven adjacent steps on the circle will produce a diatonic tone set and you can then select any one of those tones to be a tonic to create any of the seven modes. Therefore, to select any one particular scale from the circle is totally arbitrary. You can't use the circle of fifths to claim the major scale is the center, nor the Lydian mode, nor any other.
That's a long way to get to the first point which is scale degree sequences like this...

1-5-2-6-3-7-4#

...when applied to the circle of fifths are arbitrary. You can make C any scale degree you want. I can simply say C isn't ^1 but instead it's #^4... of Gb Lydian.
Putting scales degrees arbitrarily on the circle of fifth seem to just obscure the real question.
The degree of dissonance of any scale degree against the scale tonic will simply be the degree of dissonance for the interval between the two tones. The circle of fifths has nothing to do with it.

Is 4 (F) the most consonant to C?, then 5 (G), then 2(D) etc?

Even if we say for the sake of argument that the circle of fifths position does determine dissonance of a scale degree, F and G are the same distance from C how could one be more or less dissonant based on circle position? They would be equal.
This question seems to be one of a series that tries to find the inherent dissonance of individual things. (At least that's how I've been reading between the lines.) B supposedly would be very dissonant and that kind of matches up with the leading tone being a tendency tone. But then E should be the next most dissonance. However, E is the mediant, a stable scale degree. This idea seems to conflate scale degree characteristics, tendency tone behavior, and dissonance.
Along similar thinking, if we compared the perfect fifth C G and A E the first is closer to the tonic than the other. Does that mean the first perfect fifth is more consonant than the second perfect fifth. That doesn't seem to make much sense to me.
This search for inherent dissonance kind of reminds me of Charles Rosen's ideas about structural dissonance in that both go beyond viewing dissonance/consonance as a quality of intervals. But grand ideas like that seem to require a sophisticated theory. Rosen qualifies the meaning. It's not simply dissonance it's structural dissonance. It's clear that he meant to create a new concept... and he wrote a lot to explain it.
Associating a scale degree to a series position and then calling it simply 'dissonant' strikes me as a very incomplete concept.
Comment: Your idea is very incomplete. I took it a few steps further to see where it logically leads. I do think I addressed your question: you cannot measure dissonance in the way you propose.
Comment: The burden is one you to make a compelling case for your theory, not for me fill in something missing. Basically, I don't think you can make this idea work. I tried to explain that. IMO j
ust stick with the standard concepts of dissonance, scale degrees, and tendancy tones, etc.
Comment: I don't understand why you're involving CG vs. AE. In the question it's simply asking if the notes are more dissonant to C (or perhaps F) as one does to the right. You wrote a lot about your own explorations but didn't seem to address mine.
Comment: ?... if you say it's incomplete then what's missing? How would you know?

Also, why did you start your circle of fifths with C at the top? As you showed in your gamut the C Ionian starts with F ... which makes complete sense to me as keys are built on fifths ... so if you're next key is C then you must go a fifth up from a note that would also be in that following key ... F must be in C.
Comment: "Incomplete" and "wrong" are not quite the same.
Comment: I think I figured it out. My math applies to Pythagorean Tuning.
Comment: Hi Micheal, I am making videos about my 'music theory calculator' shortly and I was reviewing this thread I started and saw you use the word 'gamut' instead of what I have ('grouping') and I'd like to use gamut instead. (The dictionary.com thesaurus didn't have gamut as a synonym for group.) The question, any issue with my using 'gamut'? There's nothing else from the diagram I didn't have (see image below, circa 2007). I'd have no issue acknowledging your contribution in the video.

http://myeasywebaddress.com/Stage/MusicTheory/permission.jpg
Comment: Hi @RandyZeitman, gamut means the musical letters `ABCDEFG` and perhaps a more modern view would include all transpositions. It's sort of synonymous with "diatonic." If any of this was helpful for your project, I'm glad, but no need to credit me.
Comment: Ok, thanks. As I said I'm just using it instead of 'group'.
Answer:
resulting pattern is 1-5-2-6-3-7-

but as we know the order and the degree of dissonante  intervals is the 5th,major and minor 3rd and 6th,4th,major and minor 7th and tritonus.
https://en.wikibooks.org/wiki/Music_Theory/Consonance_and_Dissonance
so will find no correlation with the circle of fifths.
(but you'll find  other interesting relations and patterns if you search for the augmented triades or the diminished tetrades!)
Answer: Ok, just found this - dissonance follows the fifths in Pythagorean tuning.
https://en.wikipedia.org/wiki/Just_intonation#Pythagorean_tuning
Comment: critiquing your posts and comments is *not* rudeness. I've flagged your comment. If you want to participate on this forum you need to be able to accept constructive criticism.
Comment: I really don't get your concept. Are you suggesting that interval `C D` is more consonant than `C E`? You leaping from a tuning chart to a vaguely defined concept of dissonance. Maybe add to your self-answer a few examples of practical application.
Comment: That chart just states what all the answer have said: dissonance is a relationship between to pitches. B with 243:128 has that complex ratio _only because it's derived from the interval C to B_.
Comment: That doesn't mean `B` is the most dissonant note in `C` major, it means the interval `C to B` is very dissonant.
Comment: It's worth noting that the other two answers agree with mine conclusion that your concept does not work. I've tried to give you more detail about _what_ the problems are.


#include in Expect script

Question: So I am having trouble making TestCases with expect scripts, I have like 10 TestCases which all starts and ends with the same "functions" like login and logout or turning some flags off, is there a possibility to include them or execute them remotely from my script, like spawn login.exp or even better to put them in functions ?
TC01.exp
#!/usr/bin/expect -f
set timeout 5

#example of getting arguments passed from command line..
#not necessarily the best practice for passwords though...
set server [lindex $argv 0]
set user [lindex $argv 1]
set pass [lindex $argv 2]
set no [lindex $argv 3]
set counter 0

# connect to server via ssh, login, and su to root
send_user "connecting to $server\n"
spawn ssh $user@$server

#login handles cases:
#   login with keys (no user/pass)
#   user/pass
#   login with keys (first time verification)
expect {
  "> " { }
  "$ " { }

  "assword: " {
        send "$pass\n"
        expect {
          "> " { }
          "$ " { }
          "assword: " {
                    send_user "\nLogin failed\n"
                                    incr counter 1
                    exit 5

                }
        }
  }
  "(yes/no)? " {
        send "yes\n"
        expect {
          "> " { }
          "$ " { }
        }
  }
 default {
        send_user "Login failed\n"
        incr counter 1
        exit
  }
}

#TEST CASE HERE

#login out
send "exit\n"

expect {
    "> " {}
    default {}
}

if { $counter > 0 } {
    send_user "\nTestCase finished with some errors!\nFAILED!!!\nERRORS $counter\n";
    exit 4;
}

send_user "\nTestCase finished with SUCCESS!\nERRORS: $counter\n";

So i would like to have login and count_error as functions, so I would be able to create my test cases just like this:
TC01.exp
#!/usr/bin/expect -f
set timeout 5
set server [lindex $argv 0]
set user [lindex $argv 1]
set pass [lindex $argv 2]
set no [lindex $argv 3]
set counter 0

login($server, $user, $pass)

#TestCase

Errors($counter)
exit

Comment: yes. show us what you have and we can help you refactor. as you question stands now, that's about as much help as you can get.
Comment: I've updated my quiestion, I hope this time I've explained it better.
Answer: Expect is actually Tcl with some bindings to pty's and fork() thrown in. All very well described on http://tcl.tk
Functions in Tcl are done with proc (look here) eg:
lib.tcl
proc login {server user pass} {
  # Your expect code goes here
  return $errorCount
}
proc errors {errorCount} {
  if {$errorCount > 0} {
    # Your error handling code here
  }
}

test:
#!/usr/bin/env expect
source lib.tcl
set errors [login $server $user $pass]
# your test case here
errors $errors

#include in Expect script


Trying to use "rotatelogs" on Apache 2.2 as non-root

Question: We have Apache set to run as www user on a RedHat box, but it's logs are written w/root ownership as the process itself run as root. It seems the best course of running would be to 1) run rotatelogs as a non root user,  and 2)  to put the logs in a place set aside for the user w/proper permissions.
Can this be done? I don't see a method here:
https://httpd.apache.org/docs/2.2/programs/rotatelogs.html
Comment: This seems like a  reasonable question, why the down vote out of the gate?
Comment: Because people like to exercise their hard earned right to downvote at 125 rep.  Quite petty, tbh.  Personally, I think everyone should be able to see who is casting votes.
Comment: Our field is prone to people thinking that because they do not have a problem nobody else should or, worse, that if they cannot think of a way to break something nobody else can either. Working in .gov, I often find cases where it's easier to harden something like this than deal with policy or audit questions about it later.
Answer: There is no need to do anything in respect of file ownership. Red Hat ships their systems with sensible defaults. Millions of web servers run every day with their httpd logs owned by root. Best practice is to leave well alone in this instance.
By default the httpd logs on a RH system are rotated using logrotate. This is run daily as the root user so there is no issue with the ownership. You can adjust the way the logs are rotated by editing the logrotate configuration (/etc/logrotate.conf, /etc/logrotate.d/httpd).
If you are using piped logs as your link suggests, then you will need to configure your own cron job to run as root.
This is all normal stuff.
Comment: Thanks, but does not address the question, "Can this be done?". I do not see having a process run as root as a best practice, when it could be a non-root user. Whether or not others do. The owner of the file just need to be the non-root user. I suspect if one runs apache on a high port/non root  then the master process is non root and the rotatelogs is as well. My question is if you run Apache as non-root (and knowing the master process is going to run as root, can the rotatelogs process run as non root.)
Comment: Thanks again for the comments. I will respectfully slightly disagree: Running root when you could take steps not to is not best. It does depend on the risk/reward/effort though for sure.
Comment: Perhaps you should address that to the engineers at Red Hat /Apache (and other distros) I'm sure they'd love to know they've been doing it wrong all this time. You could even download the Apache source and hack  it into shape...
Comment: Agreed. someone's gotta do it.
Comment: Actually, my question is if you run Apache on port 80 as non-root (knowing the master process is going to run as root to bind and then drop privs/spawn worker processes), can the rotatelogs process run as non root? rotatelogs stays in the process list, it's not transiently run via cron. I just think there should be a method for that process to then drop privs as well. An alternative would be for the root master process to spawn another non root process that simply pipes data to rotatelogs.
Comment: Found a conversation: http://serverfault.com/questions/379285/logrotate-security. At this point I believe the answer is no you can't do it because the rotate logs program has not been programmed to call setuid to a non root user. I will have to look into the source to confirm/deny 100%.
Comment: Also using capabilities to allow not root process to bind to 443/80 would then  have all non processes, including logrotate.
Comment: So there you go, like I said, everyone is already using 'best' ;)
Comment: @Thetimehascome if you actually talked with any of the engineers who work on Apache, they'd be the first to remind you of the multiple decades which have gone into improving it rather than dogmatically asserting "the way this works now cannot be improved". Our field has seen a huge amount of engineering time going into incremental security improvements because defense in depth is the best way to avoid catastrophes.
Answer: The exact way to invoke it will depend on local configuration (e.g. apache is disabled by default on RHEL 6) but you can use sudo, runuser, etc. like this:
ErrorLog "|sudo -u apache rotatelogs -f /var/log/httpd/errors.%Y-%m-%d.log 86400"


How to clear textbox value after insert to the database in c#?

Question: In my program have 3 textboxes. which are for subtotal,discount and grand total.Subtotal value will display after some calculations and user should type discount manually.Then after type discount value in textbox it will display grandtotal value automatically in grandtotal textbox.For this I used textchange event like this
private void txtdiscount_TextChanged_1(object sender, EventArgs e)
{
    txtgrandtotal.Text = (Convert.ToDouble(txtsubtotal.Text)-(Convert.ToDouble(txtdiscount.Text) * Convert.ToDouble(txtsubtotal.Text))).ToString("#,0.00");
}

After insert data into database I need to clear this textboxes.How can do this?I used textbox.text=""; for all text boxes but txtgrandtotal didn't clear (conflict with txtdiscount_TextChanged_1 event ).
Comment: Check to see if textgrandtotal.Text = "" if so skip the line of code you have in the textchanged event
Comment: Not a wise idea to call Convert.ToXXXX on an textbox where your user can type anything. What happens if you type a letter instead of a number?
Answer: Try setting all textBox text to "" and just calculate txtgrandtotal when txtsubtotal and txtdiscount is not empty!
private void txtdiscount_TextChanged_1(object sender, EventArgs e)
{
    if (txtsubtotal.Text == "" || txtdiscount.Text == "")
        return;
    txtgrandtotal.Text = (Convert.ToDouble(txtsubtotal.Text) - (Convert.ToDouble(txtdiscount.Text) * Convert.ToDouble(txtsubtotal.Text))).ToString("#,0.00");
}


How can i increase the speed of my Blast beats and one handed rolls (grav blasts) on drums?

Question: I have the motion down for both of these techniques but I cant seem to speed them up for faster death core and metal drumming, is it just down to physical strength of the forearms that determine the speed or is the a way to alter my technique to make it bounce faster on the snare. I also think that my basts and gravs are far too quiet how can i increase the power of the hit?
Answer: In my humble opinion, there are two things you need to separate:

Physical strength plays very little role (up to the insane speeds that death core dudes operate) on your blast beats and almost zero role on the one handed rolls.
Forearms plays a big role but your whole arm is involved hence takes a part in the overall technique and again almost zero role on the one handed role.

Pillow practice is not liked here for some reason (See Drum set made from towels/pillows). But for the blast beats the main trick is avoiding flams at higher speeds (weirdly when you practice flams you avoid unison playing, brain is funny). The main issue on the real drumset is that the bounce of the cymbal is different than of the snare so you need to teach your brain to compansate it on the fly. The easiest is to play one hand on pillow one on the practice pad. And pillow at those speeds is tough to play so don't force it immediately just try to get a decent coordination and practice later on real drums too (also alternate hands).
For the one handed roll, you need to increase your technique either via Moeller motion or by stick control via finger sticking. I recommend the former as you can do other nasty things with it as a bonus. You should not try to have action from forearm that would temporarily increase your speed but your tempo and consistency will detoriate over time. Instead it should be more of a slight of hand in a natural fashion. Google Johnny Rabb for extreme use of it and how the stick moves inside in his hand (you don't need to copy it but it's a good example for economy motion).
For the quietness, it is because of your hand technique and level. Your brain is still trying to figure out how to tense up the forearm and release it at those speeds because you are not used to the economy motion using different techniques such as Moeller. Hence in nerdy terms your bandwidth is not large enough to focus on muscle tension. Think of your feet, until you have nailed the single strokes you were busy keeping them in tempo now you can concentrate more on how to contract groups of muscles without losing the tempo to speed up. One handed rolls gets faster once capture the essence of hiding the rim sound inside the actual snare sound then by increasing your stick travel gradually you can build up speed but for that again you need a proper stick control framework that you are comfortably progressing. In general one handed roll doesn't get super loud due to the bounce stroke limitation.
EDIT: Found it. Watch the very end for the mind-boggling part. If that doesn't cut it, stop playing drums :P I really really really recommend buying his DVD Secret Weapons
Comment: thanks for such an in depth answer :) ive been practicing it over the past few months and im getting the hang of it
Comment: @NathanTaylor My pleasure. Godspeed!
Answer: Increasing speed for technique is something all musicians strive for with almost all technique, and the same principles apply across all instruments.
First, you need a metronome - plain and simple.
Second, a common technique is to move up two clicks on the metronome and then back down one step.  You will need to decide on a length of time to play at that tempo - say two measures.  Make a goal to be able to play the two-measure segment 10 times correctly (continuously!) before changing the tempo.  Start at 60bpm and slowly increase over time - weeks and weeks.
Fast technique is slow technique done in a short amount of time.  If you want to develop fast technique, you need to develop excellent slow technique to develop fine motor control necessary for fast technique.
For percussion, dynamics are a function of speed, not power.  If you want to strike the drum louder, you merely need to strike the drum more quickly.  Be careful that you are not tense.  Also be aware that due to the type of music you play, there is going to be an apex between speed and volume - you're only going to eventually be able to play so loud and so fast, so just be aware.
Comment: Have you heard Blast beats before, I don't think this principle applies to those as they aren't played using your wrists or arms as any other drum technique. they require figure movement...
Comment: @NathanTaylor - Yes, I have heard of them. It doesn't matter if you're using your feet / legs / penguin to create a sound - if you want to make a fast sound, you need to practice the technique of producing that sound slowly, and then work your way up.  As with the coordination of a Blast Beat, you would need to practice coordination between all of your limbs such that they all struck at the same time.  In terms of dynamics, what I said still stands - if you want to play louder, strike the drum more quickly, not with more strength.
Comment: Dynamics is certainly not the function of speed but the distance from the drum head. Yes, strength can also make a significant difference but it is not the healthy technique. What you said is only a part of the answer as the stick naturally has to move faster when hitting the head from a larger distance. I mean, just try hitting the head from one inch distance very fast and see how loud it gets. Pay attention not to lift the stick more than one inch, though.
Comment: @AlenSiljak Apart from the ridiculous necro-comment (it's almost been 4 years), you really didn't offer any new insight - merely re-stated what I had already said in my answer and my own comment.
Comment: @jjmusicnotes, 4 years or 40 years does not make much difference as the sticks and drums have not changed much during the period. Your statement " if you want to play louder, strike the drum more quickly, not with more strength" is still technically incorrect. There is a clear distinction between playing fast and playing louder. One should be able to play fast and quiet, which would be an impossibility according to that statement. And if you find the recommendation to increase the distance to the drum head as "not a new insight" then I'm really sorry.
Comment: @AlenSiljak Apologies, please link a video show incredibly slow and loud playing with sticks five feet away from the batterhead. I show my students that they can control dynamics via letting the stick do the work. They learn that it takes just as much strength to play quietly as loudly, thus they don't strain when they play. How will your students fair in 40 years? BTW: if you increase distance, you need to increase speed (physics). A last thought to close this, as I won't be responding:
https://en.wikipedia.org/wiki/One-inch_punch
Comment: No worries. But, being an instructor, I'm amazed that you are not making a distinction between speed and distance. Here's a simple video that demonstrates the concept: https://www.youtube.com/watch?v=UOWApRns4MA. Notice how, with every step in dynamic range, the distance from the stick to the drum head grows significantly. I'd post two more videos but they're proprietary - Jojo Mayer, and Peter Erskine's Drum Method, where they clearly describe dynamics. Here, on the other hand, is Peter Erskine playing fast but also not loud: https://www.youtube.com/watch?v=y1YKQ904iDI.
Comment: It is unfortunate that you will not reply, otherwise I'd ask you to post a video demonstrating this kunf-fu one-inch strike on drums, resulting in sounds with high volume.
Comment: Anyway, to get back to the topic, I believe it is only the technique. See https://www.youtube.com/watch?v=ir_KZNsTNiQ
Comment: To substantiate the distance/speed discussion, please see Percussion Guide for the Beginning Band Student, Lesson Six: Dynamics / Height of Stick chapter. Available at Google Books.


How can I know my fit is good?

Question: I have some data and I realize a fit by myself but according to me the fit is not good at all... the problem is I would like to have a formula which tell me if the fit is good or not... I precise I fit data using Bessel functions. Thank you for your help !
Comment: No I have some data that I fit with bessel functions and I just want to check if my fit is good or not.
Answer: You cannot determine if a fit is good or bad if you do not first express what means "good" and "bad" on mathematical form.
Thus, you have to chose a criterion of fitting which allows to compute a value which is a measurement of the goodness of fit between the set of observed values and the values expected under your model (That is the formula involving Bessel functions).
First, chose a standard test which underlying measure of fit appears convenient in your case. They are a lot :
https://en.wikipedia.org/wiki/Goodness_of_fit

How can I know my fit is good?


Commutant of tensor product of commutants contains the tensor product of von Neumann algebra

Question: Let $A_1$ and $A_2$ are two von Neumann algebras acting on the Hilbert spaces $H_1$ and $H_2$ respectively. Let $A_k'$ denotes the commutant of $A_k$ in $B(H_K)$, for $k=1,2$. I want to show that $A_1 \bar \otimes A_2 \subset \left(A_1' \bar \otimes A_2'\right)'$, where $A_1 \bar \otimes A_2$ is defind to be
$$A_1 \bar \otimes A_2:=\left\{a_1\otimes a_2~|~a_i\in A_i, i=1,2\right\}''.$$
Also note that $A''$ denotes the double commutant of $A$, that is, $A''=(A')'$. Please help me to solve. Thank you.
Answer: I will change the notation to $M=A_1$ and $N=A_2$. Note that, for any subsets of the bounded operators, say $S,T$, if $S\subset T$, then $T'\subset S'$.
It suffices to show that $M'\bar{\otimes}N'\subset\{a\otimes b: a\in M,b\in N\}'$, since taking commutants reverses the inclusion. Since by definition $M'\bar{\otimes}N'=\{x\otimes y: x\in M', y\in N'\}''$, it further suffices to show that $\{a\otimes b: a\in M,b\in N\}\subset\{x\otimes y: x\in M', y\in N'\}'$, again the reason being that taking commutants reverses the inclusion.
But this is obvious: let $a\in M,b\in N$ and $x\in M',y\in N'$. We have $ax=xa$ and $by=yb$, so
$$(a\otimes b)(x\otimes y)=ax\otimes by=xa\otimes yb=(x\otimes y)(a\otimes b)$$
as we wanted.


Extrair arquivo .zip

Question: tenho um arquivo que é zipado pelo o 7-Zip, na hora de descompactar pelo c#, utilizando o seguinte código:
 try
            {
                ZipFile.ExtractToDirectory(caminho + "\\" + arquivo, arquivo);
            }
            catch (Exception e)
            {
                MessageBox.Show(e.ToString());
            }

Só que desta forma não descompacta e acontece a seguinte Exception:
The archive entry was compressed using an unsupported compression method.

Qual outra forma que eu poderia descompactar este arquivo.
Comment: http://stackoverflow.com/questions/836736/unzip-files-programmatically-in-net
Answer: O 7zip pode usar algoritmos de compressão diferentes daqueles que são usados pelo zip.
Tente usar a biblioteca SevenZipSharp, disponivel no nuget.
Comment: Muito obrigado funcionou utilizando esta biblioteca. Não conhecia.

Extrair arquivo .zip


Composição de objetos em javascript

Question: Estou estudando um pouco mais de javascript aqui e cheguei a um problema:
Tenho um objeto chamado player:
let Player = function () {
    this.name = 'player';
}

Player.prototype = new Sprite();
Player.prototype = new Shape2D();

Seria isso que eu tento fazer, e não funciona, porque o javascript por meio do prototype só aceita uma objeto herdado, e no exemplo acima atribuo as duas a prototype. Se tentarmos chamar assim, sendo "box.x" uma propriedade de Shape2D:
console.log(Player.boxX);

Recebo um "undefined". Então pensei, não quero usar herança nessa situação, e como não gosto de criar escadinha de herança, prefiro tentar achar outras alternativas. Pesquisei e vi sobre composição, mas ainda não entendo como seria em javascript, penso assim:
let Player = function () {
    this.name = 'player';
    this.sprite = new Sprite();
    this.shape2D = new Shape2D();
}

ou
let Player = function () {
    this.name = 'player';
}

Player.prototype.sprite = new Sprite();
Player.prototype.shape2d = new Shape2D();

ou
let Sprite = {x:0, y:0};
let Shape2D = {x:0, y:0, width:0, height:0};
let Player = Object.assign(Sprite, Shape2D); // Sendo Sprite e Shape2D agora objetos literais

como fazer isso de um jeito mais limpo possível?
obj1 = new Player();
obj2 = new Player();

O obj1 seria igual ao obj2? preciso de uma maneira em que os objetos sejam diferentes, que não aconteça de eu mudar algo no objeto sprite de um e acabe mudando no outro por ter apenas referenciado. Eu teria que clonar os objetos para criar essa composição?
Answer: Estava a estudar até agora, e encontrei a maneira de escrever e resolver o problema que considero mais agradavel para mim:
const Sprite = (x=0, y=0) => ({
  x: x,
  y: y
});

const Shape2D = (x=0, y=0, width=0, height=0) => ({
  x: x,
  y: y,
  width: width,
  height: height
});

const spritePlayer = Sprite(0, 0);
const shape2DPlayer = Shape2D(0, 0, 32, 32);

let Player = Object.assign({}, spritePlayer, shape2DPlayer);

Composição de objetos em javascript


How to model multiple inputs to multiple outputs?

Question: 1. My purpose
I am trying to build a model with multiple inputs and multiple outputs, which is something like this:

I am not sure if I need to firstly integrate the xi into X, and yi into Y so as to make the model easier like this:

2. Features of the training data
There are several features for the training data:

There are some missing factors for different entries due to lack of resources.
For example,
X1 may not have x3 and x5,  Y1 may not have y4 and y5,
X2 may not have x1 and x2,  Y2 may not have y4.
There might be multiple projections for X=g(Y). For example

Of course, I would like to minimise the number of projections so as to build a more generalised model.
3. My question
So may I ask how could I tackle this problem?
I feel that the regression (e.g. polynomial regression) and classification (e.g. logistic regression, neural network) models only require one sigle output for each entry.
I also do not think PLS is the right answer as PLS essentially models multiple x variables to a single yi instead of considering the Y=Σyi as a whole.
It seems that it is about modelling multivariate (not multivariable) regression. But how could I deal with missing factors?
Comment: @lanselibai what is the goal of this analysis? Can you give any practical context to your measures or the question?
Comment: @lanselibai I see so does each row of data corresponds to a single "dose" of medication which we can otherwise assume to be independent from other rows? Or are these data in a "wide" format with each row an individual and $X_1$ is a first dose, $X_2$ is a second dose, and so on with $Y_1$ a protein concentration after a period of time from first dose, $Y_2$ concentration after a period of time from second dose, etc.?
Comment: What's the goal of modeling all of your outcome variables at once? Why not model them separately with one model per outcome?
Comment: You might look into latent variable models (aka "joint models").  See this paper, for example:
http://www.cell.com/trends/ecology-evolution/abstract/S0169-5347(15)00240-2
Comment: @AdamO The X is protein properties. e.g. X=(charge, hydrophobicity, beta-sheet propensity, ...); The Y is formulation dosage that could stabilise it, e.g. Y=(pH, ionic strength, sugar, NaCl concentration, etc.)
Comment: @Reid because I do not know how to separate them. i.e. I do not know how to classify the X or Y into smaller groups. So I want to "automatically" group them.
Comment: @AdamO  I was trying to model protein properties to the excipients components in final drug dosage formulation. One particular protein may have multiple dosage forms that can stabilise it.
Answer: tl;dr I recommend this, but only pending extensive visual data exploration.
Your problem as univariate classification
I was about to write an overview of supervised multivariate techniques, but then I realized I'd rather recast your problem.
Given $X$=(charge, hydrophobicity, beta-sheet propensity, ...) and $Y$=(pH, ionic strength, sugar, NaCl concentration, etc.), predict $Z$(=1 if stable, otherwise 0).
This tidily addresses the possibility of multiple $Y$ values for a given $X$, and now you have a "simple" univariate prediction problem. You mentioned logistic regression and neural networks, and those would be good baseline methods to try... if you have any examples with $Z=0$.
Your problem as unsupervised learning
So what if you only have examples where $Z=1$? You can't train a classifier. All you can do is assume that there is something generalizable about the points in your dataset -- some set of relationships that hold up across all the proteins and their stabilizing conditions. For example, maybe sugar concentration minus salt concentration always equals inverse hydrophobicity (I'm just bullshitting, of course). Common tools for uncovering structure in data include:

autoencoders: a neural net with your data as both input and output, and a bottleneck layer in the middle.
Principal components analysis or principal curves
The graphical lasso and its cousins, which convert the observed correlation structure into a network with a limited number of edges. I talk more about my favorite options below.

Missing data overview
Regarding your missing data, it's helpful to know what it was caused by. In a clinical trial, patients might drop out due to side effects and thereby skew the results irreparably. You mentioned resource limitations, which means your missingness pattern might be independent of the missing values. (This is especially true if hydrophobic proteins are no more or less expensive to measure that the rest, and so on for all your measurements.) If that is a leap you're willing to make, then not only can you fill in the missing data; you can reasonably quantify the uncertainty in your model parameters.
One way to do this is multiple imputation:

formulate a probabilistic model for the missing data
simulate missing data from that model
complete your task as if no data were missing
repeat this many times and combine the resulting estimates via Rubin's Formulas (slide 7).

This page contains tons of information on missing data and multiple imputation. But, if you just want a point estimate, keep reading.
Specific models for missing data
In this situation, I might try a rank $r$ regression model
$$E[M] = RL$$,
where $M$ is $[X|Y]$ ($n$ rows, $p$ columns), $R$ is an unknown $n$ by $r$ matrix, $L$ is an unknown $r$ by $p$ matrix, and $E[]$ is the expectation operator. (Substitute in your preferred likelihood function or data transform.) You can fit this model despite the missing entries: using a simple squared loss as an example, minimize $$\sum_{i,j \in \Omega} (M_{ij} - \sum_k R_{ik}L_{kj})$$ (where $\Omega$ is the set of observed entries). You can alternate between updating $R$ and updating $L$, so that each update is just a regression problem, and you'll probably do well.
Rather than low rank, another option is to use a flexible distribution based on a sparse inverse covariance matrix. (Why inverse covariance? It's the Markov random field representation.) This solution seems to accommodate all of your needs at once, so I linked to it in the tl;dr.
Do due diligence
Finally, please do not fit complex models to your data without first visualizing it and exploring it. I recommend scrutinizing all of the pairwise relationships by plotting them somehow (scatterplots for continuous, contingency tables for categorical, and side-by-side boxplots for mixed). This might reveal outliers or physically implausible trends that bear investigation. It might reveal that my answer is off track or requires modification: perhaps you'll end up with a couple of protein clusters that behave very differently, so you decide to model them each separately. Perhaps the data are just really ugly, and there's nothing to see other than "super acidic and salty things are unstable." I'm interested to know what you find.
Comment: Alright I don't feel like I'm able to identify outright that this is the solution to the problem, but it's such a good quality answer that it deserves the bounty