MySQL performance is very bad

Question: PFB a sql query I am trying to run. The final output returns around 60k rows, but it takes close to 5 hours to run. There is no problem with the connections and stuff like that and I feel that my query needs to be optimized heavily. Can somebody please point me in the right direction?
SELECT
    rapidview.name AS RapidView,
    CASE
        WHEN linktype.LINKNAME ="jira_subtask_link"
        THEN sprintdest.name
        ELSE sprint.name
    END    AS Sprint,
    j.pkey AS CaseKey,
    -- Sub task arent assigned sprint details, they are directly pulled from parent task, so that
    -- logic is implemented here for pulling all sprint related info
    CASE
        WHEN linktype.LINKNAME ="jira_subtask_link"
        THEN FROM_UNIXTIME(sprintdest.start_date/1000)
        ELSE FROM_UNIXTIME(sprint.start_date/1000)
    END AS SprintStartDate,
    CASE
        WHEN linktype.LINKNAME ="jira_subtask_link"
        THEN FROM_UNIXTIME(sprintdest.END_DATE/1000)
        ELSE FROM_UNIXTIME(sprint.END_DATE/1000)
    END                     AS SprintEndDate,
    StoryPoints.numbervalue AS StoryPoint,
    c.cname                 AS Component,
    it.pname                AS Type,
    p.pname                 AS Project,
    iss.pname               AS Status,
    dest.pkey               AS linkedissue,
    dest.id                 AS destid,
    dest.created            AS linkedissuecreated,
    (cglinkedissue.created) AS LinkedIssueClosedDate,
    linktype.LINKNAME       AS LinkType,
    cfoowner.customvalue    AS Owner,
    j.created               AS Created,
    cg.created              AS ClosedDate,
    CASE
        WHEN linktype.LINKNAME ="jira_subtask_link"
        THEN (
                CASE
                    WHEN sprintdest.started=true
                    AND sprintdest.closed=false
                    THEN "Current Sprint"
                    WHEN sprintdest.started=true
                    AND sprintdest.closed=true
                    THEN "Completed Sprint"
                    WHEN sprintdest.started=false
                    AND sprintdest.closed=false
                    THEN "Future Sprint"
                END)
        ELSE (
                CASE
                    WHEN sprint.started=true
                    AND sprint.closed=false
                    THEN "Current Sprint"
                    WHEN sprint.started=true
                    AND sprint.closed=true
                    THEN "Completed Sprint"
                    WHEN sprint.started=false
                    AND sprint.closed=false
                    THEN "Future Sprint"
                END)
    END                         AS SprintStatus,
    j.TIMEORIGINALESTIMATE/3600 AS EstimatedTime,
    j.TIMEESTIMATE/3600         AS RemainingTime,
    j.TIMESPENT/3600            AS LoggedHours ,
    cg.id                       AS CGID,
    ci.groupid                  AS cigroupid,
    ci.field                    AS CIFIELD,
    ci.newstring                AS NEWSTRING
    -- DevLead.stringvalue as DevLead,
    -- PMLead.stringvalue as PMLead,
    -- QaLead.stringvalue as QALead,
    -- DevLeadName.display_name as DevleadDisplayName,
    -- PMLeadName.display_name as PMLeadDisplayName,
    -- QALeadName.display_name as QALeadDisplayName
FROM
    jiraissue j
LEFT JOIN
    customfieldvalue cfv
ON
    cfv.issue=j.id
AND cfv.customfield=11002
LEFT JOIN
    AO_60DB71_SPRINT sprint
ON
    sprint.id=cfv.stringvalue
LEFT JOIN
    AO_60DB71_RAPIDVIEW rapidview
ON
    sprint.RAPID_VIEW_ID=rapidview.id
LEFT JOIN
    nodeassociation na
ON
    j.id=na.source_node_id
AND na.association_type = ('IssueComponent')
LEFT JOIN
    component c
ON
    na.sink_node_id=c.id
LEFT JOIN
    customfieldvalue StoryPoints
ON
    j.id=StoryPoints.issue
AND StoryPoints.customfield=10572
    /*
    LEFT JOIN
    customfieldvalue PMLead
    ON
    j.id=PMLead.issue
    AND PMLead.customfield=10382
    LEFT JOIN
    customfieldvalue DevLead
    ON
    j.id=DevLead.issue
    AND StoryPoints.customfield=10380
    LEFT JOIN
    customfieldvalue QaLead
    ON
    j.id=QaLead.issue
    AND QaLead.customfield=10381
    left join cwd_user DevLeadName
    on DevLead.stringvalue=DevLeadName.user_name
    left join cwd_user PMLeadName
    on PMLead.stringvalue=PMLeadName.user_name
    left join cwd_user QALeadName
    on QaLead.stringvalue=QALeadName.user_name
    */
LEFT JOIN
    issuetype it -- To pull in issuetype
ON
    j.issuetype=it.id
LEFT JOIN
    project p -- To pull in project
ON
    j.project=p.id
LEFT JOIN
    issuestatus iss -- To pull in Case Status
ON
    j.issuestatus=iss.id
LEFT JOIN
    issuelink il -- To identify linked cases
ON
    j.id=il.destination
LEFT JOIN
    issuelinktype linktype
ON
    il.linktype=linktype.id
LEFT JOIN
    jiraissue dest -- To idenfity component for the the linked case
ON
    dest.id=il.source
LEFT JOIN
    customfieldvalue owner -- To pull in customfields
ON
    j.id=owner.issue
AND owner.customfield=10310
LEFT JOIN
    customfieldoption cfoowner -- To pull in customfields
ON
    cfoowner.id=owner.stringvalue
LEFT JOIN
    changegroup cg -- To pull in case history to identify status changes
ON
    j.id=cg.issueid
LEFT JOIN
    changeitem ci
ON
    cg.id=ci.groupid
AND ci.field='status'
AND ci.newstring LIKE '%Closed%'
LEFT JOIN
    changegroup cglinkedissue -- To pull in case history to identify status changes
ON
    dest.id=cglinkedissue.issueid
LEFT JOIN
    changeitem cilinkedissue
ON
    cilinkedissue.groupid=cglinkedissue.id
AND cilinkedissue.field='status'
AND cilinkedissue.newstring LIKE '%Closed%'
LEFT JOIN
    customfieldvalue cfvdest
ON
    cfvdest.issue=dest.id
AND cfvdest.customfield=11002
LEFT JOIN
    AO_60DB71_SPRINT sprintdest
ON
    sprintdest.id=cfvdest.stringvalue
    --  year( FROM_UNIXTIME(sprint.END_DATE/1000) /1000)>=2015
    --  or year( FROM_UNIXTIME(sprintdest.END_DATE/1000) /1000)>=2015
    -- where
    -- j.pkey='CLQ-41441'
   group by
   j.id,
   c.id,il.id,sprint.id

Execution Plan
id  select_type table   type    possible_keys   key key_len ref rows    Extra
1   SIMPLE  j   ALL (null)  (null)  (null)  (null)  891945  (null)
1   SIMPLE  cfv ref cfvalue_issue   cfvalue_issue   18  jira_rnd_p.j.ID,const   1   (null)
1   SIMPLE  sprint  eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.cfv.STRINGVALUE  1   Using where
1   SIMPLE  rapidview   eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.sprint.RAPID_VIEW_ID 1   (null)
1   SIMPLE  na  ref PRIMARY,node_source PRIMARY 8   jira_rnd_p.j.ID 1   Using where; Using index
1   SIMPLE  c   eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.na.SINK_NODE_ID  1   (null)
1   SIMPLE  StoryPoints ref cfvalue_issue   cfvalue_issue   18  jira_rnd_p.j.ID,const   1   (null)
1   SIMPLE  it  eq_ref  PRIMARY PRIMARY 182 jira_rnd_p.j.issuetype  1   Using where
1   SIMPLE  p   eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.j.PROJECT    1   (null)
1   SIMPLE  iss eq_ref  PRIMARY PRIMARY 182 jira_rnd_p.j.issuestatus    1   Using where
1   SIMPLE  il  ref issuelink_dest  issuelink_dest  9   jira_rnd_p.j.ID 1   (null)
1   SIMPLE  linktype    eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.il.LINKTYPE  1   (null)
1   SIMPLE  dest    eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.il.SOURCE    1   (null)
1   SIMPLE  owner   ref cfvalue_issue   cfvalue_issue   18  jira_rnd_p.j.ID,const   1   (null)
1   SIMPLE  cfoowner    eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.owner.STRINGVALUE    1   Using where
1   SIMPLE  cg  ref chggroup_issue  chggroup_issue  9   jira_rnd_p.j.ID 4   (null)
1   SIMPLE  ci  ref chgitem_chggrp,chgitem_field    chgitem_chggrp  9   jira_rnd_p.cg.ID    1   Using where
1   SIMPLE  cglinkedissue   ref chggroup_issue  chggroup_issue  9   jira_rnd_p.dest.ID  4   (null)
1   SIMPLE  cilinkedissue   ref chgitem_chggrp,chgitem_field    chgitem_chggrp  9   jira_rnd_p.cglinkedissue.ID 1   Using where
1   SIMPLE  cfvdest ref cfvalue_issue   cfvalue_issue   18  jira_rnd_p.dest.ID,const    1   (null)
1   SIMPLE  sprintdest  eq_ref  PRIMARY PRIMARY 8   jira_rnd_p.cfvdest.STRINGVALUE  1   Using where

Comment: Well, that's a lot of joins. Look at execution plan, see where an index might help.
Comment: Pardon me for formatting, because I am new to stackoverflow for posting questions !
Comment: Hey Thilo, Have pasted the execution plan. Can you help as I am new to SQL tuning
Answer: Try EXPLAIN.
Pay attention to possible_keys, key, rows.
Maybe you can post the EXPLAIN result, and we can see what to do.
Comment: Thanks Billy, I have added the explain result.
Answer: Do you need LEFT?  That is, are all those other tables optional?
If you can get rid of LEFT in certain cases, you can avoid scanning all 891K rows of j.
Are you only interested in "closed" items?  If so, the query does not limit itself to them, again because of LEFT.
I would start by removing LEFT wherever practical.  Then move the AND clauses that are not really part of the JOIN to a WHERE on the end.  This might allow the query to filter stuff sooner, rather than lugging 891K (or more) rows (including lots of NULLs) around before getting to the GROUP BY.

MySQL performance is very bad


How to interpret results on different t-tests for the same samples?

Question: I am performing a t-test on the same couple of vectors in R. The first vector is the list of daily returns on a stock. The second vector is the list of daily log - returns on the same stock. Now, I understand the assumptions why we do a paired t-test as opposed to a Welch t-test (the default t-test setting in R). These are following results:

Welch t-test
t.test(dailyreturns, logReturn)
Welch Two Sample t-test
data:  dailyreturns and logReturn
t = 0.10813, df = 2350, p-value = 0.9139
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.0006801156  0.0007595007
sample estimates:
mean of x    mean of y
0.0005027479 0.0004630553
Paired t-test
t.test(dailyreturns, logReturn, paired = TRUE)
Paired t-test
data:  dailyreturns and logReturn
t = 15.866, df = 1175, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
3.478409e-05 4.460108e-05
sample estimates:
mean of the differences
<code>      3.969258e-05
</code>

My question is given that the alternate hypothesis in both tests is the same and as the author mentions (see answer to Problem 14), while the assumptions in the Welch t-test is incorrect, the results are same. However, given the vastly p-values in both test, the null-hypothesis in one test will be rejected in the Welch t-test but won't be rejected in the paired t-test! And the graph in Problem 12 (from the same data set) shows that there isn't much difference in daily returns and daily log-returns. Should it not mean that the p-value should show insignificant results and therefore we will fail to reject the null hypothesis that the true difference in means is equal to 0? (Here, I am assuming true means ~ 0 and not EXACTLY 0 - am I right in this assumption?).
Comment: "The first vector is the list of daily returns on a stock. The second vector is the list of daily log - returns on the same stock.". Let first vector is X, is the second vector = log(X)?
Answer: Ok, so comparing y and log(y) in a t test essentially amounts to comparing one sample against itself. Instead of taking two measurements, they've transformed one measurement and called it different. It would make no sense in real life, but i guess it makes sense for the purposes of an exercise.
The Welch T test is a special kind of two sample T test that does not assume equality of variance between the groups. However, it is still a two sample T test. Hence, it does not match the current situation.
The paired T test is essentially a one sample T test. Both "samples'" standard deviation are the same, because they are actually the same sample/group. So you don't have to pool their standard deviations to get standard error, so they're going to have smaller standard errors. A T test is always the difference of means divided by standard error. So if you have a bigger standard error on the bottom, you'll have a smaller T test.
This is exactly what happened in this case. The numerator is the same in both tests. But the denominator is much larger for the Welch T test. So it resulted in the Welch T test being much smaller and hence non significant. You can substitute toy numbers into the tests' equations below. The standard error of the Welch is always bigger.
Paired T test
$$\frac{\bar{X_1}-\bar{X_2}}{\frac{S}{\sqrt{n}}}$$
Welch T test
$$\frac{\bar{X_1}-\bar{X_2}}{\sqrt{\frac{S_1^2}{n}+\frac{S_2^2}{n}}}$$
Does that mean you always use the paired T test? Obviously NOT. You can't just reduce the standard error because you want to. It has to come from the data, and if the groups aren't the same, then you have to pool their standard deviations to get standard error.
In your case, this situation calls for going with the paired T test and hence, rejecting your null hypothesis. But I stress this is a bad example. A better one is if you tested the same people across two conditions and recorded their scores and wanted to see if the two conditions had different means. Or recording people's weights before and after dieting to see if, as a group, they've lost weight.


How to derive the MLE of a Gaussian mixture distribution

Question: In my self-study, I consider a Gaussian mixture distribution:
$$p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0)$$
where $p(k=1)+p(k=0)=\pi_1+\pi_0=1$. I am now asked to do three things:

Write down the likelihood of the observations as a product over $n$
observations
Write down the likelihood as a product over the likelihoods for ${K_0}$
and $K_1$, where $K$ is the set of indices for $k=1$ and $k=0$, respectively.
Compute the log-likelihood and maximize for $\mu_0$ and $\sigma_0$.

I am not really sure what I am asked to do. I believe the likelihood is given by:
$$p(x|\pi_0, \pi_1, \mu_0, \mu_1, \sigma_0^2, \sigma_1^2) = \prod_{i=1}^n \bigg[ \pi_1 N(x_i|\mu_1,\sigma^2_1) + \pi_0 N(x_i|\mu_0,\sigma^2_0) \bigg]$$
So the log-likelihood is the sum of the logarithm of the sum in the parentheses. This seems correct. However no closed form solution exists of the derivative nor maximizer.
First, I am not sure which of no. 1 or 2 my likelihood expression solves. I think no. 1 but then, second, I am not sure what I am asked to do in no. 2. I suppose the solution to 2 is easier to maximize in 3. Third, it seems there are two different expressions for the likelihood then, but shouldn't there be just one?
Note: I was thinking for no. 2 along the lines of
$$p(x|\pi_0, \pi_1, \mu_0, \mu_1, \sigma_0^2, \sigma_1^2) = \prod_{i=1}^n \bigg[(\pi_1N(x_i|\mu_1,\sigma^2_1))^{k_i} (\pi_0N(x_i|\mu_0,\sigma^2_0))^{1-k_i} \bigg]$$
but got stuck.
Answer: This is the proper start but I wonder at the wording of the exercise. I would have asked the following:

Write the likelihood of the sample $(x_1,\ldots,x_n)$ when the $X_i$'s are iid from$$p(x)= \mathbb{P}(K=1) N(x|\mu_1,\sigma^2_1) +
 \mathbb{P}(K=0) N(x|\mu_0,\sigma^2_0)\qquad\qquad(1)$$and conclude at the lack of closed-form expression for the maximum likelihood estimator.
Introducing the latent variables $K_i$ associated with the component of each $X_i$, namely$$\mathbb{P}(K_i=1)=\pi_1=1-\mathbb{P}(K=0)$$and$$X_i|K_i=k\sim
 N(x|\mu_k,\sigma^2_k)$$show that the marginal distribution of $X_i$ is
  indeed (1).
Give the density of the pair $(X_i,K_i)$ and deduce the density of the completed sample $((x_1,k_x),\ldots,(x_n,k_n))$, acting as if the
  $k_i$'s were also observed. We will call this density the completed
  likelihood.
Derive the maximum likelihood estimator of the parameter $(\pi_0,\mu_0,\mu_1,\sigma_0,\sigma_1)$ based on the completed sample
  $((x_1,k_x),\ldots,(x_n,k_n))$.
Comment: My confusion is about the wording, not the problem: This is rather standard stuff about mixtures, leading towards the EM algorithm. For instance, this is how we proceed in [our book](http://www.amazon.com/gp/product/1441915753?ie=UTF8&tag=chrprobboo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1441915753) (Chapter 5). In a mixture model, the $K_i$'s are not observed, otherwise it would not be a mixture but an aggregate of two normal samples.
Comment: Your (justified) confusion emerged because the exercise does not state explicitly that $k_i$ are observed. This is implicit in the formulation in no. 2 about the sets. In particular the exercise says here "we know that $n \in  {K_0}$ are the indices for the disease free patients and $n \in K_1$ are the indices for the patients with the disease (i.e. $K_0$ and $K_1$ are non-intersecting sets of indices from 1 to N)". I believe this is supposed to tell me that $k$ is observed, but I am still not sure.
Comment: Please see my suggested solution below. Comments welcome.
Answer: I continued working on this exercise and came up with a solution. I'd be glad about comments.
Let $\theta=[\pi_0,\pi_1,\mu_0,\mu_1,\sigma_0^2,\sigma_1^2]$

The likelihood over N observations is given by:
$$ P(x|\theta) = \prod_{i=1}^n \bigg[\pi_0 N(x_i|\mu_0,\sigma_0^2)+\pi_1 N(x_i|\mu_1,\sigma_1^2) \bigg]$$
The likelihood written as product over sets $K_0$ and $K_1$ is given by

$$ P(x|\theta) = \prod_{i=1}^n \bigg[ (\pi_0 N(x_i|\mu_0,\sigma_0^2))^{1-k_i}(\pi_1 N(x_i|\mu_1,\sigma_1^2))^{k_i} \bigg]$$

The log-likelihood is given by

$$ \ln P(x|\theta) = \sum_{i=1}^n \bigg[ (1-k_i) (\ln \pi_0 + \ln N(x_i|\mu_0,\sigma_0^2))+k_i(\ln \pi_1 + \ln N(x_i|\mu_1,\sigma_1^2)) \bigg] $$
Consequently we can find the MLE for $\mu_0$ and $\sigma^2_0$ in a nearly standard way finding:
$$\hat{\mu}_0 = \frac{1}{\sum_{i_1}^n (1-k_i)} \sum_{i_1}^n (1-k_i) x_i$$
$$\hat{\sigma}^2_0 = \frac{1}{\sum_{i_1}^n (1-k_i)} \sum_{i_1}^n (1-k_i)(x_i - \hat{\mu}_0)^2$$
Comment: The answer is correct and constitutes the penultimate step to the derivation of the EM algorithm.
Comment: Out of curiosity, why is the likelihood here taken to be simply the multiplication of the individual Gaussian functions? I can understand the probability density function is given by a Gaussian, and if these variables are normally distributed and independent, then the joint density function, i.e. $p(x_1,x_2)$, is equal to $p(x_1)p(x_2)$. But why is the likelihood equal to the joint density function here? Shouldn't it be multiplied by a prior (which I suppose may simply be uniform here) to be equivalent?


ABC test/ hypothesis testing with multiple comparisons

Question: I need to do a hypothesis test with three versions (one control version and two test versions).  From what I understand I can just run a t-test (or z-test) for each test version vs control, and then use some kind of correction (such a bonferroni correction) and see if one is significant (and if they both are see which one has the highest lift).
Assuming this is right, how do I run a sample size calculation for this?  Can I just run the sample size calculation for both versions of the test (control vs test a and control vs test b) and then aggregate that way.  I guess I would take the bonferroni correction into account when determining significance level for the sample size calculation.  Does this seem right?
Answer: Maybe it is best to do an ANOVA to see if there are any significant
differences among the three levels of the factor. If so, then
do ad hoc tests along with some method (such as Bonferroni) to
avoid 'false discovery'.
Unless I had some previous experience with similar data to feel
confident that the three levels all have the same variability, I'd use
oneway.test in R, which does not assume equal variances.
Example:
ctrl = rnorm(20, 50, 2)
trta = rnorm(20, 51, 2)
trtb = rnorm(20, 52, 2.2)
x = c(ctrl, trta, trtb)
g = rep(1:3, each=20)
boxplot(x~g, col="skyblue2")

oneway.test(x~g)

    One-way analysis of means
    (not assuming equal variances)

data:  x and g
F = 4.1362, num df = 2.000, denom df = 37.975, p-value = 0.02371

So there are significant differences at the 5% level. It seems obvious that the control and treatment B must differ. Then we could do ad hoc
Welch t tests at the 2% level to see if we can distinguish between treatments A and B, and between control and treatment A.
In my fictitious example, we cannot say treatment a is significantly
different from the other two levels of the factor.
t.test(ctrl, trta)$p.val
[1] 0.06865211           # not signif
t.test(trta, trtb)$p.val
[1] 0.3030164            # not signif
t.test(ctrl, trtb)$p.val
[1] 0.007009848          # signif as surmised

There are procedures in R, other statistical software, and online
to suggest sample sizes for the three levels, given the size of
difference $\Delta$ we want to detect, the (common) SD $\sigma$ of the three groups, and the desired power. Most of these require equal
variability in the levels and equal sample sizes. However, it is
possible to relatively simple simulations in R to check whether
particular sample sizes give adequate power for given $\Delta$ and $\sigma_i.$
In the fictitious situation above we might use $\sigma_1 = \sigma_2 = 2, \sigma_3 = 2.2,$ and $\Delta = 1.$ We found that $n = 20$ was large enough for
the main ANOVA, but not large enough for the ad hoc tests (where smaller values of $\Delta$ may be required). So, as you suggest, we should focus on the power of the t tests.
By trial and error, it is easy to see that $n=80$ suffices to get power about 74% for a
Welch t test with $\Delta=1, \sigma_1=2, \sigma_2-2.2,$ and significance level $\alpha = 2\%.$ (And $n-100$ for power about 84%.)
pv = replicate(10^5, t.test(rnorm(80,50,2),rnorm(80,51,2.2))$p.val)
mean(pv <= .02)
[1]  0.74339

pv = replicate(10^5, t.test(rnorm(100,50,2),rnorm(100,51,2.2))$p.val)
mean(pv <= .02)
[1] 0.84551

The search can be hurried along by (a) using $10^4$ replications early on and (b) using standard formulas involving noncentral t distributions for pooled t tests which have slightly better power than do Welch t tests.


Difference between keys and scales?

Question: I am a Tabla (a North Indian percussion instrument) player and I am trying to learn the western music theory. My study was going well until I stumbled upon keys and scales. Now though I know what scales are. They are just a bunch of notes the composer decided to use in his music. But the problem is that sometimes the word 'scales' is interchangeably used with 'keys'. So I searched about keys and haven't found much about them except the same fact that people sometimes use 'keys' instead of 'scales'. Though I know what 'Key Signatures' are. Also I heard about "playing different scale but staying in the same key". So from that sentence and other research I am pretty sure keys and scales are different. But how are they different? Does key mean key signature? Or are key signatures part of key? And what's the difference between keys and scales?
I've recently learned about modes hoping that it would clarify my doubt but it didn't. I want an answer which has a perfect base such that I can go further study music theory without ever doubting about keys and scales again.
Answer:
[A scale is] just a bunch of notes the composer decided to use in his
  music.

This is correct, or at least it's a valid way to look at it. In western music, the key chosen for a piece implies both a scale and a tonal center, but alternate scales may be used without changing key, and in some ways and some forms of music, the tonal center may also change without it being considered a change of key.
If someone tells you a piece is in the key of A minor, then you can assume you will use the A minor scale for least the beginning, end, and a good portion of the piece. It won't always be just like that, but it's the best assumption. There may be some use of the A harmonic minor scale and/or the A natural minor scale in western music without changing the key of the piece, which can still be in A minor.
To be clear, in Western music it is not uncommon to use multiple scales in a single piece, even if that piece stays in the same key the entire time. In Western classical music, it is very common for a piece to have more than one key, with one or more scale being used in each key. In general, the number of scales used in a piece is equal to or greater than the number of keys used.
So, each key comes with one scale attached to it, with other scales that go along with that key being optional.
A mode is a special kind of scale. You can think of a mode as just being a scale, and we just use the word "mode" for some scales and "scale" for others. Confusingly, some patterns of notes have two different names, one "scale" name and one "mode" name. As you work to understand scales and keys, it's all right to consider "mode" to be another word for "scale". As you understand more, you'll see why there is a separate word for it, but at the beginning just consider them to be the same thing.
Answer: You determination that keys and scales are not the same is correct.  They are not the same but in general - most scales are based on a particular key.  Let me try to explain keys and scales in simple terms without getting into a great deal of music theory.
Keys determine the group of notes that the composer has to choose from when composing the music.  In Western Music there are twelve notes that exist between octaves.  They are A (A#/Bb) B C (C#/Db) D (D#/Eb) E F (F#/Gb) G (G#/Ab).  The letters in the parenthesis are the same note in practical application because they are played by the same key on piano or same string/fret combination on guitar and in the same place on all instruments that have a fixed tuning system.  So there are keys on the piano which would play a note which would be referred to as either an A sharp (A#) or B flat (Bb) - depending on which key you are in.
Of the twelve notes, only seven exist in each key.  A key can be major or minor.  The key signature tells which notes on the musical staff are to be considered (and played) sharp or flat.   The key of C major has no sharps or flats thus the key signature shows no sharps or flats.  The seven notes in the key of C major are C D E F G A and B.
A scale is an ascending series of notes (each one higher than the preceding note) that span an octave.  Scales are typically based on a key.  So the C major scale would consist of the the notes C D E F G A and B.  So to play a C major scale on an instrument, you would start on C and play C D E F G A and B and end the scale by playing C one octave higher.
The following types of scales contain 7 notes each: Major, Natural Minor, Harmonic Minor, and Melodic Minor.  The pattern of intervals between notes (either whole or half step/semitone) is what determines the type scale. These seven note scales are called diatonic scales.
In a major diatonic (7 note) scale the pattern of intervals is whole whole half  whole whole whole half.  Whole is two semitones and half is one.
So there is one full (whole) step between the first two notes of a diatonic major scale and one full step between the 2nd and 3rd notes but only a half step between the 3rd and 4th notes. In a natural minor scale the intervals are whole half whole whole half whole whole.
Scales can also consist of 5 notes.  5 note scales are known as pentatonic scales and also span an octave.  But to span an octave with only 5 notes, larger steps or gaps between notes are required.  Just as with diatonic scales, there are different types of pentatonic scales (ie major and minor) and again the pattern of intervals between notes is determined by the type scale (or you could say the pattern is what determines the scale type).
Other variations of scales can be invented by the musician to add a particular flavor to their music.
In summary, scales are played in ascending note order and are usually based on a particular key (with the exception of the chromatic scale which is all twelve possible notes played in ascending order).  The key determines which notes are available to choose from for a particular scale.  In general, scales will contain notes derived from the key it is based on.  So a major diatonic scale will contain the same notes as the key it is derived from.  C major scale shown above contains all the notes in the key of C major - played in ascending order.
A major pentatonic scale would also use notes from within the key it is derived from - but only 5 of the seven notes would be used.
Think of a scale as climbing or scaling a ladder.  The key and type of scale determines how far apart the steps on the ladder are from one another.  Pentatonic, or diatonic determined how many steps on the ladder.  Each ladder is the same height (an octave) but the spacing for a 5 step ladder is going to be different that for a 7 step ladder.
Hope that helps.
Comment: If I'm not wrong, melodic minor is heptatonic, but not diatonic.
Answer: A very simplistic way to look at it is this:
A scale is a defined set of ordered pitches.
A key is the use of a scale in actual music.
To say that a piece is in the Key of A, would mean that usually most of the notes in the piece would follow the tonal rules of the A Major scale.
In Indian music there are two styles that I am aware of, Carnatic and Hindustani. I have forgotten which is which, but in one of them the scale is set in unique ways for each piece where a unique scale is associated with a single raga. This is a different approach than the "western" tradition where (by comparison) there is a limited set of predefined scales that are used in almost every piece.
Answer: In the Common Practice Period (CPP) descriptions of Western Harmony (still used in much if not most music, Latin, pop, jazz, movie themes, country, rock, blues, etc.) a "key" consists of a primary note (termed the "tonic") and a set of chords commonly used to highlight this note (and its chords). A scale is a set of notes with linear order. In common practice, there is a primary scale associated with each key or vice versa (the theory is derived from the music; the theory is descriptive and is supposed to explain the way the music has been constructed.)
In CPP harmony, there are two common types of keys by pattern. Each key may be moved to another pitch. There are major keys with a major scale which is 8 notes with a particular pattern. A scale repeats at an "octave" (the historical name for a tone at double the frequency of another note). In (for example) the key of C-major the notes are C-D-E-F-G-A-B-C (the second C is double the frequency of the first.) The spacing in terms of half steps (approximately 18/17) and whole steps which are two half steps is W-W-H-W-W-W-H. I'm using C-major for notational convenience.
There are chords associated with C-major. The "tonic" chord is C-E-G. This is the "home" chord and composers write music which sets up the expectation that this chord will be used to end a piece (or section thereof). There are many chords with different usage (or function) associated with the key of C-major. For example, the "dominant" chord in C-major is G-B-D often with a fourth note G-B-D-F added. The "feeling" (I don't know a good word for "auditory feeling") of being in C-major is most often indicated by music with chords F-major (F-A-C) followed by G-major followed by C-major. There's much more. There are minor chords like a D-minor chord D-F-A (the pattern is slightly different from a major chord) and some pieces can end with D-minor followed by G-major followed by C-major.
A minor key has a minor chord as its home chord. Minor keys are quite a bit more complex than major keys. The scale associated with a minor key (I'll use A-minor for notational convenience) is A-B-C-D-E-F-G-A which is a circular permutation of the C-major scale. However, there is another and more important difference. Scale steps 6 and 7 (notes F and G here) are mutable. Each may occur with two different notes. (Lowering a note is signaled by an appended b and raised by an appended # called "flat" and "sharp" respectively.) In A major, the notes F and G may be replaced by F# or G#. There are a bunch of common patterns that are described in various books.
There's much more, especially in the idea of writing for simultaneous melodies that sound good both individually or together (counterpoint). There are also structural conventions for various types of pieces. It's big.
For more (and better I'd guess), check out various websites that do harmony. There are some good (if old because of copyright restrictions) books available. I like Frank Shephard's book on harmony as well as Francis York's books on harmony and counterpoint. They are old but serviceable and free.
"Music Theory for Ordinary People" and "Open Music Theory" are two good websites. There are lots of others.
Answer: A key usually refers to a collection of 7 notes that have a hierarchical ordering and specific functions; for example, the most important note in any key is the "tonic", which is the note where most pieces end, and gives the listener the feeling of repose. Other important notes in any key are the dominant and subdominant. Keys also have an implied harmonic structure - they allow for building chords (basically 2, 3 or more notes, played together). The chords are also called "tonic" or "dominant" etc depending on the "root" note that is used to build the chord; thus you have the tonic chord and the dominant chord, which are the most important of any key.
Another important concept is "mode", which effectively modifies some of the notes in the key by raising or lowering them. For example, a C major key contains notes CDEFGAB, while a C minor CDEbFGAbBb
A scale is basically an ordering of notes by steps, starting and ending with the tonic, going up or down. Thus, in the key of G major (which contains an F#), the G major scale is GABCDEF#G.
Scales are usually technical exercises intended to be played or sung. A key is more of an abstract concept (it's the "raw material" to write a tune, or to build a scale).
You CAN play different scales with the same key signature. But that's a different question that would be too long to answer here.
Comment: So what if I want to make my own music? How would I use both the things?
Comment: Composers usually pick a key to write a song, and use the chords commonly present in that key. Scales are more of a technical exercise - they are used to develop accuracy and fluency when playing an instrument or singing. Since scales are basically an arrangment of all the notes of a key, playing scales also "educates" the ear of the student to hear and recognize the key, and to improvise melodies using the correct notes of the key.
Answer: Key (location in sound) - Root Note
Mode (flavor or color) - Set of notes or interval pattern stacked on top of the root noted.
You can play Key of "C (Major)" (all white keys) without sharps or flats (black keys). But you're simultaneously kinda playing 7 modes (flavors).
All white keys!!!!!!!!!!
C to C is Major
A to A is minor
B to B is locrian
D to D is dorian
E to E is phrygian
F to F is lydian
G to G is mixolydian
.... they are all key of C. The interval jumping is taken care of by the skipped black keys.
So, for example, you want to play minor in any key. Look at the white keys from A to A and you get 2122122. Start on any white or black key and play that interval going to the right and you are in minor in the "key" that you started on.
Obvious follow up question: "Can you play black notes and still be in key of C?" Yes. Most definitely. Take Phrygian Dominant (1312122) and cycle it to get all 7 relative scales (jazz). Root any of them on C and you are in "Key of C." Depending on the mode, it might be easier to write it in a different key to be less cluttered. That's all.

Difference between keys and scales?


Bootstrap tooltip does not allow table rows to be clicked

Question: On my page, I have clickable table rows that open a specific link. This functionality has worked just as planned, that is, until I tried converting the traditional tooltips into Bootstrap ones. Now, I can't seem to get the clickable functionality back (even though the tooltips work normally).
HTML:

    1

        Attended seminar!

    1

JavaScript:

Comment: seems to be working fine - https://jsfiddle.net/vabii/svm9Ln1v/2/
Answer: Solved, for whatever reason I needed to remove the $(document).ready() part of the JavaScript code:

More details as to why this solved the issue would be appreciated!


Making Time Series/ Date DataFrame in pandas

Question: I have a file with date time column 'A' with several months. I have to restrict to August 2017, group by the weekday, aggregate by summing. Set the Weekday column to numbers from one to seven. Then set the column Weekday as the (row) index. Return the resulting DataFrame.
I tried below:
pd.date_range("2017-08-01", "2017-08-31",freq='D').to_series()

y=z.dt.weekday
y

This gives

2017-08-01    1
2017-08-02    2

2017-08-03    3...
 and so forth but I am not able to rename or change the index of this single column in my jupyter notebook. How can I go about  solving this exercise.Please help.
Comment: `y=z.dt.weekday.rename('new_name')`.
Comment: Why both date and number data not appearing in different columns so that I can name them both separately and also make the number column as index.
Comment: Cos you are using series and not dataframes.
Comment: Is there any feature similar to or like to_DataFrame()???
Answer: IIUC you'd like to have something like
import numpy as np
import pandas as pd
df = pd.date_range("2017-08-01", "2017-08-31",freq='D')\
       .to_frame(name="date")\
       .reset_index(drop=True)

df["n"] = np.random.randn(len(df))
df["dow"] = df["date"].dt.weekday

df.head()

        date         n  dow
0 2017-08-01  2.104356    1
1 2017-08-02  0.475884    2
2 2017-08-03 -0.849579    3
3 2017-08-04 -0.134266    4
4 2017-08-05 -1.322617    5

And then perform the following groupby
df.groupby("dow")["n"].sum().reset_index()

   dow         n
0    0 -1.579067
1    1  0.793178
2    2 -2.310629
3    3 -2.275956
4    4 -0.091897
5    5 -3.918192
6    6 -2.252314

** Update **
If you have problems with to_frame you could define df as following
df = pd.DataFrame({"date": pd.date_range("2017-08-01",
                                         "2017-08-31",
                                         freq='D')})

Comment: Thank you but the code throws an error-'DatetimeIndex' object has no attribute 'to_frame'
Comment: I guess that depends on pd.__version__ do you mind to share your version?
Comment: I am using Python 3.
Comment: I was asking about panda version. Anyway I updated the part that was giving you an error. Check it out.


Relationship between entropy and predictability

Question: The entropy of a random variable $X$ is defined as $\mathbb{E}(-\log(f(X)))$ (where $f$ is the pdf of X, https://en.wikipedia.org/wiki/Entropy_(information_theory)).
Is there any general relationship between the entropy of $X$ and the possibility to "more or less accurately" predict the value of $X$ using some constant $a$? That is, are there known general relationships between $\mathbb{E}(\log(X))$ and measures such as
$$ \min_a \mathbb{E}[|X-a|] \tag 1$$
$$ \min_a \mathbb{E}[(X-a)^2] \tag 2$$
$$ \min_a \mathbb{E}[I(X=a)] \tag 3$$
?
For a $Be(p)$, for example, it seems that the two kinds of metrics (entropy on the one hand, and either of (1), (2), or (3) on the other) move jointly with $p$ (i.e., if $Be(p)$ has higher entropy than $Be(p')$, then $Be(p)$ is also less accurately predictable --- higher minimum expected error --- than $Be(p')$ in terms of (1), (2), or (3)).
Is something like that true more generally? For example, is it true in general that an increase in entropy implies a decrease in predictability as measured by (1), (2), or (3)? If it's not true in general, is true for some class of random variables larger than the class of $Be(p)$?
Comment: *Caeteris paribus,* [the entropy gives some information about variance,](https://stats.stackexchange.com/a/415436/919) whence (*via* Chebyshev's Inequality, among others) it is related to the standard error of the mean.  The initial *c.p.* condition is crucial, though, because as a general proposition entropy tells you nothing about predictability.  You have to constrain the space of possible distributions.
Comment: See the first two words of my comment.
Comment: I guess for (1), since $\min_a \mathbb{E}[(X-a)^2] = \mathbb{E}[(X-\mathbb{E}(X))^2] = Var(X)$, the question boils down to whether RV's with lower entropy systematically have lower variance. Intuitively, it feels like there should be counter-examples to this, though I still need to take the time to think about it and construct such a potential counter-example if one exists.
Comment: In part it depends on the definition of entropy being used. Shannon's entropy is borrowed from physics, where entropy is a measure of disorder. A cloud has higher entropy than an ice cube, since a cloud allows for many more ways to arrange water molecules than a cube's crystalline structure does. In an analogous way, a random message has a high Shannon entropy — there are so many possibilities for how its information can be arranged — whereas one that obeys a strict pattern has low entropy. Given this definition, the lower the entropy, the more predictable are the events.
Comment: @whuber: Thanks for the comment. What do you mean by c.p.? Would you be able to elaborate a little on "You have to constrain the space of possible distributions"? Are there famous classes of distributions for which entropy is "more directly" related to some "standard" measure of predictability?
Answer: I think there are counter-examples for all three measures. That is, for all three measures, there are cases where the entropy remains unchanged but the measure changes. Below are my proposed counter-examples.
Mean absolute distance (1)
Note that
$$\min_a \mathbb{E}[|X-a|] = \mathbb{E}[|X-\text{med}(X)|].$$
Let $f_Y(1) = f_Y(2) = f_Y(3) = 1/3$ and $f_Z(0) = f_Z(2) = f_Z(4) = 1/3$.
We have $\text{med}(Y) = \text{med}(Z) = 2$ and $H(Y) = H(Z) = -\log(1/3)$, though
$$ \mathbb{E}[|Z-\text{med}(Z)|] = 4/3 > 2/3 =  \mathbb{E}[|Y-\text{med}(Y)|].$$
Mean squared distance (2)
Note that
$$\min_a \mathbb{E}[(X-a)^2] = \mathbb{E}[(X-\mathbb{E}(X))^2] = Var(X).$$
Let $Y = Be(p)$ and $Z = 2*Be(p)$. Note that the two RVs have the same entropy:
$$ \begin{align} H(Y) && = && f(0) * [-\log(f(0))] && + && f(1) * [-\log(1-f(1))] \\ &&=&& p* [-\log(p)] && + && (1-p)*[-\log(1-p)]. \end{align}$$
$$ \begin{align} H(Z) && = && f(0) * [-\log(f(0))] && + && f(2) * [-\log(1-f(2))]\\ && =&& p* [-\log(p)] && +&&  (1-p)*[-\log(1-p)]. \end{align}$$
Of course, the two RVs have different variances with $Var(Z) = 4*Var(Y)$.
This seems to hold for any discrete RV, that is, if $Z = a + bY$, the two RVs have different variances ($Var(Z) = a^2 Var(Y)$), but have the same entropy.
As noted in the answer @whuber points at in the comments (How does entropy depend on location and scale?), this is not true for continuous distributions ("scaling a continuous variable [by $\sigma$] (which, for $\sigma \geq 1$ "stretches" or "smears" it out) increases its entropy by $\log(\sigma)$.").
Mean error rate (3)
Note that
$$\min_a \mathbb{E}[I(X=a)|] = \mathbb{E}[I(X=\text{mode}(X))].$$
Let $f_Y(1) = 1/8$, $f_Y(2) = 1/2$, $f_Y(3) = 3/8$ and $f_Z(1) = 1/4$, $f_Z(2) = 1/2$, $f_Z(3) = 1/4$.
We have $\mathbb{E}[I(Y=\text{mode}(Y))] = 1/2 = \mathbb{E}[I(Z=\text{mode}(Z))]$. However, $H(Y) \neq H(Z)$.


How to select a Node by it's atrribute name and get it's child node in C#?

Question: Here is my XML code(module.xml)
<module code="ECSE502">
  ECSE502
  <name>Algorithms and Data structures</name>
  <semester>1</semester>
  <prerequisites>none</prerequisites>
  <lslot>0</lslot>
  <tslot>1</tslot>
  <description>all about algorythms and data structers</description>
</module>
<module code="EIGA501">
  EIGA501
  <name>3D Grapgics I</name>
  <semester>1</semester>
  <prerequisites>none</prerequisites>
  <lslot>2</lslot>
  <tslot>3</tslot>
  <description>xxxxxxxxxxxxxxxxxxxxxx</description>
</module>

According to the above xml code, I need to get the  set by only giving ECSE502 as the input.
After selecting the required node I need to get its child node's values also(name, semester,etc.).
In the XML file there are 20 nodes. this is only the 1st 2 nodes.
I tried this so far
 XmlTextReader reader = new XmlTextReader("modules.xml");
            XmlDocument doc = new XmlDocument();
            XmlNode node = doc.ReadNode(reader);

        foreach (XmlNode chldNode in node.ChildNodes)
            Console.WriteLine(reader.Value);

Answer: You can use LINQ to XML, assume in here the code returns the list of anonymous objects, but you can explicitly define your own class if you want:
var xDoc = XDocument.Load("yourpathfile");
var result = xDoc.Descendants("module")
            .Where(x => (string) x.Element("code") == "ECSE502")
            .Select(x => new
                             {
                                 Name = (string)x.Element("name"),
                                 //.....
                             });

Comment: @Ravindu: try this way also :)
Answer: try this:
wrap your xml in a unique root node i.e
<modules>
   <module code...
       ..
   </module>
   <module code...
</modules>

and then
string text="ECSE502";

XmlDocument xml = new XmlDocument();
xml.Load("physical path to module.xml");

XmlNodeList xnList = xml.SelectNodes("modules/module[@code='"+text+"']");
foreach (XmlNode xn in xnList)
{
    string code = xn.SelectSingleNode("code").innerText;
    string name=  xn.SelectSingleNode("name").innerText;
    //and similarly find other inner nodes
}

Comment: try wrapping your modules in a root tag.i.e <module.. and see the update
Comment: not worked. and what does it do in line,
XmlNodeList xnList = xml.SelectNodes("/module[@code='"+text+"']");
Comment: it finds only those module node which have their code attribute as the text
Comment: yeh it's working. But all the data is in one line as,
ECSE502Algorithms and Data structures1none01
how can I divide them according to there nodes
Comment: please check this too
http://stackoverflow.com/questions/15700848/read-from-a-xml-file-to-an-array-node-by-node-in-c-sharp
Answer: Try it after creating a root node in your xml with below code
XDocument xdoc = XDocument.Load(@"D:\data\rvyas\Projects\Client\module.xml");

string code = "ECSE502";
    var result = xdoc.Descendants("module")
                .Where(x => (string)x.Element("code") == code)
                .Select(x => new
                {
                    Name = (string)x.Element("name"),
                    Code = (string)x.Element("code"),
                    semester = (string)x.Element("semester"),
                    prerequisites = (string)x.Element("prerequisites"),
                    lslot = (string)x.Element("lslot"),
                    tslot = (string)x.Element("tslot")

                }).ToList();

You XML should be like:
<root>
<module code="ECSE502">
  ECSE502
  <name>Algorithms and Data structures</name>
  <semester>1</semester>
  <prerequisites>none</prerequisites>
  <lslot>0</lslot>
  <tslot>1</tslot>
  <description>all about algorythms and data structers</description>
</module>
<module code="EIGA501">
  EIGA501
  <name>3D Grapgics I</name>
  <semester>1</semester>
  <prerequisites>none</prerequisites>
  <lslot>2</lslot>
  <tslot>3</tslot>
  <description>xxxxxxxxxxxxxxxxxxxxxx</description>
</module>
</root>


Como separar dados de uma coluna

Question:
Gostaria de separar os dados da primeira coluna em outras duas. Há um "." entre o nível de instrução e o setor. Tentei usar a função separate mas não deu certo.
Meu código:
pnad_2019_tratada %>% separate(interaction(VD3004, VD4010), c("instrucao", "setor"), ".")

Comment: Victor, evite usar imagens. Isso dificulta a reprodução, tente o comandou `dput()`
Answer: O comando está correto, porém para usar o . como "referência para um padrão de texto", é necessário colocar como \\.
Libraries
library(tidyverse)

Dados
df <-
  tibble(
    x = c("Sem instrução e menos de 1 ano de estudo.Agricultura",
          "Fundamental incompleto ou equivalente.Agricultura")
  )

Código
Com o comando separate você pode criar novas colunas (into) a partir de uma coluna (col) e um separador (sep).
df %>%
  separate(col = x, into = c("instrucao","setor"),sep = "\\.")

Resultado
# A tibble: 2 x 2
  instrucao                                setor
  <chr>                                    <chr>
1 Sem instrução e menos de 1 ano de estudo Agricultura
2 Fundamental incompleto ou equivalente    Agricultura

Como separar dados de uma coluna


Minimization of difference

Question: Given two lists $\{a_1,a_2, \ldots,a_n\}$ and $\{b_1,b_2, \ldots,b_n\}$ and you can rearrange elements in each list individually. You have to rearrange element in it in such a way value of S is minimized.
$$
S = \sum_{i=1}^n \,\left| a_i-b_i\right|
$$
What should be strategy of rearrangement of individual lists ?
Comment: Is there a question?
Comment: Try: http://en.wikipedia.org/wiki/Least_absolute_deviations#Solving_Methods
Comment: Problem statement modified.
Comment: Why wouldn't sorting both lists in increasing order work?
Comment: Will sorting both work ? I was not able to prove it formally.
Answer: Prove that sorting works for lists containing two elements: $\{a_1,a_2\}$ and $\{b_1,b_2\}$. Thus, you'll have proved that if $a_1\leq a_2$ and $b_1 \leq b_2$, then $|a_1-b_1|+|a_2-b_2| \leq |a_1-b_2|+|a_2-b_1|$. This proves it for lists of size $n$ as well - if there is an inverted pair, the SAD (sum of absolute differences) only decreases when you exchange elements of list b to remove inverted pairs.

Minimization of difference


Restringir versões iOS

Question: Há alguma maneira de eu restringir versões do iOS?
Detalhe: Não é a versão minima
Por exemplo, eu desenvolvi um aplicativo e gostaria que ele rodasse nas versões iOS 7.x.x e iOS 8.1.x. Resumindo, gostaria de evitar que o aplicativo fosse instalado nas versões iOS 8.0.x.
Por que? Atualmente no iOS 7 funciona perfeitamente, no iOS 8.1.x também, porém no iOS 8.0.x o aplicativo apresenta vários problemas, inclusive crashes constantes.
Comment: Sugiro que você verifique e repare os erros. Se você impedir que o app rode na versão 8.0 existe uma grande chance da Apple rejeitá-lo caso o reviewer faça algum teste no 8.0.
Comment: Então @AndréRibeiro, isso é tenso, nas simulações que foram feitas e testes aqui, está tudo OK, porém tem usuário relatando problemas, o qual a versão do iOS é a 8.0. Em um iPhone 5
Answer: Impedir a instalação não é possível, mas no AppDelegate você pode verificar a versão do dispositivo e impedir que ele conclua a abertura de acordo com a versão do sistema operacional.
var systemVersion = UIDevice.currentDevice().systemVersion();

Mas sendo franco, além de não conseguir aprovação da Apple para publicação, naturalmente porque você vai ter que "matar" (com um abort()) o aplicativo antes de concluir a abertura, isso é um workaround muito ruim em qualquer caso e em qualquer área de desenvolvimento (me lembra muito quando eu não queria que meus sites rodassem no Internet Explorer).
Meu conselho é: não adianta tapar o sol com a peneira. O ideal é você permanecer com a versão mínima e tentar localizar quais as falhas ocorrem com o seu aplicativo no ambiente com iOS 8.0 para poder solucioná-las de forma que isso não seja impedimento.
Se você acatar essa ideia, vai fazendo suas perguntar, pesquisando e etc, porque problema em uma versão específica do sistema operacional é mais cara de incompatibilidade de um certo código, o que é totalmente apto para ser resolvido.
Comment: Obrigado Paulo, pois é cara, o problema que mais comum acontece é que o Core Location simplesmente se perde. Não encontra mais localização, tendo que reiniciar serviço de localização e dependendo do caso, em um iPad wifi por exemplo, tenho de reiniciar serviço de internet.
Comment: O problema é que nos testes, os mesmos apresentaram sucesso, tudo ok. Bem no fim, após o lançamento ele está ocorrendo, e em um iPhone 5 iOS 8.0. Pesquisando e tudo mais os erros de localização (Que não é aquele esquema de requisitar a permissão, pois isso está OK), o pessoal não obtiveram solução, pois parece ser algo de gerenciamento interno do iOS ou do dispositivo em questão
Comment: Basicamente o erro que está acontecendo é este: didFailWithError: Error Domain=kCLErrorDomain Code=0 "The operation couldn't be completed. (kCLErrorDomain error 0.)"
Comment: @Gian Dê uma olhada nesta resposta: [Link](http://stackoverflow.com/a/6768989)
Comment: @Paulo Rodrigues, exato foi através deste link que encontrei a conclusão de que realmente é algum gerenciamento "fora" do alcance do código do meu aplicativo, pois até mesmo o Google Maps, que utilizo em meu app, acaba se perdendo na localização do dispositivo.
Comment: Eu já me deparei com esse problema, mas sempre foi no simulador (inclusive em outras versões do iOS), assim como neste link que o *André* passou e outros que também encontrei. E tendo relação com a conexão, acredito que seja preciso fazer mais testes em outros dispositivos com iOS 8.0, pois pode ser questão específica de aparelho, seja software ou hardware.

Restringir versões iOS


Bach's keyboard concertos on fortepiano?

Question: It is known that J.S.Bach was acquainted with the fortepiano at least twice in his life, in 1726 when he played one of Silbermann's experimental models (and mildly criticized it), and toward the end during his famous visit to Potsdam, where he supposedly got the inspiration for the Musical Offering. It is further known that the mysterious instrument that appeared at the Zimmermann café around the time that Bach was director of the Collegium Musicum was some sort of "louder" or otherwise more powerful clavier built in Berlin, suitable for outdoor performances.
Is it possible that this unspecified instrument might have been some form of early fortepiano like the ones Silbermann was working on at the time? Are there any recordings of Bach's keyboard concertos (which he wrote for this instrument) on the fortepiano?
Comment: @AlbrechtHügli: The word _fortepiano_ usually means a wood-framed historical piano (or replica of such) no later than the time of Beethoven and Schubert, though sometimes usage is stretched a generation to pianos of the time of Chopin and Schumann.
Comment: I think any answer to this question would be mere speculation. It is interesting, though.
Comment: Indeed, I am expecting an answer that can offer state-of-the-art viewpoints on this matter, and possibly even a personal opinion, certainly not a definitive yes/no answer. The intent was at least to spark discussion. (Though I *am* still looking for recordings of the concertos on fortepianos)
Comment: Do you mean recordings on piano or grand piano (simply not on harpsichord)? I have them by Schiff. Just look under Bach Klavierkonzerte CD.
Comment: "some sort of louder instrument": see https://baroquemusic.org/bargerhpschd.html "Recent research has established that for his weekly concerts at Zimmermann's Coffee House Bach had a double manual harpsichord (16', 3x8', 4') mounted on a pedal harpsichord (2x16', 3x8') made by Zacharias Hildebrandt, who was both harpsichord builder, and organ builder under the direction of Bach's friend and colleague Gottfried Silbermann."  I don't doubt that Bach would have performed the keyboard concertos on a (suitable) fortepiano, but that doesn't imply that any of them was written for the fortepiano.
Answer: According to a quite new recording of well-tempered clavier Vincent Bernhardt (page German only), very short French text at the recording label, Cafe Zimmermann had a harpsichord with 4 registers, including a 16 foot register and a Lautenzug (lute stop or nasale stop, see Encyclopedia Britannica) built in the workshop of Zacharias Hildebrandt. This would contradict the assumption, that migration towards pianoforte started there.
Answer: COULD BACH HAVE PLAYED AN EARLY FORTEPIANO AT THE ZIMMERMANN CAFÉ IN 1733?
According to Eva Badura-Skoda, not only could have, but "probably" did. In "Did Bach Compose 'Pianoforte Concertos'?" (2000), she lays out evidence (primarily a history of the terms used to reference the fortepiano) ...

Considering this terminological situation it appears to be a grave error to claim that the word "cembalo" found in eighteenth-century documents always meant a harpsichord with quills. A hammer-harpsichord might have been meant instead, even in Johann Sebastian Bach's time. (6)

... that the "new kind of cembalo" mentioned in the Zimmermann concert announcement ...

Gracious permission having been received from His Electoral
and Royal Highness that the collegia musica may be resumed
after their interruption, a fine concert is to be presented by the
Bach Collegium Musicum tomorrow, Wednesday, June 17, at
Zimmermanns Garden on Grimmischen Stein-Weg at four
o'clock, with weekly continuation. At this occasion, a new
kind of cembalo, which has not been heard heretofore [ein
neues Clavicymbel, dergleichen allhier noch nicht gehöret
worden ], will be used. The public [Liebhaber] and specialists
[Virtuosen] are cordially invited. (6)

... could well have been an improved Silbermann, ...

Taking an unprejudiced view of the early history of the piano, the creation of its name around 1732 and the new findings regarding the slow acceptance of it, one is bound to conclude from this document that the instrument Bach played on this occasion was probably the "nuovo cembalo," namely, Silbermann's new fortepiano. (7)

... though she acknowledges another possibility (a Lautenwerk) in the accompanying footnote.

theoretically, it might also have been a "Lautenwerk," a stringed keyboard instrument, which, if built in the form of Flügel, was likewise sometimes called "cembalo." But these instruments were in existence since 1713. (7n8)

Badura-Skoda gives an expanded account in "The Eighteenth-Century Fortepiano Grand and Its Patrons: From Scarlatti to Beethoven" (Indiana University Press, 2017, Chapter 5, "Johann Sebastian Bach and the 'Piano et Forte'", 150 - 181).
ARE THERE ANY RECORDINGS OF BACH'S KEYBOARD CONCERTOS (WHICH HE WROTE FOR THIS INSTRUMENT) ON THE FORTEPIANO
I have not found any.
Robert Hill and Ivo Sillamaa have both recorded Bach concertos on fortepianos, but their respective instruments are either too early (K Hill after B Cristofori; thus, before 1731, the year Cristofori died) or too late (Chris Maene after Anton Walter, 1795).
There are performances on Silbermann fortepianos of other Bach works: for example, the Courante from the first Partita and the Chromatic Fantasy.


Linear transformation and Exact sequences

Question:
Let V be a vector space over the field F, and W $\subset$ V a subspace. Then the linear transformation T: V $\rightarrow$ V / W is well-defined, since v $\in$ [v].

I understand v $\in$ [v], but how v $\in$ [v] implies T is well-defined? To my knowledge, being well-defined means that there is only one possible value for each T(v).

Supopse A, B and C are three vector spaces. Furthermore, assume the sequence $0\rightarrow A\rightarrow B\rightarrow C\rightarrow 0$ is exact. Then, ker($B\rightarrow C)=A$.

By the definition of the exact sequence, ker($B\rightarrow C)$=im($A\rightarrow B)$. Could you explain why ker($B\rightarrow C)=A$?
Answer: These are two separate questions.
1) $T$ implicitly sends $v$ to $[v]$. $[v]$ is unique: it describes a single member of $V/W$. Therefore $T$ is well-defined, since it sends $v$ to a unique $T(v) = [v]$.
2) Technically saying $\ker(B\to C) = A$ is a bit inaccurate, since the kernel must be part of $B$, not $A$. However, note that the short exact sequence implies that the map $A \to B$ is injective; therefore, $A$ is isomorphic to the image of $A \to B$ in $B$ (let's call this image $I$). As you noted, $I$ is the kernel of $B \to C$. Therefore, $A$ is isomorphic to the kernel of $B \to C$.
Comment: Yes (or at least with $B/I$... technically the notation $B/A$ won't have any meaning if $A \not\subset B$, but you'll see that kind of notation used anyway)
Comment: Thank you! Since the map $B\rightarrow C$ is surjective, is C isomorphic with B / A?


IBAForms: how to create a IBATextFormField which is not editable?

Question: I am using IBAForms to collection user input, there is a case where some of the fields are editable and others are not, is there any immutable or disabled IBAFormField, or any other way to implement this?
Thanks!
Answer: you can use IBAReadonlyFormField


Explain in simple terms : $P \to (Q \lor R) $ is equivalent to $P \land\lnot Q \to R$

Question: Can any body explain the logical statement in common terms:
$$P \to (Q \lor R) ~~\iff ~~(P \land\lnot Q) \to R$$

$\lnot$ negation ("not")
$\land$ conjunction ("and")
$\lor$ disjunction ("or")
$\to$ condition ("implies")
$\iff$ logical equivalence ("if and only if")
Comment: Does Q' mean Q is not real?
Answer: Assume that $P\rightarrow Q\vee R$, we are to show that $P\wedge\neg Q\rightarrow R$. Now assume that $P\wedge\neg Q$ holds, in particular, $P$ holds, so by assumption, we have that $Q\vee R$ holds, we get that either $Q$ or $R$ holds. Since we have $\neg Q$ holds, which means that $Q$ does not hold, so from either $Q$ or $R$ holds, we get that $R$ must hold, this justifies that $P\wedge\neg Q\rightarrow R$.
Now assume that $P\wedge\neg Q\rightarrow R$, we are to show that $P\rightarrow Q\vee R$. So assume that $P$ holds but $Q\vee R$ does not hold, then neither $Q$ nor $R$ holds. In particular, $\neg Q$ holds, so $P\wedge\neg Q$ holds, so we get that $R$ holds, but we have both $R$ and $\neg R$ hold, this is a contradiction. The contradiction comes from that we have assumed that $Q\vee R$ does not hold, so $Q\vee R$ must hold, this justifies that $P\rightarrow Q\vee R$.
Answer: Look at these two truth tables: $$\begin{array}{|c|c|c|c|c|}
  \hline
  % after \\: \hline or \cline{col1-col2} \cline{col3-col4} ...
  P & Q & R & Q \cup R =:I &  P\Longrightarrow I\\
  \hline
  0 & 0 & 1 & 1 & 1 \\
  0 & 0 & 1 & 1 & 1 \\
  0 & 1 & 0 & 1 & 1 \\
  0 & 1 & 1 & 1 & 1 \\
  1 & 0 & 0 & 0 & 0 \\
  1 & 0 & 1 & 1 & 1 \\
  1 & 1 & 0 & 1 & 1 \\
  1 & 1 & 1 & 1 & 1 \\
  \hline
\end{array}$$
$$\begin{array}{|c|c|c|c|c|}
  \hline
  % after \\: \hline or \cline{col1-col2} \cline{col3-col4} ...
  P & Q' & R & P \cap Q' =:J &  J\Longrightarrow R\\
  \hline
  0 & 1 & 1 & 0 & 1 \\
  0 & 1 & 1 & 0 & 1 \\
  0 & 0 & 0 & 0 & 1 \\
  0 & 0 & 1 & 0 & 1 \\
  1 & 1 & 0 & 1 & 0 \\
  1 & 1 & 1 & 1 & 1 \\
  1 & 0 & 0 & 0 & 1 \\
  1 & 0 & 1 & 0 & 1 \\
  \hline
\end{array}$$
Answer: $$
P\Longrightarrow (Q\cup R)
$$
is equivalent to
$$
P'\cup (Q\cup R)
$$
which is equivalent to
$$
(P'\cup Q)\cup R
$$
which is equivalent to
$$
(P'\cup Q)'\Longrightarrow R
$$
which is equivalent to
$$
P\cap Q'\Longrightarrow R
$$
Answer: $P\to (Q\lor R)$ means that "At least one from $Q$ or $R$ is true, if $P$ is true."
$(P\land\lnot Q)\to R$ means that "$R$ is true, if $P$ is true but $Q$ is false."
Assume $P\to (Q\vee R)$ holds; that is at least one from $Q$ or $R$ is true, if $P$ is true.   Well, from that we can infer that $R$ will be true is $P$ is true but $Q$ is false.   So $P\to (Q\vee R)$ logically entails $(P\land\neg Q)\to R$.
Likewise if $(P\land\neg Q)\to R$ holds we can infer that $P\to(Q\vee R)$ also holds.
In summary $(P\land\neg Q)\to R$ is logically equivalent to $P\to (Q\vee R)$
Answer:
$P \to (Q \lor R)$

If it is the weekend, then it is Saturday or Sunday.

$(P \land \lnot Q) \to R$

If it is the weekend, and it is not Saturday, then it is Sunday.
Notice how both statements are equivalent to saying "the only possible weekend days are Saturday and Sunday", but neither necessarily implies that Saturday is a weekend or implies that Sunday is a weekend.
Answer: You can get this from two simple equivalences:

$P \vee Q \Leftrightarrow \neg P \rightarrow Q$
$(P \wedge Q) \rightarrow R \Leftrightarrow P \rightarrow (Q \rightarrow R)$

Here $(P \wedge \neg Q) \rightarrow R$ is equivalent (by 2) to $P \rightarrow (\neg Q \rightarrow R)$ is equivalent (by 1) to $P \rightarrow (Q \vee R)$.


Hiding a css class permanently

Question: I'm hiding the css class .pds-pd-link. This class styles the link www.polldaddy.com . The link is hidden. When the user votes and clicks "Return To Poll" the link www.polldaddy.com is viewable again. I don't think there is any way to hide this permanently without modyifying source code of poll ?
http://jsfiddle.net/Wx5mM/2/

New Poll

Comment: `.hide` will hide the element; **not sure what do you mean by permanently** do you mean remove the element? Sorry if I sound thick `:)` , cheers
Comment: @yep, remove the element. No, not thick, far from it. thanks for your comment.
Answer: .pds-pd-link {
    display: none !important;
}

Seems to do the job
http://jsfiddle.net/Wx5mM/7/
Comment: Might as well remove the `$(".pds-pd-link").hide();`.
Comment: Didn't even know that was there
Comment: I'm sorry if that's the case. I'll check it out when I get the chance
Comment: @user701254: still works for me. Which browser are you using?
Comment: @Codemonkey it seems fine actually now again, im not sure what the issue was. thanks

Hiding a css class permanently


The value of $\int^{\pi/2}_0 \frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta$

Question: Problem :
Find the value of $\int^{\pi/2}_0 \frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta$
Now how to use Leibnitz's rule : $\frac{d}{dx}(\int^{\psi(x)}_{\phi(x)} f(t)dt) = \frac{d}{dx}\{\psi(x)\} f(\psi(x)) -\frac{d}{dx}\{\phi(x)\} f(\phi (x)\}$ here
please guide will be of great help thanks as  there is  $x$ and $\theta$ both
Comment: Differentiate $\displaystyle f(x) = \int^{\pi/2}_0 \frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta$ wrt $x$, then $\displaystyle f'(x) = \int^{\pi/2}_0 \frac{1}{(1+x\sin^2\theta)}d\theta$, and then put $t = \cot \theta$
Comment: @r9m Shouldn't it be $f'(x)=\int_{0}^{\frac{\pi}{2}}\frac{1}{1+x\sin^{2}(\theta)}d\theta$?
Comment: @r9m ; That's not quite right. Differentiation w.r.t $x$ should yield a $\sin ^2\theta$ in the numerator that will cancel with the one in the denominator.
Comment: @user99163 thanks .. that was a mistake!
Comment: @r9m You're welcome.
Answer: Let:
$$ f(x) = \int_{0}^{\pi/2}\frac{\log(1+x\sin^2\theta)}{\sin^2\theta}\,d\theta.\tag{1}$$
We have $f(0)=0$ and for every $x\geq 0$:
$$ f'(x) = \int_{0}^{\pi/2}\frac{d\theta}{1+x\sin^2\theta}=\frac{\pi}{2\sqrt{1+x}}\tag{2}$$
through Weierstrass substitution. Integrating back,
$$ f(x) = \pi\left(\sqrt{1+x}-1\right).\tag{3}$$
Comment: can you elaborate from step (1) to step (2) how did you get that will be of great help thanks.
Comment: @sultan: just differentiate $\log(1+x\sin^2\theta)$ with respect to the $x$-variable.
Answer: In the interests of alternative methods: $$\int_0^{\pi/2}\frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta =-\sum_{k=1}^\infty\frac{(-1)^kx^k}{k}\int_0^{\pi/2}\sin^{2k-2}\theta d\theta.$$ Now let $u=\sin\theta$, then $\frac{du}{\sqrt{1-u^2}}=d\theta$ and we have
$$-\sum_{k=1}^\infty\frac{(-1)^kx^k}{k}\int_0^1\frac{u^{2k-2}}{\sqrt{1-u^2}}du = -\frac{\sqrt{\pi}}{2}\sum_{k=1}^\infty\frac{(-1)^kx^k}{k}\frac{\Gamma\left(k-\frac{1}{2}\right)}{\Gamma(k)}.$$ Mathematica quickly evaluates this to have closed form
$$\pi\left(\sqrt{x+1}-1\right),$$
hence we have
$$\int_0^{\pi/2}\frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta=\pi\left(\sqrt{x+1}-1\right).$$
There may be conditions on $x$ and the exchange of integral and summation would need to be justified. Proof of the summation would also be required of course.
Comment: Nice methodology, but the result is not quite correct.
Comment: Sorry my mistake - updated.
Comment: No worry.  +1 for the nice solution!
Answer: Let $I(x)$ be the integral
$$I(x)=\int_0^{\pi/2}\frac{\log(1+x\sin^2 \theta)}{\sin^2\theta}d\theta$$
and assume that $x>0$ is real-valued.
Now, taking a derivative with respect to $x$ gives
$$I'(x)=\int_0^{\pi/2}\frac{1}{1+x\sin^2\theta}d\theta=\frac{\pi}{2\sqrt{1+x}}$$
where this latter integral was evaluated using contour integration.
Next, integration of $I'(x)$ gives
$$I(x)=\pi\sqrt{1+x}+C$$
where $C$ is found by noting $I(0)=0$.  Thus, $C=-\pi$.
Finally, we have
$$I(x)=\pi(\sqrt{1+x}-1)$$

NOTE:
Here, we will evaluate $\int_0^{\pi/2}\frac{1}{1+x\sin^2\theta}d\theta$ for $x<0$ using contour integration.  Note that in general, we have
$$\begin{align}
\int_0^{\pi/2}\frac{1}{1+x\sin^2\theta}d\theta & = \int_0^{\pi/2}\frac{1}{1+x\left(\frac{1-\cos 2\theta}{2}\right)}d\theta \tag 1 \\\\
&=-\frac{1}{2x}\int_0^{2\pi}\frac{1}{\cos \theta-a}d\theta \tag 2
\end{align}$$
where $a=\frac{2+x}{x}=1+\frac2x$.  In going from $(2)$ to $(3)$ we exploited the periodicity of the cosine function and effected the substitution $2x \to x$.
Next, we move to the complex plane by letting $z=e^{i\theta}$ so that $dz=ie^{i\theta}dz$ and the integration is over the closed-contour unit circle$C$, for which $|z|=1$.  Then, we have
$$\begin{align}
-\frac{1}{2x}\int_0^{2\pi}\frac{1}{\cos \theta-a}d\theta &=-\frac{1}{ix}\oint_C \frac{dz}{z^2-2az+1}\\\\
&=-\frac{2\pi}{x}\text{Res}\left(\frac{1}{(z-a+\sqrt{a^2-1})(z-a-\sqrt{a^2-1})}\right)\\\\
&=-\frac{2\pi}{x}\text{Res}\left(\frac{1}{(z-\frac{x+2}{x}-\frac{2\sqrt{x+1}}{|x|})(z-\frac{x+2}{x}+\frac{2\sqrt{x+1}}{|x|})}\right)
\end{align}$$
There are three cases we need to examine.

Case 1: $x>0$
For $x>0$, the only pole is at $z=\frac{x+2-2\sqrt{x+1}}{x}$.  The residue is thus $-\frac{x}{4\sqrt{x+1}}$ and we have
$$I'(x)=\frac{\pi}{2\sqrt{x+1}}$$

Case 2:
For $<-1x<0$, the only pole is at $z=\frac{x+2}{x}+2\frac{\sqrt{x+1}}{|x|}=\frac{x+2-2\sqrt{x+1}}{x}$.  The residue is $-frac{x}{4\sqrt{x+1}} and we have
$$I'(x)=\frac{\pi}{2\sqrt{x+1}}$$

Case 3:
For $x<-1$, we note that $|\frac{x+2}{x}+2\frac{\sqrt{x+1}}{|x|}|=1$ and the poles are complex conjugates residing on the unit circle.  Thus, the integral diverges.
We can intepret, however, a Cauchy Principal value of the integral.  Here, we exclude the poles by modifying the contour $C$ with a semi-circle deformation around each pole.  The two contributions from integration around the deformations are equal in magnitude and of opposite sign.  Thus, we have
$$I'(x)=0$$
Comment: you used Weierstrass substitution to obtain $I'(x)=\frac{\pi}{2\sqrt{1+x}}$ ?
Comment: I make a topic because I meet some trouble Dr.MV
Comment: @Lucas The integral can be evaluated using the Weierstrauss substitution or contour integration.
Answer: $$I(x) = \int_0^{\pi/2}\frac{\log(1+x\sin^2\theta)}{\sin^2\theta}d\theta$$
$$I'(x) = \int_0^{\pi/2}\frac{1}{1+x\sin^2\theta}d\theta$$
Writing 1 as  $\sin^2\theta + \cos^2\theta$, we have
$$\int_0^{\pi/2}\frac{d\theta}{\sin^2\theta(1+x) +\cos^2\theta}$$
Multiplying and dividing by $\sec^2\theta$ and setting $\sqrt{1+x}\tan\theta = t$,
We evaluate the integral to be
$$I'(x)=\frac{\pi}{2\sqrt{x+1}}$$
Then by elementary integration, we get the desired result, and checking that $I(0) = 0$
$$I(x) = \pi(\sqrt{1+x}-1)$$


Can I make iOS Simulator listen to keyboard events for debugging purposes?

Question: What I want to do is make some shortcut key for debugging on iOS simulator. e.g. Right Arrow key to call my speed up function to make game run faster. Note that the soft keyboard is not on when I hit debug keys.
Answer: Override the keyCommands property to enable shortcut keys. It is a property on UIResponder, so this can be done in any of your views, view controllers, or even your app delegate.
Answer: FLEX does all the tricks for you.
You just need to register keys that you are interested in,
FLEXManager *flex = FLEXManager.sharedManager;

// open or close debug
[flex registerSimulatorShortcutWithKey:@"D" modifiers:(UIKeyModifierFlags)0 action:^{
    // Do what you want to do when user click "D" on hardware keyboard
} description:@"Open/Close DebugViewController"];

Answer: The only straightforward way of achieving this (that I can think of) is to create a few very small buttons in your app and connect them to the functions you want to call. You can place them somewhere where they don't interfere with the user interface. Because your developing for a mobile device you unfortunately can't press a key on the keyboard and have it call a function.
Comment: thanks for the answer, @JoJoe. I was hoping to have a workaround since it is actually running on a Simulator. I'll wait a bit more in case somebody has another suggestion. Otherwise, mark your answer as correct. I know that I can use buttons. Thanks again :)


Problem with web screenshots requiring authentication

Question: I am making app that takes a screenshot of a URL requested by the user. I want to make it as transparent as possible when sites that require username and passwords are in question.
For instance, if user wants to screenshot its iGoogle page, he will send the server URL but, the screenshot will not be the same as what he sees on his screen.
Is there any way to do this ? I guess that in such cases I will have to actually request screenshot from the user. Perhaps user can even deliver me his cookie for that domain.
Any thoughts ?
Ty.
Answer: Yes, in most cases you'll need user's cookies.
If site uses regular cookies, you can create bookmarklet that reads document.cookie. This will not work with httpOnly cookies which are used increasingly often for sessions.
Some sites limit sessions to certain IP, and in that case you can't take screenshot without proxying request through user's computer.
If you can get user to use bookmarlet, an interesting trick would be to read and send DOM to your server:
image.src = 'http://example.com?source=' +
            escape(document.documentElement.innerHTML);

For HTTP authentication easiest solution would be to ask user for login/password.
Comment: Interesting. So, what of above you suggest as a most generall solution. The user can literally request anything to be screenshoted. The "interesting trick" is really interesting :) The problem is how to recognise such URLs that have a related cookie. If I can't recognise that, i can offer a user a choice, but not otherwise. So, in other words, there is no thing like "if site uses this or that". Site is random :)
Comment: More about interesting trick... Are you totally sure it will produce exact thing on the servers browser ? I kinda like the trick a lot :) Its really great thinking man.
Comment: It is _supposed_ to be exact, but of course reality, especially in IE, is different. IE might not show `` elements or contents of inline `


Removing an item from a BlockingCollection

Question: How can an item be removed from a BlockingCollection? Which of the following is correct?
myBlockingCollection.Remove(Item);

or
myBlockingCollection.Take(Item);

Answer: You can't specify a particular item to be removed from a BlockingCollection<T>.
The Take() method removes an item from the underlying collection and returns the removed item.
The TryTake(out T item) method removes an item from the underlying collection and assigns the removed item to the out parameter. The method returns true if an item could be removed; otherwise, false.
The item which is removed depends on the underlying collection used by the BlockingCollection<T> - For example, ConcurrentStack<T> will have LIFO behavior and ConcurrentQueue<T> will have FIFO behavior.
Answer: What about this code? - It's working but change the order of the collection. (And I didn't checked it in multi threads state).
public static bool Remove<T>(this BlockingCollection<T> self, T itemToRemove)
    {
        lock (self)
        {
            T comparedItem;
            var itemsList = new List<T>();
            do
            {
                var result = self.TryTake(out comparedItem);
                if (!result)
                    return false;
                if (!comparedItem.Equals(itemToRemove))
                {
                    itemsList.Add(comparedItem);
                }
            } while (!(comparedItem.Equals(itemToRemove)));
            Parallel.ForEach(itemsList, t => self.Add(t));
        }
        return true;
    }

Comment: The trouble with this is none of the other methods on the collection are going to lock on self. For instance what happens if another thread adds a new item after you create itemsList.
Answer: I think only TryTake() is an option? I can't find documention on the Remove() method on MSDN.
Answer: I think TryTake(out item) will work. Remove does not exist in BlockingCollection class and Take does not take item as parameter.
Comment: TryTake(out item) does not specify which item to remove like Remove(item) does. There is not equivalent.


Change of basis: find the coordinates of the point relative to the coordinate system.

Question: This is the complete problem:

The solution:

I tried to find out why -v1' = (2/3)v1 + (1/3)v2 but I still don't get it. If anyone can help me, I'll be very grateful.
Answer: To calculate $v_1'$ you have to find other vectors that compose $v_1'$. Since $\frac{|\overrightarrow{BD}|}{|\overrightarrow{DC}|} = \frac{1}{2}$ you get
$v_1' = \frac{1}{3} \overrightarrow{CB} - v_1$. With $\overrightarrow{CB} = -v_2 + v_1$ you get
$$v_1' = \frac{1}{3}(-v_2 + v_1) - v_1 = -\frac{1}{3}v_2 - \frac{2}{3} v_1$$


What's wrong with my way of applying the convolution formula

Question: Suppose $X_1, X_2, X_3$ are iid random variables with uniform$[0,1]$. I'm trying to find $\mathbb{P}(X_1 + X_2 > X_3)$. I know that there are answers elsewhere (it's $\frac{5}{6}$). To improve my familiarity with the convolution formula, I decided to do it this way. First, $Y = X_1 + X_2$ has a pdf that looks like a triangle, i.e. $f_Y(y) = \chi_{[0,1]} y + \chi_{[1,2]} (2-y)$. Then I define $Z = Y - X_3$, where $-X_3$ follows uniform$[-1,0]$ and apply the convolution formula
$$f_Z(z) = \int f_Y(y) f_{-X_3}(z - y) dy$$
To figure out the limits of integration, note that $z - y \in [-1,0] \implies y \in [z, z+1]$ so
$$f_Z(z) = \int_{z}^{z+1} f_Y(y)$$
When $z$ is between $-1$ and $0$,
$$f_Z(z) = \int_{z}^{1} z dz + \int_{1}^{1-z} 2 - z dz = 1/2 - z^2 - z$$
And integrating $\int_{-1}^0 f_Z(z) = \int_{-1}^0 1/2 - z^2 - z dz = \frac{2}{3}$.
Then $\mathbb{P}(Z > 0) = \frac{1}{3}$ (but it's supposed to be $\frac{5}{6}$).
What went wrong? I'm completely stuck.
Answer: When $-1 <z<0$ we have $\int_z^{z+1} f_Y(y)dy=\int_0^{z+1}ydy$ because $z+1 $ is between $0$ a and $1$ and $f_Y(y)$ vanishes for $y <0$. The value of this integral
is $\frac 1 6$ so the required probability is $1-\frac  16=\frac 5 6$.


Same environment(jenkins node), slower executable jar

Question: I have set up a new node in Jenkins.
Except for the server specifications, everything else is the same, but it takes twice as much execution speed on the new node server.
Has anyone ever experienced this? I need help.

Old Node
New Node

Server Spec
4vCPU, 8GB Mem
8vCPU, 16GB Mem

JDK
1.8.0_161 OpenJDK
1.8.0_161 OpenJDK

Jar
same
same

Jar execute option(e.g. -XX:MetaspaceSize=512m and etc...)
same
same

Build time
7 min
13 min
Comment: Well, this *can* be caused by the application itself - for example, if the number of threads depends on the number of available cores, but synchronisation overhead drowns out the benefits even for a small number of threads - this obviously depends on what the application does.
Comment: Not to mention that there are all other sorts of specs that could matter.
Comment: Sometimes, providing more memory can also slow things down (GC overhead slowly increases with size of managed memory, etc. some things that fail quickly with little memory take longer to fail with more memory, and lots of other effects).
Comment: You do not provide enough information in your question. What do you execute exactly and which CPUs do you use? If your code is single-threaded the number of CPU cores is irrelevant, if your code is IO bound the spec of your storage or even network speed may be more relevant. Depending on what you execute the access times of your memory may be more relevant than its size ...
Comment: I note you said "vCPU" rather than CPU.  Note that the performance of a "vCPU" depends on how busy the compute node is, and depends on whatever other virtual servers may be doing on the compute node.  It could also depend on how busy the server's disk back-end is.  In short, you should be talking to your cloud computing provider ....
Comment: Hello everyone. Sorry for the late comment. I've compared all the server setting in
 old node server with new server, and I find out ssh config was set in the old server: `StrictHostKeyChecking=no`. So I added the setting on the new node server, and the job runs in time as I expected. Thank you for all.
Answer: Sorry for the late comment.
I've compared all the server setting in old node server with new server, and I find out ssh config was set in the old server: StrictHostKeyChecking=no.
So I added the setting on the new node server, and the job runs in time as I expected.
Thank you for all.


AUI textboxlist - retrieve removed elements value

Question: I am using Liferay 6.2 and want to retrieve the value of removed element from a Textboxlist component. I have stored a list of values in a hiddenInput element, and I display the list in a Textboxlist. As I remove the element, I want to update the values stores in the hidden input element. But I do not know how to retrieve the removed element.
AUI().ready('aui-textboxlist-deprecated', function (A) {

    var source = A.one('#hiddenInput').val().split(',');

    var tagslist = new A.TextboxList({
        contentBox: '#demo',
        dataSource: source,
        matchKey: 'name',
        schema: {
            resultFields: ['key', 'name']
        },
        schemaType: 'json',
        typeAhead: true,
        width: 500
    }).render();

    var values = A.one('#hiddenInput').val().split(',');
    A.each(values, tagslist.add, tagslist);

    var updateHiddenInput = function (event) {

        //how to get the removed element?

    }

    tagslist.entries.after('remove', updateHiddenInput);
});

How to achieve this?
Comment: If you just need the label, then you can get it using `event.attrName`. If you need to work with the element, it is passed in `event.item.entry`.
Answer: As @jbalsas said in the comments:

If you just need the label, then you can get it using event.attrName. If you need to work with the element, it is passed in event.item.entry.

So you should be able to do it like this:
var updateHiddenInput = function (event) {
    var hiddenInput = A.one('#hiddenInput');
    hiddenInput.val(hiddenInput.val() + ',' + event.item.entry); // or event.attrName
}


Slice sampling in Particle Gibbs with Ancestral Sampling

Question: Bear with me as I am not from statistical background. My question is about the implementation of PGAS algorithm as given in Lindsten et. al 2014 concerning sampling in state-space models. The two algorithms in question taken from the aforementioned paper are given below:

The algorithm makes use of PGAS Markov kernel as given below.

I essentially have two (possibly naive) questions:

For sampling the hyperparameters, $\theta[n]$ above, I am using Metropolis-Hastings or slice sampling. As the sampling generally has a burn-in period, I suppose I always run the sampler for say $K$ iterations and take only the $K^{\text{th}}$ sample? If this is the way to go, it seems waste of computation time. Or are there other ways like generating $K+M$ samples and then sampling uniformly from the $M$ later samples which does not make much sense to me to be honest? What should the value of $K$ (or $M$ if applicable) be ideally?
Second question is about the optimality/learning of the hyperparameter $\theta$. I want to know how the algorithm learns the optimal (in some sense) $\theta$. I can see how the improvement is trajectory, $x_{1:T}[n]$, forces the conditional $p(\theta \vert x_{1:T}[n], y_{1:T})$ to be more and more accurate, which in turn improves the quality of the sampled $\theta$. Is this all there is to the learning process or am I missing something?

I guess for both questions, a simple 'yes' or 'no' would suffice but I would appreciate an insightful answer that can help somebody who is new to the topic have a better understanding of the concept at hand.
Answer: Second question
You're not optimizing a function here. This is Bayesian statistics, not frequentist, so you're trying to get at the posterior distribution. In your case, it's the posterior of the state process paths and the parameter vector. Yes, you can boil down that distribution into a point estimate, or an interval, but generally that's not done by optimization (unless you're doing MAP estimates)--it's done by integration. MCMC samplers draw samples to approximate this posterior distribution. Typically the samples are averaged to approximate expectations, not to estimate optima.
Ideally MCMC samplers are ergodic. This means, roughly, that if you run the sampler for long enough, taking a sample at some iteration down the line will be very close to taking a sample from the true posterior distribution. There are multiple forms of ergodicity, and some are stronger than others.
On the face of this, this isn't a great guarantee. How long until the samples get close? Also, this is a guarantee for one time point's sample. What most people want to do is to take averages of many samples, and they want guarantees that these sample averages will be close to the true theoretical expectation of the posterior. Fortunately, laws of large numbers and central limit theorems hold for chains that are ergodic in a stronger sense. This is discussed in chapter 17 of Meyn and Tweedie's book.
First question
Algorithm 4 is a Gibbs sampler, which means you alternate sampling from the conditional posteriors. In this case you alternate between sampling state paths and parameter vectors. A standard Gibbs sampler would sample from these two posterior distributions exactly.
If you use another Markov chain technique to sample parameter vectors, instead of sampling from the conditional posterior exactly, then that is complicating the algorithm. This is a strategy that's used in more simplistic Gibbs sampling (e.g. standard Metropolis-within-Gibbs), but your complicating the algorithm might void the ergodicity guarantees provided in the paper.
Even if you did verify that the sampler was still ergodic, the answer to this question would still depend on the specifics of which of these algorithms you used, and how you tuned the MH bit.
Comment: I will rephrase your answer to my questions for my understanding. For the second point I, more or less, understand the difference in frequentist and Bayesian approaches, ergo the use of 'optimal in some sense'. I guess to get anything that can be meaningfully called optimal, I would have to consider the MAP. I just wanted to confirm this.
Comment: For the first point, only the unnormalized distribution $p^*(\theta \vert x_{1:T}, y_{1:T})$ is known. That is why I use MH or slice sampling because my understanding says that this is equivalent to sampling from the exact posterior. People refer to burn in periods but I don't have any guideline to follow to determine it. In general I am looking for some thumb rules/heuristics to define this burn-in period. I am using simple MH with Gaussian proposal distribution if that helps.
Comment: I would be surprised if there were any reputable rules of thumb for tuning your MH sampler inside PGAS. "I guess to get anything that can be meaningfully called optimal, I would have to consider the MAP." <- No: mean/medians are optimal in the decision theoretic sense after you define your loss function.


Canadian Senior Math Contest Question

Question: I was trying to solve the problem below but was completely unable to. My question is that which general category do questions like these belong to?. Once, I know that I would go learn the specific math concept and attempt at solving the question.

For each positive integer $n$, define $a_n$ and $b_n$ to be the positive integers such that
$$\left(\sqrt 3 + \sqrt 2\right)^{2n} = a_n + b_n\sqrt 6$$
and
$$\left(\sqrt 3 - \sqrt 2\right)^{2n} = a_n - b_n\sqrt 6$$
(a) Determine the values of $a_2$ and $b_2$.
(b) Prove that $2a_n - 1 < \left(\sqrt 3 + \sqrt 2\right)^{2n}<2a_n$ for all positive integers $n$.
(c) Let $d_n$ be the ones (units) digit of the number $\left(\sqrt 3 + \sqrt 2\right)^{2n}$ when it is written in decimal form. Determine, with justification, the value of $d_1+d_2+d_3+\cdots+d_{1865}+d_{1866}+d_{1867}$ (the given sum has 1867 terms.)
Comment: (a) is arithmetic. (b) might be induction. (c) could be Number Theory.
Comment: It is important that $\sqrt 3 - \sqrt 2 \lt 1$ so when you raise it to a power it gets small.  Then note that $(\sqrt 3 + \sqrt 2)^{2n}+(\sqrt 3 - \sqrt 2)^{2n}$ is an integer because the terms with square roots in them cancel.  This is a recurring theme in olympiad problems.
Comment: For a) take n=2 and calculate. You get two equations from it. Solve this system of $a_2$ and $b_2$
Comment: Look at the units digits of both $a_n$ and $b_n$ for small $n$ (say, through $n=5$) and see if you notice anything useful.
Answer: I would call all of these algebra, but this is definitely competition math algebra. These are solvable by expanding through binomial expansion (and I believe this is the intended solution).
If you need more hints, ask below.

Canadian Senior Math Contest Question


Complex Gaussian Integral - $\int_{-\infty}^\infty e^{-p(t+c)^2}dt = \sqrt{\frac{\pi}{p}}$?

Question: I found some formulas on books, especially the complex gaussian integral formula:
$$
\int_{-\infty}^\infty e^{-p(t+c)^2}dt = \sqrt{\frac{\pi}{p}}
$$
for $p,c\in\mathbb C$.
Then if $p=i=\sqrt{-1}$, the integral may have two different values, since $\sqrt{i}
$ has two different values on the complex plane.
Then how can I justify the above integral formula?
Comment: @karvens For $p=i$ you will get Fresnel-like integrals since $e^{ix^2}=\cos x^2+i\sin x^2$.
Comment: Are you sure that the integral converges for $p=i$? Am I missing something here?
Comment: Well, according to wolframalpha, $p=i$ results in the integral taking the principle value of $\sqrt{i}$.
Answer: That is true when $p$ has positive real part. For $p$ with non-positive real part, the integral diverges.
For real $c$, the proof of the real Gaussian integral still largely works.
For $p$ with non-zero real part, a case can be made to choose the square root with positive real part, based on the analycity of $ f(z) = \int_{-\infty}^{\infty} e^{-zu^2} du $ around the positive real semi-axis.
Purely imaginary $p$ is more complicated.
For non-real $c$, the Fourier transform of the Gaussian gives the same result. Showing that the derivative with respect to $c$ is zero should also work.
Comment: Answers on MSE should be as self-contained as possible, and with an eye towards semi-permanence.  I notice that two of your links are off-site, which means that they could someday suffer from linkrot.  Could you please take a few minutes to discuss (in your answer) the relevant details found on the linked pages?


Solution to equation $4n+1 = (2k+1)^2-(2u)^2$ over the natural numbers(!)

Question: Let $n \in \mathbb{N}$ be given, then I want to know if there are $k ,u \in \mathbb{N}$ (with $k \neq n, u \neq n$) such that $4n+1 = (2k+1)^2-(2u)^2$ holds.
What makes it difficult for me is the fact that I am looking for solutions over the natural numbers. Otherwise one would see that there are infinitely many solutions (for example by completing the square).
Comment: Can you find an $n$ satisfying
$$
n = (k+u)(k-u) +2k
$$ this looks nicer to my untrained eye!
Answer: A solution exists iff $4n+1$ is composite and not the square of a prime.
Factor the right side as $a\cdot b$ where
$$a=2(k-u)+1,\ \ b=2(k+u)+1.$$
Since $ab>0$ we have $a>0$ so $k\ge u;$ however $k=u$ leads to $4n+1=b=4k+1$ which is the excluded case $k=n.$ Now since $a,b$ are odd [with $a<b$ since $u\ge 1$] we can rule out $4n+1$ being the square of a prime. Or being a prime for that matter, since at this point $a \ge 3$ and $a<b.$
Now a subtraction gives $b-a=4u,$ so for later we can put $u=(b-a)/4$ toward recovering $u,k.$ Addition gives $b+a=4k+2,$ and in this putting $b=a+4u$ leads to $a+2u=2k+1,$ or with our above $u=(b-a)/4,$ we can solve for $k.$ In sum,
$$k=(1/4)(a+b-2), \ \ u=(1/4)(b-a). \tag{*}$$
To apply the above, we start with a composite number $4n+1$ which is not the square of a prime, write it as $ab$ with $1<a<b,$ and use formulas $*$ to find values for $k,u.$ Note that if one tries it on the square of a prime, one either must use $a=1$ which doesn't work as noted, or else one must use $a=b$ which gives $u=0$ which is also to be ruled out, provided $0$ is not considered a natural number.
Note: Since $ab$ is 1 mod 4, and both are odd, we automatically have $a=b$ mod 4, so that $u=(b-a)/4$ is a positive integer provided $a<b.$ Since each is odd and they are equal mod 4, one can also show that $(a+b-2)/4$ is an integer in equations $*.$
Answer: Given

$$
4 n + 1 = \Big( 2 k + 1 \Big)^2 - \Big( 2 u \Big)^2.
$$

Set

$$
k = u + a,
$$

then we get

$$
n = a^2 + a + \big( 2 a + 1 ) u.
$$

The case $a=0$ is the solution

$$
n = u = k,
$$

which is to be excluded.
For every $a > 0$ we have a linear relation between $n$ and $u$.

$$
\begin{array}{l|ll}
a & n\\
\hline
1 & 2 + 3u\\
2 & 6 + 5u\\
3 & 12 + 7u\\
4 & 20 + 9u & = 2 + 9(u + 2)\\
5 & 30 + 11u\\
6 & 42 + 13u\\
7 & 56 + 15u & = 2 + 3 ( 5 u + 18 ) = 6 + 5(3 u + 10)\\
\vdots & \vdots
\end{array}
$$

The equation is not general solvable for a given $n$.

In case $n$ is even, we can find solutions for:

$$
n = 8, 14, 16, 20, 26 (3\times),32, 36, 38 (2\times),40, 44, 46, 50, 52, 54, 56 (3\times),62,\\
66, 68 (2\times),74 (2\times),76, 82, 86 (2\times),92, 94, 96 (3\times), \cdots
$$

For example

$$
\begin{eqnarray}
4 \times 26 + 1 &=& \big( 2 \times 9 + 1 \big)^2 - \big( 2 \times 8 \big)^2\\
&=& \big( 2 \times 6 + 1 \big)^2 - \big( 2 \times 4 \big)^2\\
&=& \big( 2 \times 5 + 1 \big)^2 - \big( 2 \times 2 \big)^2
\end{eqnarray}
$$


How to assign and use variable in MVC Razor View

Question: I've identified several of my views that require some simple logic, for example to add a class to a set of controls created with @Html helpers. I've tried several different ways, but they either throw errors in the View or just don't work.
A simple example:
Assign variable:
<code>@if( condition )
{
    var _disabled = "disabled";
}

@Html.CheckBoxFor(m => m.Test, new { @class = "form-control " + @_disabled })
</code>
Or:
<code>@if( condition )
{
    var _checked = "checked";
}

@Html.CheckBoxFor(m => m.Test, new { @checked = @_checked })
</code>
Of course, these doesn't work. I'm just trying to eliminate a bunch of <code>@if</code> conditions in my Views, but I have other lightweight logic uses for using variables. My problem might be more of how to use a variable in this way than actually assigning it?
Answer: It would seem that you're understanding razor fine.
The problem with your code seems to be that you're using a variable out of scope. You can't define a variable inside an if statement, and then use it outside, because the compiler won't know for sure that the variable outside actually exists.
I would suggest the following:
<code>@{
var thing = "something";//variable defined outside of if block. you could just say String something; without initializing as well.
    if(condition){
        thing = "something else";
    }
}
@Html.Raw(thing);//or whatever
</code>
As a side note, (in my opinion) it's better to do stuff in the controllers when you can, rather than the views. But if things make more sense in the views, just keep them there. (-:
Hope this helps.
Answer: Try this;
<code>@{
    var disabled = string.Empty;
    if (true)
    {
        disabled = "disabled";
    }
}
@Html.CheckBoxFor(m => m.RememberMe, new { @class = "form-control " + disabled })
</code>
Thanks!
Comment: Saranga, I know your answer is right and it helps me out. Thank you. I see now that (at least in my example) I assigned the variable within the `if` block as Jack pointed out. I'm not sure I did it that way when I was actually testing. Anyway, thx.
Comment: I'm happy if it helped you rwkiii.