Treating ordinal variables as continuous for regression problems

Question: In the social sciences I have encountered that it is common to treat ordinal variables as continuous, for example variables originating from rating or Likert scales (strongly disagree, disagree, agree, strongly agree).
This topic has been discussed for example in this post from 2010:
Under what conditions should Likert scales be used as ordinal or interval data?
I am looking for a more formal comparison/evaluation especially in the context of regression modeling. Rhemtulla et al. (2012) examine the performance of treating ordinal variables as continuous and make recommendations for structural equation models (SEM). I am not very familiar with SEMs, so I'm not sure if their results would also apply to regression problems.
Does anyone know about similar studies/literature in the context of regression?
Edit:
Just to answer the question below:
I'm mainly interested in the case where the outcome variable is ordinal (with possibly an ordinal covariate).
Comment: It's not clear to me whether your focus is on ordinal variables as (1) responses or outcomes (2) predictors or explanatory variables (3) either. (2) is easiest as you just compare predictions for different representations of the ordinal variables. (1) and (3) are difficult because you have to compare quite different kinds of models with quite different kinds of prediction. However, even if you treat ordinal responses as continuous that makes your life easier until you have to work out what a prediction of 4.5678 means when your categories are 1,...,5. (Or 1,...4!).
Comment: I guess it is (1) and (3). But mainly (1).
Comment: Another aspect is the number of categories. Without reference, I think there is some simulation research that has shown Likert scales with more than 5 categories can often be treated as continuous. If I remember the reference, I will post it below. Nevertheless, can you explain your major reason why you do not want to model the outcome variable by ordinal regression such as proportional odds models?
Comment: @Scortchi I did not know about that R package.  Wonderful.  tomka if it is $Y$ that is ordinal then the proportional odds, proportional hazards, or other ordinal models may indeed be excellent choices.
Comment: Re @Frank's suggestion of penalizing predictors towards ordinality: see the reference given [here](http://stats.stackexchange.com/questions/77796/coding-for-an-ordered-covariate/77827#77827)
Comment: @ tomka: I do not really have a reason for not using ordinal regression, actually I would prefer to use it. I was just wondering, because many of the studies I found in psychology do indeed treat ordinal outcome variables as numeric without giving a proper justification. I can see though that it might be easier for researcher to interpret the results of a linear regression than that of an ordinal regression.
Comment: @ ThomasKlausch   May I suggest the following reference for the 5 categories rule-of-thumb?                                                              Rhemtulla, M., Brosseau-Liard, P. E., and Savalei, V. (2012). When Can Categorical Variables Be Treated as
Continuous? A Comparison of Robust Continuous and Categorical SEM
Estimation Methods Under Suboptimal Conditions. Psychological Methods, 17(3): 354-373. It is actually mentioned above. Never mind.
Answer: It's important to distinguish, as pointed out by Nick Cox, between iV and dV. As far as dV is concerned, why not use a ordinal regression model, as discussed excellently e.g. by Agresti: http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470082895.html
I am less sure about the iV case. Standard would perhaps use dummy coding. I suppose this is what Frank Harrell means. Maybe Agresti discusses this as well.
Comment: The abbreviations iV and dV are fairly easily decoded, but (minor point) IV usually means "instrumental variable" to many economists and some others (major point) perpetuate the awful, if still popular, terminology of dependent and independent variables. "Independent" variables usually aren't. At an elementary level, many people still get confused about which is which.
Answer: With the luxury of time we would use dummy variables as with nominal predictors, then penalize them (penalized MLE) towards ordinality of effects.  Something like that was discussed in a paper by Hans van Houwelingen some years ago.  Short of that, we often approximate the effect of ordinal variables by fitting a quadratic effect.  It would also not be ridiculous to use AIC to select between a regular nominal dummy variables model and a restricted model that assumed the ordinal predictor was continuous (like the quadratic).
I'm not sure that the SEM results would apply, but they might.
Answer: I have one source, Snijders and Bosker`s (2012) multilevel analysis book, page 310, saying:
"if the number of categories is small (3 or 4), or if it is between 5 and 10, and the distribution cannot well be approximated by a normal distribution, then statistical methods for ordered categorical outcomes can be useful"
My understanding is, if you have at least 10 categories and approximately normally distributed dependent variable, it is safe to treat it as a continuous variable. For a more concrete answer, I would run a small scale simulation analysis.
Comment: It's hard to see how one could implement this advice (which you repeat *verbatim* in [an answer to a different kind of question involving *counted* responses](https://stats.stackexchange.com/a/407829/919)) until the categories have been assigned meaningful numerical values.  Providing exactly the same answer to different questions  suggests you ought to consider modifying or qualifying at least one of the answers.  If you truly believe this answer is correct and appropriate for both questions, then the right response is to flag one of the questions as a duplicate of the other.
Comment: See my attempt and let me know if it satisfies your concern.
Answer: If an outcome is ordinal, one should want a method of analysis that is invariant to the codes used to label the levels. For example, suppose the outcome has levels : SD, D, N, A and SA. Then one might label the levels with codes 1, 2, 3, 4, 5. If one analyzes this outcome with a t-test, the p-value is invariant only to location or scale changes. For example, -2, -1, 0, 1, 2 or -4, -2, 0, 2, 4. The p-value from the t-test is not invariant to other codings like -10, -1, 0, 1, 100 or any coding that does not preserve 'distance'. The proportional odds model and the multinomial model give p-values that are invariant to the coding selection. [maybe this point has been made earlier?]
Answer: Liddell & Kruschke (2018) is another source which discusses problems associated with treating ordinal data as continuous.  The paper illustrates a number of the problems that can occur.
They advocate using ordered-probit models to deal with ordinal data.  While they specifically advocate for a Bayesian approach, they note that Frequentist approaches may also work


Infinite Series $\sum^{\infty}_{k=2}\frac{1}{k^2\ln(k)}$, Every Test Fails!

Question: So we have this infinite series
$$\sum^{\infty}_{k=2}\frac{1}{k^2\ln(k)}$$
It is apparent that $\ln(k)<k$ for $k>2$
Therefore we can compare
$$\sum^{\infty}_{k=2}\frac{1}{k^2\ln(k)} \text{ to } \sum^{\infty}_{k=2}\frac{1}{k^2(k)}=\sum^{\infty}_{k=2}\frac{1}{k^3}$$
Which is a convergent $p$ series where $p>1$
Establishing comparison:
$$x^2\ln(x)<x^3,\:\frac{1}{x^2\ln(x)}>\frac{1}{x^3}$$
Therefore comparison test fails $a_n>b_n$ where $b_n$ converges
Limit comparison test:
$$\lim_{k\to\infty}\frac{a_n}{b_n}=\lim_{k\to\infty}\frac{\frac{1}{k^2\ln(K)}}{\frac{1}{k^3}}=\lim_{k\to\infty}\frac{k}{\ln(k)}$$
We must use L'Hopitals Rule, let $f(x)=\frac{x}{\ln(x)}$
$$\lim_{x\to\infty}f(x)=\lim_{x\to\infty}\frac{x}{\ln(x)} = \lim_{x\to\infty} \frac{1}{1/x} \to\infty$$
$b_n$ is convergent, so the limit comparison test fails.
According to Wolfram, the ratio test fails, and $a_n$ cannot be analytically integrated (as far as I can tell). However, Wolfram states that the series IS convergent.
What test can I use? Or a trick to prove convergence would be appreciated.
Thanks Stax!
Comment: Just compare to $\sum\frac{1}{k^2}$.
Comment: Here is the argument: $\ln k > 1$.
Comment: That is a lot easier than I imagined. What argument can be used to prove that that series is comparable?
Comment: $\log(k)\ge 1$ for $k$ large enough
Comment: Thanks guys, didnt expect it to be that simple
Answer: One may write, for $k=3,4,\ldots$,
$$
\frac{1}{k^2\ln(k)}<\frac{1}{k^2}
$$ and conclude.
Comment: @helpmeh: Whether a series converges or diverges only depends on the "tail" behavior. The first few terms (indeed, any finite number of terms) are irrelevant.
Comment: @helpmeh You may observe that $$0<\sum_{k=2}^\infty \frac{1}{k^2\ln(k)}=\frac{1}{4\ln 2}+\sum_{k=3}^\infty \frac{1}{k^2\ln(k)}<\frac{1}{4\ln 2}+\sum_{k=3}^\infty \frac{1}{k^2}<\infty$$ giving the convergence.
Comment: Since my series starts at k=2, can this still be used?
Answer: Not quite:
$$\dfrac1{k^2\ln k}=o\Bigl(\frac1{k^2}\Bigr),$$
hence it converges.
Answer: By the Cauchy condensation test, its convergence is equivalent to
$$ \sum_{k=1}^\infty 2^k \frac{1}{2^{2k} \log 2^k} = \sum_{k=1}^\infty \frac{1}{k 2^k \log 2}
$$
Then use the root test.
Comment: above my current scope of infinite series, but thank you
Comment: This one seems more complicated than what is needed for the occasion. $\qquad$
Answer: How about the integral test for convergence?
Note that $$\int_2^\infty \frac{\text{d}x}{x^2\ln x}$$ converges
and thus your sum converges.


Domain Controller Upgrade 2016 to 2022

Question: There are 3 Domain Controllers in my environment, all running Windows Server 2016 Standard:

Primary DC : On-prem (FSMO role)
Secondary DC : On-prem
Tertiary DC : Cloud

I want to upgrade The domain environment to Windows Server 2022. I have tried to direct upgrade from mounted ISO. This didn't work and returned BSOD errors "SYSTEM THREAD EXCEPTION NOT HANDLED".

I have tried by removing all third party applications, change CPU count and run memory diagnostics but this did not resolve the problems.
I decided to install another DC and replace the secondary domain controller. But I need to keep it same as secondary domain controller IP and Hostname. What is best practice achieve this?
Comment: Install new, move FSMO and your done, why trying a useless inplace upgrade for a DC?
Comment: `They need to keep it same as secondary domain controller IP and Hostname.What is best practice achieve this?` Demote second DC, format hard drive, install Windows, promote to DC using the same name and IP address.
Comment: They same hostname and ip can be tricky. Why do they need to keep the same ip and hostname? And can you still logon to that dc that has issues?
Comment: any idea what are the steps to follow clean all the record of old domaincontroller records. or just domain controller from AD?
Comment: Just demote it, that action should clean everything up. But, before doing any stuff like DC promotion/demotion, check consistency with at least `repadmin /replsummary`.
Answer: If you have still access to that failed domain controller, you can use this powershell script to demote it
# Import the ADDSDeployment module
Import-Module ADDSDeployment

# Demote the server
Uninstall-ADDSDomainController -DemoteOperationMasterRole:$true -RemoveDnsDelegation:$true -Force:$true

# Run the following command after the reboot to remove the Active Directory Sites and Services
Uninstall-WindowsFeature AD-Domain-Services -IncludeManagementTools

Then reboot and remove it from sites and services.
https://lazyadmin.nl/it/demote-domain-controller/
Then wait a half hour so to make sure everything is replicated.
Then turn off the dc that you just have demoted, and make sure there are no dns records left for that machine. Also run dcdiag /v to make sure your domain is still healthy.
Then spin up a new machine, maybe good to sysprep it, assign it the proper ip, and set the primary dc as primary dns server in the IP configuration. Then change the name and reboot and then after the reboot start the dcpromo.

Domain Controller Upgrade 2016 to 2022


scrollorama reverse scroll issue

Question: Trying to animate a pic on scroll using superscrollorama!
My issue is when I scroll back to the top, it doesn't work!
The code I'm using is:
<code><script>
$(document).ready(function() {
var controller = $.superscrollorama();
// individual element tween examples
 controller.addTween('#logo_3', TweenMax.fromTo( $('#logo_3'), .8, {css:{opacity:1,       top:'50px'}, immediateRender:true, ease:Quad.easeInOut}, {css:{opacity:0.5, top:'400px'}, ease:Quad.easeInOut}));

controller.addTween('#logo_2', TweenMax.fromTo( $('#logo_2'), 1, {css:{opacity:0.5, top:'0px'}, immediateRender:false, ease:Quad.easeInOut}, {css:{opacity:1, top:'600px'}, ease:Quad.easeInOut}));
        });
</script>
</code>
any idea?
thanks in advance!
Answer: i got the same issue , i fixed it by giving pixel  instead to div
instead of
<code> controller.addTween('#logo_3', TweenMax.fromTo( $('#logo_3')
</code>
use pixel where tween will fire like
<code> controller.addTween(10, TweenMax.fromTo( $('#logo_3')
</code>
10 is the pixel when tween will fire
Answer: So the good news is I know what the problem is,  I found a bug in superscrollorama that when animations happens at 0 pixels may not always reverse correctly because of a line of code problem.
https://github.com/johnpolacek/superscrollorama/pull/105
Is my suggested fix for the issue.  Specifically my issue was happening with pins but I see other places where this could happen with other animations.  This boils down to a < vs <= on 0 pixel items.  So be glad it isn't your fault but my proposed fix is still in discussion.

scrollorama reverse scroll issue


TouchJSON and SOAP

Question: How can i make a SOAP call using TouchJson?
it just work fine with REST call.
Answer: You're missing some fundamentals here. SOAP is XML based. JSON is an alternative markup to XML that's more compact. Making a "REST call" simply means you are requesting a resource. If the resource is returned in a JSON format, it makes sense that it works for you using TouchJSON. If you are calling out to a SOAP web service, however, it is not going to return JSON but rather XML that needs to be made into an object using a SOAP library.
Best regards.

TouchJSON and SOAP


VB. NET: LINQ to SQL equivalent to Count - group by query

Question: I'am trying to convert this query (already working):
Select building.Name, Count(people.ID) as NumberOfUser
From tblBuilding as building left outer join tblPeople as people on building.ID = people.buildingID
Group by building.Name

which gave the result like this:
Name          | NumberOfUser
----------------------------
WestBuilding    50
EastBuilding    70
SouthBuilding   0
NorthBuilding   2

To Linq to SQL. Look at my trying:
Dim db As New MyDatabaseDataContext
Dim query = From building In db.tblBuilding
            From people In db.tblPeople.Where(Function(x) building.ID = x.buildingID).DefaultIfEmpty
            Group building, people By building.ID, building.Name Into grp
                Select ID, Name, grp.Count
Return query.ToList

It appeared to work at first. But when I checked a building that contained no people, the Count column said there is 1. Like this:
Name          | NumberOfUser
----------------------------
WestBuilding    50
EastBuilding    70
SouthBuilding   1
NorthBuilding   2

Because although that building had no people, it did appeared once in the result query so it counted as 1. In another word, the Count statement actually count how many time each building appear in result table instead of how many people in each building, thus making the result wrong for no people building.
Any work around ?
Comment: Your SQL query use `LEFT JOIN` and `COUNT()` does not counts `NULL` values. This is the issue. Solution is already given.
Answer: You are correct, to solve it you need to test whether there are people, like you did with count(people.Id)
Dim db As New MyDatabaseDataContext
Dim query = From building In db.tblBuilding
            From people In db.tblPeople.Where(Function(x) building.ID = x.buildingID).DefaultIfEmpty
            Group building, people By building.ID, building.Name Into Group, Count(Not IsNothing(people))
'Select ID, Name, grp.Count
Return query.ToList


What does "Time on" mean in the context of a Lead Sheet?

Question: I was looking though my real book and saw an interesting phrase that I've never seen before in the song Stuff by Miles Davis. On the first and last bar there is the phrase "Time on D♭7(#9)" with what looks to be a multi-measure rest as seen below.

So what exactly does "Time on D♭7(#9)" mean and what exactly does it convey in this song?
Comment: Just listening to it seems like it means something like "rhythm section vamp/intro on that chord for about 14 measures", or perhaps exactly 14 measures (I didn't count). It definitely starts on a single chord vamp and then the changes begin when the trumpet comes in.
Comment: Agree with Todd.  I would interpret as time spent on D♭7(#9) = 14 measures.
Answer: "Time" in this context means that the rhythm section is to play the song's groove, and then the chord Db7#9 is given as an indicator of what the harmonic instruments will be playing around. Depending on what the groove is, i.e. the feel of the tune, the piano player might improvise some material for those 14 bars.
So the correct interpretation of this is "Play time, on chord X", rather than "Time On".
And of course as Todd mentioned in the comments, some informed perspective could definitely be gained from listening to recordings of the tune.


What is the difference between Consumer and Client SSD?

Question: Does anyone know what are the differences between Consumer and Client SSD :
https://www.samsung.com/us/computing/memory-storage/solid-state-drives/ssd-970-pro-nvme-m2-1tb-mz-v7p1t0bw/ versus
https://www.samsung.com/us/business/products/computing/ssd/client/970-pro-1tb-mz-v7p1t0e/
https://www.samsung.com/us/computing/memory-storage/solid-state-drives/ssd-860-pro-2-5--sata-iii-2tb-mz-76p2t0bw/ versus https://www.samsung.com/us/business/products/computing/ssd/client/860-pro-2tb-mz-76p2t0e/
Answer: Your two exampels earch are identical - one is a product presentation, the second link a shop, but the products on each pairs are EXACTLY THE SAME. Some marketing guy came up with different terms on different websites, that is all. Once you look into the specs both are Client PC usage and both are basically the same product (obviously 2 products, but I go with your pairs here).


Prove $|x+y| \leq (1+x^2)(1+y^2)$.

Question:
Prove $|x+y| \leq (1+x^2)(1+y^2)$.

I don't know if this is true, but I'm trying to build a proof with this property.
I know this is true when both $x,y \geq 1$ and $x,y =0$ as well as $\vert x+y\vert < 1$. But the case when $1 < \vert x+y\vert < 2$ is what's giving me trouble. If this property is false, could you provide a counterexample?
(Sorry for the tag, not sure what else to put it under).
Answer: By C-S
$$\sqrt{(1+x^2)(y^2+1)}\geq\sqrt{(x+y)^2}=|x+y|.$$
Also, by AM-GM and C-S we obtain:
$$(x^2+1)(y^2+1)=x^2y^2+x^2+y^2+1\geq x^2+y^2+1\geq$$
$$\geq2\sqrt{x^2+y^2}=\sqrt2\sqrt{(1^2+1^2)(x^2+y^2)}\geq\sqrt2|x+y|\geq|x+y|$$
Comment: What do you mean by C-S?
Comment: @Young Chin It's Cauchy-Schwarz inequality.
Comment: Ahh thank you. Haven't heard of Cauchy-Schwarzinequality before.
Comment: @Young Chin If so it's $(x^2+1)(y^2+1)-(x+y)^2=(xy-1)^2\geq0.$
Comment: I think you need to use the fact that $x\ge \sqrt{x}$ when $x\ge 1$ in the first solution so you get $(1+x^2)(1+y^2)\ge \sqrt{(1+x^2)(1+y^2)}\ge |x+y|$
Comment: @timon92 I saw it, but I posted, which you see.
Answer: Denote by $\langle u,v \rangle = \sum_{j=1}^2 u_j v_j$ the scalar product of two vectors $u=(u_1,u_2)$, $v=(v_1,v_2) \in \mathbb{R}^2$. As
$$x+y = \langle \begin{pmatrix}x \\ 1\end{pmatrix}, \begin{pmatrix} 1 \\ y  \end{pmatrix} \rangle$$
it follows from the Cauchy Schwarz inequality that
$$|x+y|  \leq \sqrt{x^2+1} \sqrt{y^2+1}.$$
Since $\sqrt{z} \leq z$ for any $z \geq 1$, this implies that
$$|x+y| \leq (x^2+1)  (y^2+1).$$
Answer: Consider that $0 < \frac{1}{2} + (x - \frac{1}{2})^2 + (y - \frac{1}{2})^2 + x^2y^2$ since every term on the RHS is $\geq 0$.

Prove $|x+y| \leq (1+x^2)(1+y^2)$.


Can an original composition be an arrangement?

Question: Sometimes I'll see on discogs.com a song will say string arrangements by so and so... and it's an original song by them.

Can original compositions be arrangements?
What makes an arrangement?
Can  it be like making the song you composed in a certain style of music or genre than your first draft?

Cambridge dictionary defines arrangement as:

a piece of music that has had changes made to it [so] that it can be played in a different way, especially by different instruments

If the piece of music can be played in a different way, isn't that also considered an original piece?
Comment: *arrangement* : "3. a composition adapted for performance with different instruments or voices than those originally specified. 'Mozart's symphonies in arrangements for cello and piano'" – Google
Answer: A "composition" is traditionally thought of as being the most essential musical idea of the song, mostly its melody. Arranging is the act of deliberately modifying or adjusting or adopting a composition. You tweak or refine some aspects of a composition. When you go and realize a song with several instruments, for example a string section, it involves many decisions about what notes and how exactly to play ... and these decisions may or may not be trivial. AFAIK it's up to the composer to decide whether or not the arranging work was substantial enough to warrant crediting the arranger for the work. For example harmony progressions can add significant dimensions to a melody. If your melody is just a few notes of a pentatonic scale, it can be harmonically quite colorless and odorless and bland, and to make it feel like anything you have to make up some harmonic movement. Is that an arrangement? If the composer or "composer" refuses to acknowledge it as an arrangement, tough luck.
Arranging credits have to do with how royalty money is split between people, so they might be used just as a way to get a certain split. Then again, what's printed on the record cover might not necessarily be the same as what's actually written in official information records. "You can call yourself an arranger on the record cover, but I won't give you a penny."
If you see "string arrangement by ..." or "horn arrangement by ..." on a record, it might mean that the band couldn't do those arrangements by themselves and they hired a string or horn specialist to do it.
Answer: I suppose it depends on how far you stretch the definition of 'arrangement'.
If I were to write a brand new song & sing it to you without any accompaniment, that could qualify as having been 'arranged for solo voice'.
Let's take the simple yet rather beautiful Yesterday, technically by The Beatles, but really written & performed by Paul McCartney… with string arrangement by George Martin

The 'song' as such is by Paul, arranged for solo voice & acoustic guitar.
When he first took it to the studio, that's all it was. By the time they left, however, it included 2 unison voices in some parts, & a simple yet very effective string quartet arrangement.
If someone were [gods forbid] to make a heavy metal version of that song, it would be still the same song, but arranged for heavily distorted guitars, bass, drums & probably a shrieking vocal ;)
It is widely known that George Martin - often called the fifth Beatle - was responsible for taking some of the simpler ideas & arranging far grander surroundings for them.
He never received official credit for any of this, though he is, of course, credited as Producer.
Comment: Definitions of words don't change depending on how far you stretch them, +1
Answer: An original piece could be anything. Words, music and chord chart up to a full blown symphony.
What happens after will be the arrangement. It could have extra instrumental parts added, could be re-arranged with parts in different orders, changes of key, extra bridges, etc.
Answer: You can change aspects of one of your previous original compositions and create a new original composition that's also an arrangement. One example is Franz Liszt's Transcendental Etudes, which are slightly easier arrangements of an earlier set of extremely difficult etudes Liszt composed...which are themselves expanded reworkings of an even earlier set of etudes Liszt also composed. (Liszt has a habit of rearranging his earlier works.)
Video game themes arrangement-wise can do anything from quote each other to be extensions of each other to be pitch-shifted versions of each other (and probably more). I can come up with tens of examples of same-series video game theme rearrangements (one example is the sheer number of times the Kirby series has remixed King Dedede's theme).
Answer: Different (but connected) meanings of the word 'arrangement'.
The original way a song is put together - the instrumentation, the verse/chorus structure can be called 'the arrangement'.  Maybe a songwriter came up with just a melody and lyrics then handed it over to an 'arranger' to flesh it out.  Maybe the performers worked out this aspect of 'the arrangement' themselves and just brought in an expert 'string arranger' to add string parts.
Or maybe the song is finished, recorded and hits the charts.   So the publisher hires an 'arranger' to make a version for school bands to play, or one for the Boston Pops.  These are 'arrangements'.
We could also discuss the difference between 'arrange' and 'orchestrate'.  Another time :-)
Answer: I'll add one thought that isn't already represented here. You added the Cambridge definition of "arrangement" that emphasizes that a piece has "had changes made to it," and ask:

If the piece of music can be played in a different way, isn't that also considered an original piece?

Simple answer: nope. But wait: can you make enough changes that it is no longer the same piece? Of course. Change all the notes and it's now a different piece. So how much change is necessary for it to count as an original piece?
That's a question that can't get a hard and quantitative answer, but the distinction exists. Vivaldi's Four Seasons has been arranged for probably every instrument by now, and most of those count as "arrangements." But Max Richter deconstructed the elements of Vivaldi's work and recombined them in the work Recomposed by Max Richter: Vivaldi – The Four Seasons. Although he uses Vivaldi's material, and one might not call the resulting work an "original composition," it is unarguably a new and distinct work.
Wendy Carlos, in Switched-On Bach, recorded J.S. Bach works on Moog synthesizers. (Disclaimer: I'm not intimately familiar with the album, and copyright issues make it difficult to access. My perception is that she didn't significantly add to Bach's works, e.g. adding drum beats etc., simply arranged them.) While many shocked purists would deny that the result is "the same thing," this seems like a clear case of arrangement. (A similar experiment is "Cans and Brahms," in which Rick Wakeman straightforwardly crams the third movement of Brahms' fourth symphony into three keyboards (or is it four?), changing few if any notes in the process. The impact is certainly different than a symphony orchestra, but it's hardly an "original composition."
What about remixing? When a band writes a head-banging hit and then, 20 years later, releases a gentler easy-listening cover so their aging fan base can shop to it, is that an arrangement? (Probably.) If their song is chopped up and glued back together in a techno remix, is it an arrangement? (Arguably not.) Is it an original composition? (Uh... arrrguably not? It's a derivative work.) What about mashups? When Danger Mouse combines the Beatles' White Album and Jay-Z's Black Album to create The Grey Album, is that a new work? Well, all sorts of legal teams got excited about the question, but from an aesthetic standpoint, I think the combination is clearly a different "thing" than the two component "things." It's certainly not an "arrangement"!
Answer: Yes, if you use the word "arrangement" in the same sense as "orchestration." But, the usage of those terms can be a bit mixed.
"Orchestration" is the art of selecting instruments in a composition. "Arrangement" is adapting existing music for different instruments than the original composition. Those are just basic definitions.
But sometimes "orchestrate" is used for an arrangement when a full orchestra is used in the arrangement, such as Ravel's orchestration of Mussorgsky's Pictures at an Exhibition or Debussy's orchestration of Satie's Gymnopedie. Those are certainly arrangements, but the credit is often stated as "orchestration by..."
And sometimes we have the reverse. As you say, "arrangement" is often used in regard to an original work, like in Stevie Wonder's album Song in the Key of Life. His credits read: "produced, arranged, and composed by..." So, in a song like Sir Duke the brass is not an arrangement of some previously existing version of the music for different instruments. The whole conception was Wonder's, including the use of the instruments, but the word "arrange" is used instead of "orchestrate." I suppose the reason "arrange" is used instead of "orchestrate" is because an orchestra - apparently synonymous with string instruments in the pop world - was not used.
I think this is the real answer to your question: "arrange" is often used synonymously for "orchestrate."
I think it's worth mentioning the uncredited contribution of arrangers, especially  because the meaning of the word "arrange" it tied up with the idea of what is original creation. Many times an arrangers contributions are vaguely credited or simply completely uncredited despite contributing original musical content, especially melodic content. For example the credits for the strings in the recording of Buddy Holly's True Love Ways are simply "orchestra directed by Dick Jacobs." But the strings aren't simply doubling the vocal part or fleshing out a chord progression, they contribute unique melodic material with elaborate interplay with the vocal. In cases like those the arranger isn't just orchestrating there is significant original composition too.
Answer: I think the word original composition and composer would be too triumph to describe the pathetic effort  of any pop song producer today.  Song writer seems more adequate to me. The setting of a song for an orchestra or a band can be called an arrangement, not more than this.
Answer: No. Arrangement is arrangement, original composition is original composition.
You can use same methods and techniques, but in arrangement you have a pre existing material per se. But you can queue and quote other pieces in an original composition by the way.


Case-crossover method with conditional logistic regression for poisson model generated data

Question: I would like to run a semi-symmetric bi-directional case-crossover method on some generated data using conditional logistic regression.
I generated data from Poisson distribution Poiss(lambda) with
$\textrm{log}(\lambda)=\lambda_0 \textrm{exp}(x\beta), $
where $\lambda_0$ and $\beta$ are known and $x$ is a known time series. Then for each timepoint I split generated value into pairs: case in this timepoint and control in referent period (that is if in generated data there are $5$ cases at day $11$, I get $10$ observations for this timepoint: $5$ $(1$, $x$ at day $11)$ and $5 (0$, $x$ at reference period for day $11)$). Now I run conditional logistic regression on this data, with separate stratum for each pair.
The question is, should the parameter estimates that I get from CLR be $\beta$ from parametrer of Poisson distribution? I found few articles with this procedure applied, but I can't figure out why estimetes from CLR and $\beta$ from Poisson distribution should be the same.
Reference articles:

T.Bateson, J.Shwartz, Control for Seasonal Variation and Time Trend in Case-Crossover Studies of Acute Effects of Environmental Exposures, Epidemiology 1999, 10(4), 539-544;
S.Wang, B.Coukk, J.Shwartz, M.Mittleman, G.Wellenius, Potential for Bas in Case-Crossover Studies With Shared Exposures Analyzed Using SAS, American Journal of Epidemiology 2011, 174(1), 118-124;
K.Fung, D.Krewski, Y.Chen, R.Burnett, S.Cakmak, Comparison of time-series and case-crossover analyses of air pollution and hospital admission data, International Journal of Epidemiology 2003, 32, 1064-1070;

Thanks for help
Answer: In a case-crossover study, the referent time periods are randomly sampled from the person-time at risk of each individual. Therefore, the odds ratio in a case-crossover estimates the incidence density ratio that would have been obtained in a cohort study - it is just an efficient sampling scheme, similar to an incidence density or risk set sampled case-control study, albeit one that doesn't require any reference people and automatically accounts for between-person confounding and within-person non-time-varying confounding. So, when you use conditional logistic regression your $\beta$ estimates are odds ratios which are in turn estimates of the incidence density ratios, which is what the $\beta$ parameters of your Poisson time-series are.


How do I properly use a function under a class?

Question: I am currently learning Python. Since I am a big fan of OO (object-oriented) programming, obviously it's not hard to apply it in Python. But when I tried it, it seems very different to C#.
As you can see below, I am trying to create a character class, with three attributes Id, Hp, and Mana. The score is calculated by adding up Hp and Mana and then times 10.
As you can see, after defining MyChar where id=10 hp=100 mana=100, I was expecting MyChar.Score is (100+100)*10, which is 2000, but weirdly, it says:
bound method Character.Score of <__main__.Character object at 0x0000021B17DD1F60> as the result of print(MyChar.Score).
How can I fix this problem?
Here is my code:
class Character:

    def __init__(self, Id, Hp, Mana):
        self.Id = Id;
        self.Hp = Hp;
        self.Mana = Mana;

    def Score(self):
        return (self.Hp + self.Mana)*10;

MyChar = Character(10, 100, 100);

print(MyChar.Score)

Comment: Score is not an attribute but a member function, invoke it like print(MyChar.Score())
Comment: I would swear that this kind of functions are also called methods and are invoked with `()` in C#.
Comment: @KunalMukherjee yes it is - the `MyChar.Score()` expression _first_ resolves the `"Score"` attribute on `MyChar` object (yielding a method object), _then_ applies the call operator (the parens) on it.
Comment: @Kunal It *could* be, though. This is especially useful when passing functions/methods into higher-order functions such as map/filter. :)
Comment: @Goyo you may want to read this about what Python "methods" really are: https://wiki.python.org/moin/FromFunctionToMethod - as a general rule, Python's object model is wildly different from C#'s one, so while you'll find the same basic concepts of class, instance, attribute, method etc, you won't have a 1:1 mapping with the way C# implement those concepts.
Comment: @KunalMukherjee please check your facts - `Score` IS an attribute (of the `MyChar` class object).
Comment: @brunodesthuilliers I mean its not meant to be invoked as an attribute
Comment: But in the context of OP its not meant to be called without parenthesis / passed around
Comment: @brunodesthuilliers Yet I always see `()` being used to call methods in C#. I might be missing something though.
Comment: @Goyo I'm afraid I didn't read your comment properly, sorry.
Comment: You don't need the semicolons at the end. Your output, by the way, was `__repr__(method)`
Comment: @Goyo I'm guessing OP expects `Score` to work as a get-property.
Comment: @JAD I realize that, I just don't think that mistaken expectation can be related to how Python "seems very different to C#" because as far as I know they work pretty much the same in this particular case.
Answer: If you want to use it like a property in C#, decorate the function with @property, like so:
class Character:

    def __init__(self,Id,Hp,Mana):
        self.Id=Id;
        self.Hp=Hp;
        self.Mana=Mana;

    @property
    def Score(self):
        return (self.Hp+self.Mana)*10;

MyChar=Character(10,100,100);

print(MyChar.Score)

So you don't have to call it like a function.
For more advanced usage of properties (e.g. also having a setter func), see the official docs: https://docs.python.org/3/library/functions.html#property
Comment: While that's a nice suggestion, it doesn't really answer the OP's question.
Comment: While the title does ask how to use the function, the body simply asks "How can I fix this problem?"  It seems that either `MyChar.Score()` or `@property def Score` cause the print statement to work as OP wanted, and this approach has the advantage of encapsulating more logic into the class definition.
Comment: @brunodesthuilliers I don't understand.  This adds a single line (the decorator) to OP's example and fixes the problem.  How does this not answer the question?
Comment: @Teepeemm  The OP asked why the expression `MyChar.Score` didn't eval to the return value of the `Score` method, not how to make it a computed attribute instead of a method.
Answer: tl;dr
Use it like any other function by calling it: print(MyChar.Score()) (note the additional pair of parentheses).

As you've correctly stated, MyChar.Score is a "function under a class" (aka "method"). So just use it like any other function by calling it: suffixing it with a pair of parentheses.
print(MyChar.Score())
#                 ^^

Without the call, simply doing print(MyChar.Score) prints <bound method blah blah>, i.e. the informal string representation of the method. The print function internally calls __str__() magic method (or __repr__(), if the former isn't defined). Hence, the following print equivalent lines:
print(MyChar.Score.__str__())
print(str(MyChar.Score))
print(MyChar.Score.__repr__())
print(repr(MyChar.Score))

In Python, functions are first-class citizens, hence they are objects and have  the __str__() and __repr__() methods.
Comment: Without the parentheses, you were not calling the function. Instead, you were printing a representation of the function itself, which is a method of the character class called 'Score'.
Answer: In Python, everything is an object, including classes, functions and methods, so MyChar.Score (without the parens) only resolves the Score attribute on MyChar object. This yields a method object, which happens to be a callable object (an object that implements the __call__ special method). You then have to apply the call operator (the parens) to actually call it.
You may want to check the official documentation for more on Python's object model.
Answer: class Character(object):
    def __init__(self):
        print ('Starting')

    def method(self):
        print ('This is a method()')
ch = Character()

'''When we don't add the bracket after the method call, it would lead to a method bound error as in your case'''
print (ch.method)
'''This can be solved by doing the following line'''
ch.method()


Optimize $f(x,y,z) = xyz$ restricted to $g(x,y,z)= x^2+2y^2+3z^2= 6$

Question: I'm stuck doing this problem.
Optimize $f(x,y,z) = xyz$ restricted to $g(x,y,z) = x^2+2y^2+3z^2 = 6$
First, I found ${\nabla}f$ and ${\lambda}{\nabla}g$, and for Lagrange Multipliers, I got these four equations
$$ \begin{matrix}
yz = 2{\lambda}x \hspace{8mm} (1) \\
xz = 4{\lambda}y \hspace{8mm} (2) \\
xy = 6{\lambda}z \hspace{8mm} (3) \\
x^2+2y^2+3z^2 = 6 \hspace{8mm} (4)
\end{matrix} $$
But I don't know how to find $x,y,z$ or $\lambda$. I've been trying by summing two of those equiations, but from the expressions I got (which are too complicated to write here) I don't know what to find first.
Thanks for your help
Comment: Try multiplying equations 1,2 and 3 by x, y, and z respectively. Then the left hand sides will all be equal and you can work from there.
Answer: $\textbf{Hint:}$ First check that $\lambda \neq 0$. Next multiply (1) by $x$ and (2) by $y$, you get:
$$xyz=2\lambda x^2$$
$$xyz=4 \lambda y^2$$
So $2 \lambda x^2=4 \lambda y^2$, then $x^2=2y^2$. The same with (1) and (3), next (2) and (3), finally substitute to (4).


How probability distributions help a statistical analyst/data scientist

Question: How exactly the probability distributions help a statistician/data scientist in modelling/decision making?
Or how using distributions a data scientist derive any inference or make decisions when modelling?
I understand probability distributions theoretically but not sure how its utilised in day to day business operations. Some day to day business examples would be very helpful.
Thanks.
Comment: It e.g helps to pick an appropriate loss function
Comment: @Michael M Can you elaborate or provide a link to literature where use of distributions to choose apt loss function is explained
Answer: I suppose you are referring to the standard probability distributions? In that case, an example would be the arrival times for a certain service.
For example, suppose that a hospital has a limited amount of beds to accommodate patients. The hospital wants to know how many beds it should have at any given time to make sure that the probability that they run out of beds is 0.01. In order to model this, we can look at the arrival times of patients. We note that patients do not influence each others arrival times and that during our period of measurement, say, 12 hours from 8 AM to 8 PM, patients arrive randomly.
Now, if we observe the number of patients that arrive say, every 5 minutes, we notice something very peculiar: it follows a Poisson distribution. Finding the amount of beds the hospital needs has now become incredibly easy, since from our observations we can obtain an 'arrival rate', which is the lambda in a Poisson distribution, and with this arrival rate we can compute this probability using this particular distribution.
Comment: Thanks Anon, example is very practical but assumption that addimissons follow poisson may be very simplistic. I had a doubt on application of poisson that it gives very low probability for (calculated using survival function) for higher values of the data. I posted it here https://datascience.stackexchange.com/questions/61485/probabilities-of-a-poisson-distribution-not-making-sense/61498?noredirect=1#comment67004_61498. So the issue is in books we are taught that for admissions or bus arrivals or restaurant orders follow poisson but there are practical issues in Application to real problems
Comment: If you read my question you will understand my concern regarding application of poisson distribution and therefore decision  making based on poisson probabilities may be not the right thing to do. So question now becomes how to fit the right distribution and how much to trust the probabilities for taking business decisions


Mixed Conditioning - Two Normal Distributions

Question:
Let $Z \sim \mathcal{N}(0,1)$ and $Y|Z \sim \mathcal{N}(Z, 1)$.
Show that $f_{Z|Y}(z|y)$ is a normal density, and find the parameters of this density.

What I have so far:
\begin{align*}
f_Z(z) &= \phi(z)\\
f_{Y|Z}(y|z) &= \frac{1}{\sigma} \phi\left( \frac{y - \mu}{\sigma} \right) = \phi\left( y - z \right)\\
f(y,z) &= f_Z(z) \cdot f_{Y|Z}(y|z) = \phi(z) \cdot \phi\left( y - z \right)
\end{align*}
From the above, we can use Bayes' theorem express $f_{Z|Y}(z|y)$:
\begin{align*}
f_{Z|Y}(z|y) &= \frac{ f_Z(z) \cdot f_{Y|Z}(y|z)}{ f_Y(y) }\\
&= \frac{ \phi(x) \cdot \phi(y-z) }{ \int_{-\infty}^\infty f(y,z)\,\mathrm{d}z }\\
&= \frac{ \phi(x) \cdot \phi(y-z) }{ \int_{-\infty}^\infty\phi(z) \cdot \phi\left( y - z \right)\,\mathrm{d}z }\\
\end{align*}
However, I am stuck at this point. I'm not exactly sure how to compute the integral in the denominator, and then express $f_{Z|Y}(z|y)$ as a normal distribution function. Is there a better way to approach this? Thanks!
Answer: $$
f_{Z|Y}(z|y) =
\frac{f_Z(z)f_{Y|Z}(y|z)}{\int_{-\infty}^{+\infty}{f_Z(a)f_{Y|Z}(y|a)da}} =
\frac{\frac{1}{\sqrt{2\pi}} e^{-\frac{z^2}{2}} \frac{1}{\sqrt{2\pi}} e^{-\frac{(y-z)^2}{2}}}
{\int_{-\infty}^{+\infty}{\frac{1}{\sqrt{2\pi}} e^{-\frac{a^2}{2}} \frac{1}{\sqrt{2\pi}} e^{-\frac{(y-a)^2}{2}}da}} =
\frac{e^{-z^2+yz-\frac{y^2}{2}}}{\int_{-\infty}^{+\infty}{e^{-\frac{2a^2-2ya+y^2}{2}}da}}
$$
Now we'll calculate the integral in the denominator:
$$
\int_{-\infty}^{+\infty}{e^{-\frac{2a^2-2ya+y^2}{2}}da} =
\int_{-\infty}^{+\infty}{e^{-\frac{(\sqrt{2}a - \frac{\sqrt{2}}{2}y)^2 + \frac{y^2}{2}}{2}}da} =
e^{-\frac{y^2}{4}} \int_{-\infty}^{+\infty}{e^{-\frac{(\sqrt{2}a - \frac{\sqrt{2}}{2}y)^2}{2}}da}
$$
Defining $b=\sqrt{2}a$ and multiplying and dividing by $\sqrt{\pi}$ we get:
$$
e^{-\frac{y^2}{4}}\sqrt{\pi} \int_{-\infty}^{+\infty}
{\frac{e^{-\frac{(b - \frac{\sqrt{2}}{2}y)^2}{2}}}{\sqrt{2\pi}}db}
$$
Notice that we have the pdf of the normal distribution $\mathcal{N}\left(\frac{\sqrt{2}}{2}y, 1\right)$  inside the new integral. Therefore it equals 1.
Substituting
$
\int_{-\infty}^{+\infty}{e^{-\frac{2a^2-2ya+y^2}{2}}da} = e^{-\frac{y^2}{4}}\sqrt{\pi}
$
into the initial formula we have:
$$
f_{Z|Y}(z|y) = \frac{1}{\sqrt{\pi}}e^{-z^2+yz-\frac{y^2}{2}+\frac{y^2}{4}} =
\frac{1}{\sqrt{\pi}}e^{-(z-\frac{y}{2})^2} =
\frac{1}{\sqrt{2\pi} \frac{1}{\sqrt{2}}}e^{-\frac{(z-\frac{y}{2})^2}{2(\frac{1}{\sqrt{2}})^2}}
$$
That is,
$$
Z|Y \sim \mathcal{N}\left(\frac{Y}{2}, \frac{1}{\sqrt{2}}\right)
$$


Why is fwrite() writing `#!/usr/bin/env php` to my output file?

Question: I've been exploring how to create a downloadable file on my WP plugin page. I've got as far as creating a text file and outputting the string 'Hello?' to it. This happens in the download() method.
It almost works as expected but the downloaded file contains the following line: #!/usr/bin/env php. Why? What do I need to do to prevent that from being written to the file?
The downloaded file looks like this:
#!/usr/bin/env php
Hello?

This is the plugin page's controller:
<?php

defined( 'ABSPATH' ) || die();

class helloSettingsPageController {

    private static $settings_page;

    // scripts,
    public static function init() {
        add_action( 'admin_menu', array( __CLASS__, 'add_menu' ) );
        add_action( 'admin_enqueue_scripts', array( __CLASS__, 'load_scripts' ) );
        add_action( 'admin_action_hello_download', array( __CLASS__,'download') );
    }

    // setup
    public static function add_menu() {
        self::$settings_page = add_menu_page( 'hello', 'hello', 'edit_pages', 'hello', array( __CLASS__, 'render_page' ), false, 62 );
    }

    public static function load_scripts( $hook ) {
        if ( self::$settings_page !== $hook ) {
            return;
        }

        $path = plugin_dir_url( __FILE__ ) . 'js/hello-script.js';
        wp_enqueue_script( 'hello-script', $path, array( 'jquery' ) );
    }

    // view
    public static function render_page() {
        $html = ( 'html/settings_page.html' );
        include plugin_dir_path( __FILE__ ) . '../' . $html;
    }

    // actions
    public static function download() {
        if (!current_user_can('edit_pages')) { wp_die('Permission denied'); }
        $retrieved_nonce = $_REQUEST['_wpnonce'];
        if ( !wp_verify_nonce($_REQUEST['_wpnonce'], 'admin_action_hello_download') ) {
            wp_die('Nonce verification failed!');
        }

        header( 'Content-type: text/csv' );
        header( 'Cache-Control: no-store, no-cache' );
        header( 'Content-Disposition: attachment; filename="text.txt"' );

        $outstream = fopen( "php://output", 'w' );

        fwrite( $outstream, "Hello?");
        fclose( $outstream );

        exit();
    }
}

Answer: It turns out it's nothing to do with the sample code in my question.
I tracked it down to psysh. The hashbang disappears when I removed the psysh include. The problem is documented on github: psysh prints the php hashbang when used as library.


Intuition - Impact of baseline conversion rate on sample size

Question: In an A/B test we calculate the needed sample size before we run the test. The required sample size is dependent on the significance level, the power, the minimum detectable effect (MDE) and the baseline conversion rate. The baseline conversion rate is the percentage of visitors to your website that complete a desired goal that you are observing without changing anything. In other words the conversion rate of the control group.
Let's say we set those value to

Significance level: 5 %
Power: 80 %
Relative MDE: 2 %

And plug them into a sample size calculator.
For different baseline conversion rates we get different sample sizes. The higher the baseline, the lower the sample size.

Baseline 10 %: 354,139 per variant
Baseline 20 %: 157,328 per variant
Baseline 30 %: 91,725 per variant

The relative change we are trying to detect stays the same. I am trying to get an inituition for why we need bigger samples if the baseline is lower.
Comment: For anyone curious, since the page doesn't describe the method, here is the sample size calculation in js:
function num_subjects(alpha, power_level, p, delta) {
    if (p > 0.5) {
        p = 1.0 - p;
    }
    var t_alpha2 = ppnd(1.0-alpha/2);
    var t_beta = ppnd(power_level);

    var sd1 = Math.sqrt(2 * p * (1.0 - p));
    var sd2 = Math.sqrt(p * (1.0 - p) + (p + delta) * (1.0 - p - delta));

    return (t_alpha2 * sd1 + t_beta * sd2) * (t_alpha2 * sd1 + t_beta * sd2) / (delta * delta);
}
Comment: Could you provide a definition of baseline conversion rate as it is an application specific phrase? I had intuitively thought it meant something different to what Google suggested. Also I got very different results plugging in your values into the linked calculator.
Comment: Hey @ReneBt. Baseline conversion rate is the percentage of visitors to your website that complete a desired goal that you are observing without changing anything. In other words the conversion rate of the control group.
For the different values you are getting, did you select "Relative". The default is "Absolute".
Answer: Thanks to khol for supplying the js function in the comments to the OP, it helps to see what exactly this function is doing, although I think it appears to be the one for the absolute change rather than relative unless delta is pre-processed before this function. As an intuitive answer was requested I will not delve too deeply unless requested. I have however provided an explanation of what the function is doing if you want to go that far.
Intuitive Explanation
It might be expected that since higher conversion rates are associated with a higher variance it would require more samples to counter the extra variance. But this is only one side of the calculation.
At 10% background rate and a relative 2% change, your signal you want to detect is 0.47% of the expected variance (noise) under null. At 30% background rate it is 0.93% of the expected variance under null, approximately twice the signal to noise ratio. Since noise scales by the square root of N, this means you would expect a 10% background rate to require about 4x the numbers required for 30% if you want to maintain the same confidence in the result. You will observe this approximates you figures well.
So the intuitive explanation is that a 2% relative change at 10% background rate is much smaller relative to the noise than it would be at 30% background rate. Because it has a smaller signal to noise value you need larger numbers to average out your noise and attain the same level of confidence in your result.
Explanation of the calculation based on code
<code>num_subjects(alpha, power_level, p, delta)
{ if (p > 0.5) { p = 1.0 - p; }
      var t_alpha2 = ppnd(1.0-alpha/2);
      var t_beta = ppnd(power_level);
      var sd1 = Math.sqrt(2 * p * (1.0 - p));
      var sd2 = Math.sqrt(p * (1.0 - p) + (p + delta) * (1.0 - p - delta));
      return (t_alpha2 * sd1 + t_beta * sd2) * (t_alpha2 * sd1 + t_beta * sd2) / (delta * delta); }
</code>
p appears to be the background conversion rate. Line 1 simply inverses the probability if it is >0.5 since it is simply the mirror image (the direction of change is not important).
The second line var t_alpha2 calculates the variance associated with the specified significance level (alpha)
The third line var t_beta is the variance associated with the specified power (power_level)
The fourth line var sd1 is the variance associated with the background rate. Since backround rate (p) is involved this will change as p changes. The closer to 0.5 p is the bigger the resulting sd1. For your values of background rate this will give sd1 = 0.424264069, 0.565685425 and 0.64807407 respectively.
The fifth line var sd2 is the variance associated with the relative change in background rate (i.e. the background minus and plus the relative change). With relative change the absolute value of change is smaller for smaller background rates. So 2% of 10% is 0.2%, for 20% it is 0.4 and for 30% it is 0.6%. This gives values for sd2 of 0.426140822, 0.567788693 and 0.649895376 respectively.
This means the last line returns the square of the sum of the products of (background rate and significance) and (relative change and power) divided by the square of the relative change.
The background rate is your null hypothesis which is why its variance is multiplied with the variance associated with your significance (which is conditional on no change).
The relative change is your alternative hypothesis, which is why its variance is multiplied by the variance associated with your power (which is conditional on the expected change).
All in all this means that if you make the denominator (delta squared) really small then the result will be a big number. Since the relative change squared is 9 times higher (0.000036 compared to 0.000004) at baseline 30 vs 10 %, while the variances SD1 and SD2 only change by a factor of 1.527525232 and 1.525071861 respectively.


Combine distinct list of dict with ansible

Question: I have two list in ansible:
toto:
  - name: titi
  - name: tata

titi:
  - name: titi
    ack: true

Is it possible to combine these two lists by the name key to get the following:
new_list:
  - name: titi
    ack: true
  - name: tata

I found the way to combine dict, to combine list, by I don't know if I can do the following.
Thanks.
Answer: Q: Is it possible to combine these two lists by the name key?
A: Yes. It  is possible with the filter selectattr. The tasks below
- set_fact:
    new_list: "{{ new_list|default([]) +
                  [ item|
                    combine(titi|selectattr('name', 'match', item.name)|
                    list) ] }}"
  loop: "{{ toto }}"
- debug:
    var: new_list

give
"new_list": [
    {
        "ack": true,
        "name": "titi"
    },
    {
        "name": "tata"
    }
]


Content encoding error after enabling cache, aggregation and compression

Question: Yesterday I turned on caching, aggregation of css and js files and turned on compress cached pages. Then I logged out and closed my browser (FF). After a couple of minutes I openend my browser again, typed in the url of my website and got this error in FF: Content encoding error.
I tried opening my site in FF, Chrome and IE. None of them work.
I read some similar threads here and tried:
- setting display_errors=Off in php.ini
- disabling gzip of files in .htaccess
- Changing "Header append" to "Header set" in .htaccess
Nothing seems to work.
What I would like to do now is manually disable caching, aggregation and compression in the database, but I don't know which tables these settings reside in.
Answer: Add this lines to your settings.php file.
$conf = array(
'cache' => '0',
'preprocess_css' => '0',
'preprocess_js' => '0',
'block_cache' => '0',
'page_compression' => '0',
);

or with Drush
// To turn on JS Aggregation
drush vset preprocess_js 1 --yes

// To disable JS Aggregation
drush vset preprocess_js 0 --yes

// To clear cache of JS and CSS only
drush cc css+js

// To enable CSS Aggregation
drush vset preprocess_css 1 --yes

// To disable CSS Aggregation
drush vset preprocess_css 0 --yes

Comment: @Smos Drupal sends these files gzipped to save bandwidth. But your servers messes with http headers. Some hosting companies thinks it's good idea, dunno why :|
Comment: Thanks for the quick response! It works, I can access my site again. Now I still need to figure out what exactly causes the "Content encoding error".
Comment: @Smos  Using Drupal output compression along with Apache mod_deflate can result in double compression.  I don't recall getting a "Content encoding error" with this.  When this happens, I have always seen a page full of junk characters.
Comment: @MPD http://home.pl used to mess my headers even without own compression. Content was compressed, once, but browser didn't get an info about that at all. So this can be screwed up at least 2 ways. But I bet there are more.
Comment: @Mołot  Odd, never seen that one.  Sounds line a bad proxy not honoring Vary: Accept-Encoding
Comment: @MPD there are so many ways to make a problems to your users ;) Fun fact - while Drupal can't be sent from home.pl with any compression, their own main site is gzipped all right. It was a reason of some puns in Polish Drupal related forums, as home.pl is one of the big players in hosting business here.


Inverse Match Help in Python

Question: Hello I am looking to trim a McAfee log file and remove all of the "is OK" and other reported instances that I am not interested in seeing. Before we used a shell script that took advantage of the -v option for grep, but now we are looking to write a python script that will work on both linux and windows. After a couple of attempts I was able to get a regex to work in an online regex builder, but I am having a difficult time implementing it into my script.
Online REGEX Builder
Edit: I want to remove the "is OK", "is a broken", "is a block lines", and "file could not be opened" lines so then I am just left with a file of just the problems that I am interested in. Sort of of like of like this in shell:
grep -v "is OK" ${OUTDIR}/${OUTFILE} | grep -v "is a broken" | grep -v "file could not be opened" | grep -v "is a block" > ${OUTDIR}/${OUTFILE}.trimmed 2>&1

I read in and search through the file here:
import re

f2 = open(outFilePath)
contents = f2.read()
print contents
p = re.compile("^((?!(is OK)|(file could not be opened)| (is a broken)|(is a block)))*$", re.MULTILINE | re.DOTALL)
m = p.findall(contents)
print len(m)
for iter in m:
    print iter
f2.close()

A sample of the file I am trying to search:
eth0
10.0.11.196
00:0C:29:AF:6A:A7
parameters passed to uvscan: --DRIVER /opt/McAfee/uvscan/datfiles/current --    ANALYZE --AFC=32 ATIME-PRESERVE --PLAD --RPTALL RPTOBJECTS SUMMARY --UNZIP -- RECURSIVE --SHOWCOMP --MIME --THREADS=4 /tmp
temp XML output is: /tmp/HIQZRq7t2R
McAfee VirusScan Command Line for Linux64 Version: 188.8.131.524
Copyright (C) 2014 McAfee, Inc.
(408) 988-3832 LICENSED COPY - April 03 2016

AV Engine version: 5700.7163 for Linux64.
Dat set version: 8124 created Apr 3 2016
Scanning for 670707 viruses, trojans and variants.

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/ATIME-PRESERVE

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/RPTOBJECTS

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/SUMMARY
/tmp/tmp.BQshVRSiBo ... is OK.
/tmp/keyring-F6vVGf/socket ... file could not be opened.
/tmp/keyring-F6vVGf/socket.ssh ... file could not be opened.
/tmp/keyring-F6vVGf/socket.pkcs11 ... file could not be opened.
/tmp/yum.log ... is OK.
/tmp/tmp.oW75zGUh4S ... is OK.
/tmp/.X11-unix/X0 ... file could not be opened.
/tmp/tmp.LCZ9Ji6OLs ... is OK.
/tmp/tmp.QdAt1TNQSH ... is OK.
/tmp/ks-script-MqIN9F ... is OK.
/tmp/tmp.mHXPvYeKjb/mcupgrade.conf ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/uninstall-uvscan ... is OK.
/tmp/tmp.mHXPvYeKjb/mcscan ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/install-uvscan ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/readme.txt ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/uvscan_secure ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/signlic.txt ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/uvscan ... is OK.
/tmp/tmp.mHXPvYeKjb/uvscan/liblnxfv.so.4 ... is OK.

But am not getting the correct output. I have tried removing both the MULTILINE and DOTALL options as well and still do not get the correct response. Below is the output when running with DOTALL and MULTILINE.
9
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')
('', '', '', '', '')

Any help would be much appreciated!! Thanks!!
Comment: With `re.findall`, you extract all captured values. Try `re.compile(r'^(?:(?!\b(?:is OK|file could not be opened|is a broken|is a block)\b).)+$', re.DOTALL | re.MULTILINE)` with `re.findall`
Comment: Can you show what you target output is, I will solve it instantly.
Comment: I guess I wasn't clear I want all the lines that do not end with "is OK" or "is a broken" kind of like if I were to run: grep -v "is OK" ${OUTDIR}/${OUTFILE} | grep -v "is a broken" | grep -v "file could not be opened" | grep -v "is a block" > ${OUTDIR}/${OUTFILE}.trimmed 2>&1
Comment: Wiktor I tried your solution and still no success...
Comment: @el_loza try mine below.
Answer: Perhaps think simpler, line by line:
import re
import sys

pattern = re.compile(r"(is OK)|(file could not be opened)|(is a broken)|(is a block)")

with open(sys.argv[1]) as handle:
    for line in handle:
        if not pattern.search(line):
            sys.stdout.write(line)

Outputs:
eth0
10.0.11.196
00:0C:29:AF:6A:A7
parameters passed to uvscan: --DRIVER /opt/McAfee/uvscan/datfiles/current --    ANALYZE --AFC=32 ATIME-PRESERVE --PLAD --RPTALL RPTOBJECTS SUMMARY --UNZIP -- RECURSIVE --SHOWCOMP --MIME --THREADS=4 /tmp
temp XML output is: /tmp/HIQZRq7t2R
McAfee VirusScan Command Line for Linux64 Version: 126.96.36.1994
Copyright (C) 2014 McAfee, Inc.
(408) 988-3832 LICENSED COPY - April 03 2016

AV Engine version: 5700.7163 for Linux64.
Dat set version: 8124 created Apr 3 2016
Scanning for 670707 viruses, trojans and variants.

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/ATIME-PRESERVE

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/RPTOBJECTS

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/SUMMARY

Comment: I thought he said he wanted to remove these lines. He wants everything else.
Comment: Yeah, I think you want `if pattern.search(line) is None` (or `if not pattern.search(line)`).
Comment: @CPanda, clearly, a trivial fix if the code is simple enough!
Comment: well, if the target lines start somewhere and continue till the end of the file, all except the first match are a waste. You could just break out. Couldn't you?
Comment: @cdlane I didn't get what you just said.
Comment: @CPanda, I'm not sure I follow you but the pattern will do what you describe, it will succeed on the first alternation that matches so there is no 'break out' involved.  It should only try them all on failure.
Comment: @cdlane can you tell me one thing, as I dont know about McAfee log files, say you hit a line that matches. From that point onward do they they continue to the end of the file??
Comment: @CPanda, again I'm not sure I follow but this solution has nothing to do with McAfee.  It simply walks the file line by line.  If a line doesn't match the pattern, it prints it.  That's all.  Any given line has no effect on the others.
Comment: @cdlane I understand what you are saying, and I never said what you are saying is incorrect. I am just asking - is the pattern intermittent or continues to the end of the file.
Comment: @cdlane Thank You! It looks like this solution works for me! Now I can just redirect the output to a log file and be done with the script!
Answer: Sometimes regexes are more complicated, but if you're really only looking for these patterns then I'd probably just try the simple approach:
terms = (
    'is OK',
    'file could not be opened',
    'is a broken',
    'is a block',
)

with open('/tmp/sample.log') as f:
    for line in f:
        if line.strip() and not any(term in line for term in terms):
            print(line, end='')

It might not be faster than the regex, but it's about as simple as it gets. Alternatively you could also use a slightly more strict approach:
terms = (
    'is a broken',
    'is a block',
)

with open('/tmp/samplelog.log') as f:
    for line in f:
        line = line.strip()
        if not line:
            continue
        elif line.endswith('is OK.'):
            continue
        elif line.endswith('file could not be opened.'):
            continue
        elif any(term in line for term in terms):
            continue
        print(line)

The approach I would take largely depends on who I expect to be using the script :)
Answer: Try this (and it's done in one line)
p = re.compile("^(?:[if](?!s OK|s a broken|s a block|ile could not be opened)|[^if])*$")

It means that if in a line you have an "i" or an "f" it cannot be followed the suffix mentioned or it's not an "i" or an "f" then it's ok. It repeats that for all the charaters in the line.
Edit: After testing at regex101.com, I found why it was not working.  Here is the one line regex that will work.
p = re.compile("^(?:[^if\n]|[if](?!s OK|ile could not be openeds OK|s a broken|s a block|ile could not be opened))*$", re.MULTILINE)

Comment: I think it doesn't like the `(?:s OK|s a ...` I took it out.
Comment: Still no success...The log file checks file by file so potentially all of the files could be some variant of "is OK" "is a broken" but if not, I would like to keep all of the lines that indicate a problem, but at the very least I would like to get the header lines at the beginning and footer lines at the end of the log
Comment: I just tried `^([^i]|[i](?!s OK))+$` in a text editor that supports regex and it doesn't work.  I'm intriged.
Answer: I know it is too late to answer. But I see that no answer is a correct solution.
Your regex for this case is wrong. You have unnecessary additional groups, a period is missing "."  Also, it will only match if "is OK|file could not be opened|is a broken" is at the beginning of the sentence.
"hello world is OK": does not match
"is OK hello world": matches

In a reverse match just use Non-capturing group '(?:)' instead of Capturing group '()'. This is to not get an empty string.
If you want to remove the entire sentence, you can use the following expression:
 r"^(?!.*(?:is OK|is a broken|file could not be opened)).*"

"is OK. hello world": matches
"hello world is OK.": matches
"is Ok.": matches

If you want to remove the entire sentence but only the ones ending in "is OK.|File could not be opened.|Is a broken.", You can use the following expression:
r"^(?!.*(?:is OK|is a broken|file could not be opened)\.$).*"

"is OK. hello world" does not match
"hello world is OK.": matches
"is Ok.": matches

Remember to use Non-capturing group '(?:)' instead of Capturing group '()', otherwise you will get an empty string:
                #Capturing group
regex = r"^(?!.*(is OK|file could not be opened|is a broken|is a block)).*"
print(re.findall(regex,text,flags=re.MULTILINE))

output:
['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']

Use the join() function to get the full text
                #Non-capturing group
regex = r"^(?!.*(?:is OK|file could not be opened|is a broken|is a block)).*"
print("\n".join(re.findall(regex,text,flags=re.MULTILINE)))

output:
eth1
10.0.11.196
00:0C:29:AF:6A:A7
parameters passed to uvscan: --DRIVER /opt/McAfee/uvscan/datfiles/current --    ANALYZE --AFC=32 ATIME-PRESERVE --PLAD --RPTALL RPTOBJECTS SUMMARY --UNZIP -- RECURSIVE --SHOWCOMP --MIME --THREADS=4 /tmp
temp XML output is: /tmp/HIQZRq7t2R
McAfee VirusScan Command Line for Linux64 Version: 220.127.116.114
Copyright (C) 2014 McAfee, Inc.
(408) 988-3832 LICENSED COPY - April 03 2016

AV Engine version: 5700.7163 for Linux64.
Dat set version: 8124 created Apr 3 2016
Scanning for 670707 viruses, trojans and variants.

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/ATIME-PRESERVE

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/RPTOBJECTS

No file or directory found matching /root/SVN/swd-lhn-build/trunk/utils/SUMMARY

Test it

Inverse Match Help in Python


Why won't sort work

Question: I write the following program to sort vector of struct by time after read in and process certain info, but it won't work and give me error message:

`no matching function for call to 'swap(collisionOfCars&, collisionOfCars&)'`,`error: no type named 'type' in 'struct
std::enable_if<false, void>'` and `error: use of deleted function
'collisionOfCars& collisionOfCars::operator=(collisionOfCars&&)'
*(__first + __holeIndex) = _GLIBCXX_MOVE(*(__first + __secondChild));`

My guess is sort function has some issues, but I do not know how to fix it.
Here is my code;
  class Car{
 public:
    string ID;
    double xPos, yPos, xVel, yVel;
    bool collided;
    Car(){

    }
    Car(string ID, double xPos, double yPos, double xVel, double yVel, bool collided){
      this->ID = ID;
      this->xPos = xPos;
      this->yPos = yPos;
      this->xVel = xVel;
      this->yVel = yVel;
      this->collided = collided;
    }
};
struct collisionOfCars{
 public:
  Car &firstCar;
  Car &secondCar;
  double time;
};
bool tupleCompare(collisionOfCars first, collisionOfCars second) {
  return first.time < second.time;
}
collisionOfCars willCollide(Car &car1, Car &car2) {
   double posDifX = car1.xPos - car2.xPos;
   double posDifY = car1.yPos - car2.yPos;
   double velDifX = car1.xVel - car2.xVel;
   double velDifY = car1.yVel - car2.yVel;

   double a = velDifX*velDifX + velDifY*velDifY;
   double b = posDifX*velDifX+posDifY*velDifY;
   double c = posDifX*posDifX + posDifY*posDifY - 100;
   double possibleSol = b*b - a*c;     // b^2-4ac

  double sol1, sol2;
  if ((a == 0) || possibleSol < 0) {     // check for a = 0 or b^2-4ac < 0
    return collisionOfCars{car1, car2, -1};
  } else if (possibleSol == 0) {
    sol1 = -b/(2*a);
    if(sol1 >= 0){
      return collisionOfCars{car1, car2, sol1};
    }
  } else {
    sol1 = (-b + sqrt(possibleSol))/(a);
    sol2 = (-b - sqrt(possibleSol))/(a);
    if (sol1 <= sol2 && sol1 >= 0) {
      return collisionOfCars{car1, car2, sol1};
    } else if (sol2 <= sol1 && sol2 >= 0) {
      return collisionOfCars{car1, car2, sol2};
    } else {
      return collisionOfCars{car1, car2, -1};
    }
  }
}
int main() {
  vector< Car > collection;     // All info of cars
  vector< collisionOfCars > collision;      // all possible collision
  string ID;
  double xPos, yPos, xVel, yVel;
  while (cin >> ID >> xPos >> yPos >> xVel >> yVel) {
    collection.push_back(Car(ID, xPos, yPos, xVel, yVel, false));
  }
  for (int i = 0; i < collection.size(); i++) {
    for (int j = i+1; j < collection.size(); j++) {
      collisionOfCars result = willCollide(collection[i], collection[j]);
      if (result.time != -1) {
        collision.push_back(result);
      }
    }
  }
  sort(collision.begin(), collision.end(), tupleCompare); // where goes wrong
}

In case you are wondering, I use reference in struct to make sure any change to Car pass in to struct will reflect on vector< Car > collection

I have changed the reference to pointer but now it gives me new error message as following
cannot convert 'Car' to 'Car*' in initialization
       return collisionOfCars{car1, car2, sol1};

Comment: `collisionOfCars` is not trivially assignable because it contains reference members, which cannot be reseated. This means that `std::sort()` is *unable to swap `collisionOfCars` instances* around, which is required to sort the vector. Consider using pointers in this struct instead of references.
Comment: Show us the definition of `tupleCompare`
Comment: Or there are very complicated solutions involving proxy arrays of pointers or indexes which get sorted and then create a new sorted array by copying the original items in sorted order. But you'd only do that if you really HAD to.
Comment: where does it give the error?
Comment: The error does not appear till I add sort() to my code
Comment: Try writing your own operator= for that class. You can't do it. Neither can the compiler. Use pointers instead.
Comment: I must say that even though C++ error messages have improved immensely over the last 10-20 years, the error given here is still terrible and probably very difficult to understand for anyone who isn't an expert in C++.
Comment: There are 2 things to try when error messages are too confusing for you.  The easiest is to try compiling your code, unchanged, with clang++ (i.e. simply replace your current use of g++ with clang++, you have it installed, right?)
Answer: In order for std::sort() to work, std::swap() must be instantiated for the type of object being sorted (collisionOfCars). In order for this to happen, it must either be copy- or move-assignable (or have an std::swap() specialization).  Your type does not declare either operator, and the compiler is not generating one for you because your type does not meet the requirements to have the operator generated.
This is because the type has non-static data members of a reference type. (Keep in mind that references cannot be reseated, so it's impossible to swap two references such that they refer to different objects -- you can only swap the content of the referred-to objects, which is not what you want.)

A defaulted copy assignment operator for class T is defined as deleted if any of the following is true:
...

T has a non-static data member of a reference type;

Source

The implicitly-declared or defaulted move assignment operator for class T is defined as deleted if any of the following is true:
...

T has a non-static data member of a reference type;

Source
This means that collisionOfCars instances cannot be assigned on top of each other and this is required to sort the vector.  (How else can the vector be sorted if not by swapping elements around on top of each other?)
The simplest solution would be to change collisionOfCars to use Car * (pointer-to-Car) instead of Car & (reference-to-Car).  You could also use std::reference_wrapper<Car> (which is just a wrapper around a pointer anyway).
Comment: @ZhiyuanRuan The error is likely because you have to now treat it as a pointer everywhere. E.g. you have to change `collisionOfCars{car1, car2, sol1}` to `collisionOfCars{&car1, &car2, sol1}`, etc.

Why won't sort work


SP2-0552: Bind variable "ELSE" not declared?

Question: I don't understand why this error shows when I specified the conditions and treatement
sql>declare
    experience int;
    salaire employees.salary%type;
    begin
    select datediff(year,getdate(),hire_date),employees.salary into experience,salaire from employees where employee_id=115;
    dbms_output.put_line('Avant id=115 '||' experience='||experience||' salaire='||salaire);
    if experience>10 then salaire=salaire*1.2;
    elsif (experience>5 and experience <= 10) then salaire*1.1:
    else salaire=salaire*1.05;
    end if;
    dbms_output.put_line('Apres id=115 '||' experience='||experience||' salaire='||salaire);
    end;

Comment: `then salaire*1.1;`
Comment: This is not PL/SQL or Oracle code -- or at least it has a healthy does of SQL Server mixed in.  Are you sure you have tagged the correc database.
Comment: I see now that I have syntax errors
Answer: When errors are fixed (:= instead of =, ; instead of :, non-existent functions), then code looks like this:
DECLARE
   experience  INT;
   salaire     employees.salary%TYPE;
BEGIN
   SELECT EXTRACT (YEAR FROM SYSDATE) - EXTRACT (YEAR FROM e.hire_date),
          e.salary
     INTO experience, salaire
     FROM employees e
    WHERE e.employee_id = 115;

   DBMS_OUTPUT.put_line (
         'Avant id=115 '
      || ' experience='
      || experience
      || ' salaire='
      || salaire);

   IF experience > 10
   THEN
      salaire := salaire * 1.2;
   ELSIF (    experience > 5
          AND experience <= 10)
   THEN
      salarie := salaire * 1.1;
   ELSE
      salaire := salaire * 1.05;
   END IF;

   DBMS_OUTPUT.put_line (
         'Apres id=115 '
      || ' experience='
      || experience
      || ' salaire='
      || salaire);
END;

It depends on what experience is; I just subtracted hire_date year from current year.
Answer: This line is not Oracle:
select datediff(year,getdate(),hire_date),employees.salary
into experience, salaire
from employees
where employee_id = 115;

I can speculate that you want something like this:
select floor( months_between( sysdate, hiredate ) / 12 ),
       employees.salary
into experience, salaire
from employees
where employee_id = 115;

This is not exactly the same logic that you have.  But it is probably a better definition of experience given what datediff() really does.
Comment: What, _exactly_, is "sql command line"? it sounds like it might be oracle's sqlplus.  Ultimately, it doesn't matter what your client program is. If you are working with oracle database, you have to use oracle sql and/or pl/sql.  You can't mix in syntax from other databases.
Comment: I'm working with sql command line and oracle database. is there another function that could calculate the years of experience . Could you please explain the difference btw using datediff and months_between


What is the most elegant way to check if all values in a vector are true?

Question: This question is similar to this question:
What is the most elegant way to check if all values in a boolean array are true?
But for vector<bool> in c++.
How can use all_of for this purpose (for example I want to check the value between ranges, element 10 to 20) so I can not use begin and end.
Comment: Maybe `std::find(cbegin(vector) + 10, cbegin(vector) + 20, false) == cbegin(vector) + 20`.
Comment: You know that functions that take iterators, take any valid iterator, not only `begin()` and `end()`, right?
Answer: If you want to use std::all_of, then just return the value in the predicate function :
#include <vector>
#include <iostream>
#include <algorithm>

int main()
{
    std::vector< bool > v( 30, true );

    const bool all = std::all_of( std::begin(v)+10, std::begin(v)+20, []( const bool v){ return v; } );

    std::cout << all << std::endl;
}

Comment: How can I set the range?
Comment: @user1436187 The same way in the other answers.
Comment: what is the different between `std::begin` algorithm and `cbegin`?
Comment: @user1436187 One is a function, the other is a member function.
Comment: Thanks, which of them is faster?
Comment: Is `std::end(v)+20` actually `std::begin(v)+20`?
Answer: I'd just try std::find(begin(v)+10, begin(v)+20, false) - it returns begin(v)+20 if all values between 10 and 20 are true.


Is there an easier way to solve the linearization of this composite function?

Question: Given $f(x)=x^4-4x^2-6x+4$ and $g(x) = 4x^3+3x^2-10x+2$
a. Approximate the change in $f(g(x))$ as x changes from 1 to 1.02.
b. Approximate $f(g(1.02))$
Is there an easier way to solve this problem without having to find what $f(g(x))$ is, finding the derivative of it, and also plugging in 1 and 1.02? Thanks.
Edit: answers are: a. $-0.32$ and b. $6.68$
Comment: It's not an approximation if you do that. Read the section in your Calculus book on how to approximate with differentials.
Answer: We don't have to find $f(g(x))$ explicitly.
We can use chain rule to get the derivative:
$$\frac{d}{dx}f(g(x))= f'(g(x))g'(x)$$
$$f(g(x)) =f(g(1)) + \frac{d}{dx}f(g(x))|_{x=1 }(x-1)$$
Comment: [*Chen Lu*](https://youtu.be/9UWVTzhzmlE). `:)`
Answer: To approximate the change in a function, use differentials. For example, suppose $f(x)=x^2$ and we want to know what $f(1.01)$ is. We can easily square this to get $1.0201$ which is exact but can we approximate it?
To approximate we take the derivative to get $\frac{df}{dx}=2x$ which we rewrite as $df=2xdx$.
Now we get that $f(1.01)\approx f(1)+ df=f(1) + 2(1)(.01)= 1+.02=1.02$ where $x=1$ and $dx=.01$.
You should be able to perform the same logic with $x=1$ and $dx=.02$.


If $f(n) = \sum_{i = 0}^n X_{i}$, then show by induction that $f(n) = f(n - 1) + X_{n-1}$

Question: I am trying to solve this problem by induction. The sad part is that I don't have a very strong grasp on solving by inductive proving methods. I understand that there is a base case and that I need an inductive step that will set $k = n$ and then one more step that basically sets $k = n + 1$.
Here is the problem I am trying to solve:

If $f(n) = \sum_{i = 0}^n X_{i}$, then show by induction that $f(n) = f(n - 1) + X_{n-1}$.

Can I have someone please try to point me in the right direction?
*EDIT: I update the formula to the correct one. I wasn't sure how to typeset it correctly and left errors in my math. Thank you for those that helped. I'm still having the problem but now I have the proper formula posted.
Comment: I presume you mean $f(n) = \displaystyle \sum_{i=1}^n X_i$ and $f(n) = f(n-1) + X_n$ rather than what you wrote.
Comment: I only fixed the formatting of the question. I second @CliveNewstead's comment. What's the upper limit of the sum?
Comment: And what's your definition of $\sum$? Usually this is how it's recursively defined.
Comment: I'm sorry I am unfamiliar with the code tags for a math type setting. The limit is actually i = 1 to n-1. And the second part of the equation should be f(n-1) + x ^ (n-1).
Comment: Could you please write the problem *precisely* as it was stated, because the above can be interpreted in various ways. Do you already have available some rigorous definition of the sum that differs from the stated recursion? One most employ great care here to avoid circular proofs.
Comment: From the source:

Let $$f(n) = \displaystyle \sum_{i=1}^{n-1} X^{i}$$.

Show that $$f(n) = f(n -1) + x^{n-1}$$
Comment: What is the source? Does it say to use induction? Does it discuss inductive/recursive definitions before this? To reply to someone you need to write @username else they will not be notified.
Comment: @BillDubuque The source is a homework problem. It does not discuss any type of recursive/inductive definitions. That is the question in its entirety. It does not build from any previous questions it is just asking for a proof. I thought that proof by induction was a good method to try.
Comment: @Michael A homework problem from where? A class in discrete math, or algorithms, or foundations of math? What is your textbook?
Comment: @BillDubuque This is a class on algorithms and data structures. We are using Introduction to Algorithms 3rd edition. My professor is what many would call very very unclear on explaining how to derive and solve these algorithms and I fear that teaching myself is making me only more confused.
Comment: If you wish to understand this *rigorously* then you need to understand so-called inductive or recursive definitions. For a rigorous and insightful introduction see the award-winning Monthly exposition of Leon Henkin mentioned in my related posts [here](http://math.stackexchange.com/a/143539/242) and [here.](http://math.stackexchange.com/a/120586/242) Many algorithm textbooks gloss over these formalities at the cost of rigor. It's hard to know what the appropriate compromise may be for your course without knowing further details.
Comment: @BillDubuque Thank you, I will take a look. I wish I knew what was really expected in this course as well. I would be all for proving and showing if an algorithm is correct or not if my professor actually knew what was going on. Most of the time its just him trying to solve his own lecture problem (which doesn't help much with strategy.)
Comment: Why isn't "That's the _definition_ of $\sum$." the complete proof?
Answer: The way induction works is based on two steps:

Check a base case.
Show that if it holds for one number, then it holds for the next.

More formally, let $P$ be a property, and let $P(n)$ be the assertion that the property holds true for the natural number $n$; so for instance, $P(n)$ could be the assertion that $\sum_1^n i = \frac{n(n+1)}{2}$. Then to prove by induction that $P(n)$ holds for all $n \ge 1$, it suffices to

Check that $P(1)$ holds;
Check that, for any given value of $n$, if $P(n)$ holds then $P(n+1)$ holds.

Then we have a sort of domino effect. $P(1)$ holds as you showed for (1); and $P(2)$ holds because $2=1+1$ and $P(1)$ holds; and $P(3)$ holds because $3=2+1$ and $P(2)$ holds; and $P(4)$ holds because $4=3+1$ and $P(3)$ holds; and so on.
But here, induction is not necessary, since you can prove it directly for any given value of $n$. For any function $g$, $\displaystyle \sum_{i=1}^n g(i) = \sum_{i=1}^{n-1} g(i) + g(n)$ just by the definition of the $\sum$ symbol (unless you have a weird definition), and the result follows immediately from this fact.
Comment: I had mistakenly given you the wrong equation.
Comment: @MichaelGuantonio: That's okay, just subtract $1$ from everything :)
Answer: To prove by induction, you need to prove two things. First, you need to prove that your statement is valid for $n=1$. Second, you have to show that the validity of the statement for $n=k$ implies the validity of the statement for $n=k+1$. Putting these two bits of information together, you effectively show that your statement is valid for any value of $n$, since starting from $n=1$, the second bit that you proved above shows that the statement is also valid for $n=2$, and then from the validity of the statement for $n=2$ you know it will also be valid for $n=3$ and so on.
As for your problem, we proceed as follows:
(Part 1) Given the nature of your problem, it is safe to assume $f(0)=0$ (trivial sum with no elements equals zero). In this case it is trivial to see that  $f(1)=f(0)+X_{1}$.
(Part 2) Assume the proposition is true for $n=k$, so $$f(k)=\sum_{i=1}^kX_i=f(k-1)+X_k$$
Adding $X_{k+1}$ to both sides of the equation above we get
$$f(k)+X_{k+1}=\sum_{i=1}^kX_i=f(k+1)$$
So, the truth of the statement for $n=k$ implies the truth of the statement for $n=k+1$, and so the result is proved for all $n$. As Mr. Newstead said above, the induction step is not really necessary because of the nature of the problem, but it's important to have this principle in mind for more complicated proofs.
Comment: I have mistakenly given you the wrong equation.


Java e MySQL - Erro - Timezone

Question: Tento conectar com o MySQL, com o código abaixo. Não uso Hibernate, nem nada.
É só para conectar ao banco e executar uma cláusula SQL simples. Recebo o erro que está abaixo.
Já vi que tem algumas perguntas com respostas aqui, mas confesso que não entendi.
Mensagem de erro:

java.sql.SQLException: The server time zone value 'Hora Padrão da Europa Ocident' is unrecognized or represents more than one time zone. You must configure either the server or JDBC driver (via the serverTimezone configuration property) to use a more specifc time zone value if you want to utilize time zone support.

Código:
import java.sql.*;

public class ConnectMySql {
    public static void main(String[] args) {
        try {
            // Estabelece a conexão
            Connection cnx = DriverManager.getConnection("jdbc:mysql://localhost:3306/mondovino", "root", "senha");

            // Cria um Statement
            Statement stmt = cnx.createStatement();

            // Executa a query
            ResultSet rset = stmt.executeQuery("seletc * from paese");

            // Navega no result set
            while (rset.next()) {
                System.out.println("Pais " + rset.getString("fips") + " -> " + rset.getString("nomepaese"));
            }
        }
        catch(Exception e) {
            e.printStackTrace();
        }
    }
}

Comment: Eu adicionei isso na minha URL do BD e funcionou: ?useSSL=false&useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=UTC
Answer: Tente adicionar na sua url de conexão estes parâmetros: useTimezone=true&serverTimezone=UTC
Ficará assim:
DriverManager.getConnection("jdbc:mysql://localhost:3306/mondovino?useTimezone=true&serverTimezone=UTC", "root", "senha");

Comment: Perfetto. Niente come parlare con chi sa cosa si deve fare. Certinho. Valeu.
Comment: @Alano, marque como correta a resposta. Assim indicará aos outros que a resposta está correta.

Java e MySQL - Erro - Timezone


Will installing Django on Linux VPS server, cause any issue to live sites

Question: I have my live sites running on Linux VPS. Now i want to install Django on that.
I just want to make sure that will it be safe. Will it interfere in current php sites running with apache.
Any particular file/files i can backup , just in case problem occurs
Answer: There is no definitive "Yes" or "No". Its always good to have a backup. If you misconfigure it, yes ist can affect the whole VPS but this is not necessarily the case. There is always the chance that something goes wrong. The question is too generic to give a qualified answer. Try it, if it fails then recover your backup.
Comment: By Backup, i want to know is there any specific files rather than full backup , to save just for worst case thing. i mean websites needs apache,php . i am not touching php , so apache is where something can go wrong. so any particular files of apache
Comment: I haven't worked with CentOS so far but there are numerous ways. What VPS provider are you on? Did they have a backup feature, that would be the easiest way in my opinion. There are also script at the CentOS wiki: http://wiki.centos.org/HowTos/ As for apache alone I would backup httpd.conf and the DocumentRoot.


Status is a duplicate error. I'm trying to make my bot post a tweet, but it's not working. (Python)

Question: import tweepy

CONSUMER_KEY = 'My key'
CONSUMER_SECRET = 'My secret'
ACCESS_KEY = 'my key'
ACCESS_SECRET = 'my secret'

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)

api = tweepy.API(auth)

try:
    api.verify_credentials()
    print("Authentication OK")
except:
    print("Error during authentication")

api.update_status("Hello")

Obviously at the top I don't actually have my secret, my key, etc. up there, but I have the actual keys and it connects just fine. The problem is I keep getting an error whenever I try to send the tweet hello. Can anyone help?
Comment: Fix what? You haven't told what the actual problem is, which is why I asked for clarification
Comment: Please [edit] your question to include the full error traceback, as that contains valuable information about the problem
Comment: The error message is that it says "Status is a duplicate."
Answer: You need to catch such exception.
I bet that the api throws DuplicateError (or similar as you don't show the stacktrace) when the status you set is identical.
something like:
from contextlib import suppress
from <your api module> import DuplicateError
...
with suppress(DuplicateError):
    api.update_status("Hello")
...

Status is a duplicate error. I’m trying to make my bot post a tweet, but it’s not working. (Python)


How do i create a field for a content type from a content type? Like a list for add multiples values from a content type, inside another

Question: It is simply as a 1 to many relationship between content and I need to manage multiple content within another.
I do not know if there is any plugin or library or method that allows me to. Please I need help. Thanks in advance
Comment: Could you elaborate ? From what I understand you want to use a field B (which is in content-type B) in content-type A.
Comment: Have you checked https://www.drupal.org/project/entityreference (entity reference module) ? it's a way of building the one to many relationship you need. for more check similar question
[https://drupal.stackexchange.com/questions/13862/relation-vs-references-vs-entity-reference-for-managing-node-reference-type-re]
Answer: It is as simple as use the module Entity Reference.
You'll have a new field type to reference a node (or multiple) into an other node.


Criando um programa de chat. Como fazer que dois conversem simultaneamente?

Question: # -*- coding: utf-8 -*-
#!/usr/bin/python3
import socket
# nao tem servidor UDP no google -> vamos usar netcat como servidor UDP!
#Programa de chat: so fala um de cada vez
#implementar falando ao mesmo tempo

client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

"""
pacotes_recebidos = client.recvfrom(1024) devolve uma tupla:
(' llallalaaaa\n', ('192.168.1.4', 667))

msg recebida + (IP,porta)
"""
try:
    while 6: #while True
        #client.sendto(input("Voce: ") + "\n", ("192.168.1.4", 668))
        #client.sendto (bytes(input("Voce: ")).encode('utf8') + bytes("\n").encode('utf8'), bytes(("192.168.1.4", 668)).encode('utf8'))
        client.sendto((input("Voce: ")).encode('utf8') + ("\n").encode('utf8'),("192.168.1.7", 661))
        # endereço do servidor UDP do kali linux usando netcat
        msg, friend = client.recvfrom(1024)
        print(str(friend) + ": " + str(msg))
 #se quiser apenas o ip: use friend[0]

    client.close()

except Exception as erro:
    print("Conexao falhou ")
    print("O erro foi: ", erro)
    client.close()

O programa acima em Python 2.7 funcionou ao trocar input por raw_input e não sei o porquê.
Ao rodar em Python 3.5 (com input ao invés de raw_input) eu tentei enviar uma mensagem "oi" e o seguinte erro ocorreu:

('O erro foi: ', NameError("name 'oi' is not defined",))

Gostaria de fazer com que 2 pessoas pudessem conversar simultaneamente, como fazer? No momento, apenas uma pessoa de cada vez consegue digitar a mensagem.
Temos que aguardar um dos participantes escrever e teclar Enter para continuar.
Este é o cliente. Estou usando como servidor o netcat.
Alguem poderia me ajudar?
Comment: tem um bom toturial http://www.bogotobogo.com/python/python_network_programming_tcp_server_client_chat_server_chat_client_select.php
Comment: `raw_input()` não existe em python3, fica só `input()`, talvez seja isso
Comment: Mas em python 3.5? Eu estou usando pycharm e python 3.5 em BackBox Linux.
Comment: @miguel, obrigado
Comment: Consegue me dizer o que ocorre com input() x raw_input() ?
Comment: Ao rodar em python 2 com raw_input() funciona. Em python 3 eu troco para input() e aparece o erro que descrevi!
Comment: Não sei, corri esse codigo aqui e não deu esse erro
Comment: ha... Eu uso python 3.4
Comment: @Miguel, eu li o tutorial mas não consegui resolver. Alguma sugestão?
Comment: Veja: https://www.youtube.com/watch?v=PkfwX6RjRaI . Já o fiz a uns anos e lembro-me que é bom.. Python2 mas deve dar para converter em 3
Comment: @Miguel, vc disse que executou o script em python 3.4 e não deu o erro. Como você executou? No pycharm dá erro!
Comment: Deu mas não foi o mesmo que lhe deu a si. Experimenta na linha isto: `input("Voce: ") + "\n".encode('UTF-8')...`
Comment: client.sendto( input("Voce: ") +"\n".encode('UTF-8'), ("192.168.1.4", 668))  : SyntaxError: invalid character in identifier
Comment: Eu tentei:         client.sendto (bytes(input("Voce: "),'utf-8') + bytes("\n",'utf-8'), bytes(("192.168.1.4", 668),'utf-8'))    . O erro foi:  unknown encoding: utf-8
Comment: Se eu tiro o 'utf-8' : client.sendto (bytes(input("Voce: ")) + bytes("\n"), bytes(("192.168.1.4", 668))) ->O erro foi:  string argument without an encoding
Comment: Resolvi o erro: só falta a conversa simultanea:         client.sendto((input("Voce: ")).encode('utf8') + ("\n").encode('utf8'),("192.168.1.7", 661))
        print(str(friend) + ": " + str(msg))
Answer:
O programa acima em Python 2.7 funcionou ao trocar input por raw_input e não sei o porquê.

No Python 2.x o método input é usado para interpretar e avaliar expressões, veja um exemplo:
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> input()
5 + 9
14
>>> x = input()
"stack"
>>> y = input()
"overflow"
>>> x + y
'stackoverflow'
>>>

O erro informado, NameError, indica que você está executando o script no Python 2.x.
Para mais informações veja a pergunta: input() e raw_input()

No momento, apenas uma pessoa de cada vez consegue digitar a mensagem, temos que aguardar um dos participantes escrever e teclar para continuar.

Isso acontece porque por padrão um socket é bloqueante, configurado para enviar e receber informações, pausando a execução do script até que uma determinada ação seja concluída.
Por exemplo, as chamadas ao método send() irão esperar por espaço disponível no buffer para fazer o envio dos dados, as chamadas ao método recv() por sua vez, irão esperar até que a outra parte da comunicação envie os dados a seres lidos.
O controle não é devolvido ao programa até que tenha espaço no buffer para o envio, ou que seja recebido algum byte da outra parte da comunicação, ou algum erro ocorra.
No Python para usar o socket como não-bloqueante, usa-se o método socket.setblocking com o argumento False, ou o método socket.settimeout com o valor 0.
import socket

cliente = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
cliente.setblocking(False)

Vai ser usado o módulo select, para monitorar a entrada de dados do usuário no terminal.

This module provides access to the select() and poll() functions
  available in most operating systems, devpoll() available on Solaris
  and derivatives, epoll() available on Linux 2.5+ and kqueue()
  available on most BSD. Note that on Windows, it only works for
  sockets; on other operating systems, it also works for other file
  types (in particular, on Unix, it works on pipes). It cannot be used
  on regular files to determine whether a file has grown since it was
  last read.

Nota: Se estiver usando Windows, o select pode não ter o comportamento esperado como pode ser visto destacado no texto acima.
Código:
#!/usr/bin/python3
# -*- coding: utf-8 -*-

import socket, sys, select

cliente = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
cliente.setblocking(False)

serverAddr = ("192.168.1.4", 667)

def mensagem():
    # Verifica se foi digitado algo
    if select.select([sys.stdin,],[],[],0.0)[0]:
        # Formata a string para mandar "Cliente: <entrada>"
        entrada = "Cliente: {}".format(sys.stdin.readline())
        entrada = entrada.encode('utf-8')

        return entrada
    return False

try:
    while 1:
        try:
            msg, friend = cliente.recvfrom(1024)
            # friend[0] - IP / friend[1] - porta

            # rstrip() é para eliminar a quebra de linha
            msg = msg.decode('utf-8').rstrip()
            print("{}: {}".format(friend[0], msg))
        except:
            pass

        try:
            entrada = mensagem()
            if(entrada != False):
                cliente.sendto(entrada, serverAddr)

        except KeyboardInterrupt:
            sys.exit("\nChat encerrado!")

    cliente.close()

except Exception as erro:
    print("O erro foi: {}".format(erro))
    client.close()