Examples of "Textbook" Retrospective Studies

Question: I am a physician and have never had any formal training in statistics/biostatistics. However every time I come across a journal article, specifically on retrospective studies, I cannot help but feel a little suspicious that (a) the authors are trying to pull a fast one on me and/or (b) they are like myself and don't really understand what they are doing as well.
So I was wondering if there are any retrospective studies, hopefully in medicine, that are considered to be "classics" or are stellar examples of logical thinking and sound applications of statistical principles.
I know that the knee-jerk response might be to tell me to read textbook A or B, but unfortunately I don't have that kind of luxury at this point. I also do not have the luxury of quick access to a biostatistician to ask these kinds of questions at my current institution.
-- EDIT --
In response to some of the answers below, all of which are much appreciated, let me add the following:
I am a surgeon, so I am interested in diseases pertaining to the field of surgery. AdamO suggested that I look at the NEJM, JAMA, or the Lancet, which I did. One of the studies that I found interesting but am somewhat skeptical of is: "Risk Factors for Retained Instruments and Sponges after Surgery" by Atul A. Gawande et al, published in the NEJM, January 2003.
Now Dr. Gawande is something of a rock-star in general surgery, and this is one of the reasons why I chose to look at his paper after getting the suggestion from AdamO. His paper is published in the NEJM, so he must be doing something good when designing his study.
He is trying to see what are risk factors for surgeons accidentally leaving things inside patients. This is a (thankfully) very rare occurrence: scouring through a Massachusetts malpractice insurance company's records from 1985 to 2001, he found only 54 cases.
However, his design leaves me somewhat confused and unhappy. For instance he needs to compare 54 cases with a risk-adjusted cohort that did not have retained objects, which he will treat as control. He writes "Given an estimated 60 cases available for review, we determined that four controls for each case would give the study sufficient power to detect a risk factor present in 30 percent of patients that produced a doubling of the likelihood that a foreign body would be left behind." Most likely some biostatistician told him that would be good enough, but unfortunate statistics-ignorant lay-folk have no way of knowing whether this is indeed good practice.
In any case he now has 54 patients to compare with a randomly chosen risk-adjusted cohort of 235 patients acting as control (which magically allows him to compare risks in the way he describes). When doing the chart reviews he told his residents to go through the charts looking for a predefined set of variables and to see how often they appear in all the charts. These variables include age, sex, body-mass index, whether counts of sponges and instruments were performed, duration of operation, whether the operation was performed on an emergency basis, etc.
He then performs logistic regression on each of these variables to see whether they were associated with an increased likelihood of retention of a foreign body. However, he sets the level of statistical significance to be P <0.2 as the inclusion criterion for further analysis. This is something that I find somewhat disturbing. I have haunted these forums and looked up methods for variable selection and almost always the answer is that using P-values for variable selection is improper or erroneous. Again I could be wrong and Dr. Gawande might be using it properly in this case for all I know, but I am skeptical. The only people who seem to support its use are those in medicine or epidemiology.
So he then uses 8 variables that have P <0.2, for multiple logistic regression to find that only 3 variables are now significant (operation performed on an emergency basis, unexpected change in operation, and high BMI).
As a clinician, I would love to think that Dr. Gawande's approach to a retrospective analysis is indeed a model of the format. Except for the first part where he decides that approximately 240 subjects are needed as control to give the study sufficient power, everything else is pretty straightforward. However, is this true? Is this study really a model example of the retrospective analysis form as AdamO implies? Or has Dr. Gawande's status within the medical community perhaps allowed a top name journal to give their stamp of approval on what would otherwise be considered a subpar statistical experiment by the statistics community?
Answer: I find that high profile medical journals in the State (NEJM, Lancet, JAMA) tend to publish retrospective studies that are of high quality from a biostatistician's point of view.
There are a few nuanced aspects to such studies, notably that the interpretation of model parameters are often different and not immediately palpable as relevant association measures. For instance, hazard ratios and odds ratios from Cox and logistic regression models approximate relative risks for outcomes comparing exposure levels, but only when the outcome is rare. In association studies of rare events, like the Wilm's tumor or Non-Hodgkin's lymphoma, retrospective studies work at the population level. The great benefit of retrospective designs is that one can sample individuals based on their case status (e.g. whether they're diseased) and compare them to controls in their prevalence of exposure. I think this seems like a paradox at first blush, but mathematically it works out. We can explore associations in the populations this way.
In general, I critique retrospective designs using all the same tools with which I critique clinical trials: was the outcome prespecified? Were eligible populations well defined? Were matching and confounding variables clearly described and accounted for? Are the major conclusions consistent with the data obtained?
Answer: Your question has a few parts and I will try to answer them all.
(1) There is no single study design called a 'retrospective study', but rather a multitude of types which use retrospective data. These study designs are also often called 'efficient sampling designs' which emphasizes the idea that you are sampling only a subset of the people you would use in a cohort design (specifically, you are typically sampling a subset of the people who don't develop the outcome during the period of interest, while taking all of the people who do).
(2) You seem to be interested in a case-control study, rather than some of the more complicated designs, but even then, there are multiple types which differ based on sampling strategies as well as the object of inference. The traditional case-control design uses "cumulative incidence" sampling, and, under the rare disease assumption (which AdamO) described, the odds ratio estimates the risk ratio. Other sampling schemes are available which allow the odds ratio to estimate the rate ratio, without the rare disease assumption.
(3) Given that you don't have time to go in depth into a textbook, I'm not sure if this will be too much reading, but let me suggest a series of three comprehensive papers by Wacholder which describe in detail the conduct of case-control studies. These are somewhat old, but should give you a good foundation and lots to think about in terms of designing a study yourself. Wacholder et al. 1992. Selection of controls in case-control studies (parts I, II, and III). Am J Epidemiol, 135(9):1019-50. The abstracts can be found here:
http://www.ncbi.nlm.nih.gov/pubmed/1595688
http://www.ncbi.nlm.nih.gov/pubmed/1595689
http://www.ncbi.nlm.nih.gov/pubmed/1595690
(4) When conducting or evaluating a retrospective study, the key first question you need to ask is "what is the source population from which the cases arose?" The answer to this question tells you several things: (i) what is population the about which you can reasonably make inference/draw conclusions; (ii) what is the population from which the controls should be selected. Second, you should ask "are the controls drawn as a sample from this population?" If the answer to this second question is 'no', then (in most cases) the study is not going to give you valid conclusions.
(5) To answer your first question, unfortunately most 'classic' papers that I have seen used for teaching about retrospective designs are examples of what not to do. Partly this is because the field of observational study design is constantly moving forward and so many older studies use methods that were believed to be sound but are now known to be inappropriate. But, partly this is because many studies repeat the same basic problems, so it is important for students to develop strong skills in identifying these issues so they can avoid them: (i) the authors are not clear on what population gave rise to their cases and as a result use an inappropriate control population; and/or (ii) the authors use an inappropriate method to select confounders.
(6)I know you asked for example papers, but I think it might be a bit more helpful for you to have a critique of the paper you discuss in your edit, since this seems like a relatively good paper. In reference to the source population, Gawande is assessing individuals who underwent a surgical procedure between January 1, 1985, and January 1, 2001, at one of 22 Massachusetts hospitals, and who, if they had suffered a complication of surgery due to provider error, would have had a claim or incident report filed with the Controlled Risk Insurance Company (CRICO). Gawande does a good job of selecting controls that come from this population, since he matches on time period (within 6 months), provider/hospital, and surgery type. These seem reasonable matching factors, since they are likely associated with risk factors and with the outcome (retained instruments/sponges).
Selection of confounders is where Gawande may have problems. Using p-values to select variables for inclusion in a multivariate model is a common practice in biomedical research, and the process of using a p-value cut-off of 0.2 in univariate analyses and then of 0.05 in the multivariate analysis is also commonly used. However, as you have gathered, this method is not necessarily justifiable. Another commonly used, but often inappropriate, method of variable selection is the so-called 10% rule, where you include any variable that changes your effect of interest by 10% or more.
The 'gold standard' for variable selection is subject matter knowledge. If you know that a variable is causing bias (i.e. causing confounding) when excluded from your model, you need to include that variable. But which variables to include can depend on which other variables are already included in your model (a problem for model selection procedures, since this clearly implies that the order of adding variables can affect the variables included). Including inappropriate covariates can make it harder to fit your model, can bias your effect estimate to the null (for example, if the variable is actually an intermediate), or can inflate your effect estimate, and can invalidate the test of the null hypothesis.
A final caveat is that Gawande appears to be doing a more exploratory analysis - he isn't trying to determine whether a single specific exposure causes retained instruments or sponges. Rather he is trying to generate some hypotheses about what variables may need to be explored. In studies such as this, the authors may tolerate some potential bias in the estimates or test of the null, since the goal is to not to draw definitive conclusions. In this case, Gawande's study can serve as a reasonable model of a retrospective design which asks the question "what potential risk factors may be of interest?", even if it has some limitations as a model of a retrospective study which asks the question "does this exposure cause this disease?".
Comment: Also, I am surprised that I couldn't come up with good example papers. I'm going to keep looking and will update my answer when I find some, but it seemed like some background would be useful to the oort.
Answer: I agree with everything AdamO said above (I can't upvote anything yet).
In addition, can you be more specific? Is there a particular disease/pathology you are interested in? If you can narrow it down, people may be able to suggest a particular study, or understand a methodology that may be standardly used, and could then point you to teaching examples (i.e., already existing data sets you could start practicing on).
In addition, when you say "retrospective" studies, do you mean "file-drawer" studies (i.e., the authors had an accumulating pile of cases, didn't really have any hypotheses, ran some stats, and said "Gee we found this to be significant?"). Or do you mean something else, like meta-analysis?
In either case, there will be standard strengths and weaknesses associated with typically used methodologies, which will be somewhat well-known (some more than others), and easy to understand and apply. Some will be common-sensical and easily understood by anybody, and others will require pulling out a textbook. Again, the more specific you can be, the more specific an answer you can get.
Parenthetically, you haven't said your level of interest & commitment to this. If you are will to spend some time and effort on this (if you Googled for several hours, you may be very motivated). Learning some basic stats & a stats package may be useful, or may be overkill.
Comment: I may be mistaken, but neither of you have provided one example of a good retrospective study! If you feel like you do not have enough information, comment on his question, rather than answer.


Stop an already running cron job

Question: I have a PHP script on server that sends newsletter emails to my site users using a cron job. The users list is approximately 55,000 and takes nine hours.
Is there any way to stop it (urgently)?
Comment: `ps -e` will give you the list of process, and `kill processid` (processid is a number)
Comment: bg; to get the processes runing, kill to it
Comment: Please describe how do i kill this?
Answer: http://unix.derkeiler.com/Newsgroups/comp.unix.admin/2006-09/msg00132.html
http://unix.ittoolbox.com/groups/technical-functional/shellscript-l/how-to-kill-the-cronjob-which-is-running-currently-477250
You need to get the PID of your running cron job and then perform simple kill command.
Comment: If you don't have any kind of SSH access to command terminal, you can't do that.
Comment: You may use the [`system`](http://php.net/system) command to exec something from your php script and see the output.
Comment: Cron job script file is on host server. I have cPanel only. how do i check all processes by ps -e command?
Answer: I wouldn't suggest allowing the use of system(), exec(), passthru(), etc. Its a security issue.
If you do have a control panel, there has to be a way to manage the cron jobs, and I believe with most of them you also have the ability to launch an SSH session. You could use this method to stop/start the running process.
Last but not least, if you can't stop it yourself, contact your host.
Answer: someone already wrote u the answer,
try:

ps -e | grep "php name file"
the above command will display you the process id, so:
kill - 9 'process ID'
: )
Answer: Just comment out the specific cron job you want to not run. Switch to the user that controls the task, export your editor of choice into the environment, then use crontab -l:
<code>$ su - root
...
# EDITOR=vi; export EDITOR
# crontab -l
</code>
If you are using cPanel. GO to Advance section of the Corn Job and click the Delete button.
If the delete button is not there, switch cPanel skin to 'X' to view the Delete button.
Comment: Either you didn't read the question or you failed to understand it. The OP is asking how to stop an already running cron job, not how to edit it.

Stop an already running cron job


Cheroke URL rewrite rule

Question: I'm looking to create a URL rewrite rule for Cherokee 1.2.2 web server where the URL is:
http://example.com/pricing?remove=358&referer=cHJpY2luZw==

and I would like:
http://example.com/pricing

I also have a rule setup to remove index.php from the the URLs. That rule being:
internal | ^/(.*)$ | /index.php?q=$

Answer: If I understood you correctly this regex, substituted to pricing, should work:
^/pricing\?

Are you sure you want to lose the GET parameters like that?

Cheroke URL rewrite rule


Forcing Laravel response status code

Question: I'm building a REST API in Laravel and when a new user is created I want to return a 201 status code and a Location header to point to the new resource.
Most of that is being achieved using this code:
$response = Response::make(null, 201)->header('Location', Config::get('app.url') . '/v1/users/' . $user->id);
return $response;

However Laravel seems to be overriding what I set as the status code because I'm setting a Location header, as I'm getting a 302 Moved Temporarily header back.
How can I force a 201 status code even when I'm specifying a Location header?
Comment: Fair play for implementing proper response codes. I really don't know why Laravel (which I have huge respect for) doesn't encourage returning "201 Created" for success in the Create method by default. Good work!
Answer: The problem is when you set the header() location, it is overwriting the Laravel response.
If you add 201 status to the header() function - it should work I think:
$response = Response::make(null, 201)->header('Location', Config::get('app.url') . '/v1/users/' . $user->id, true, 201);
return $response;

Comment: Brilliant thanks, wasn't aware the `header()` method took those extra parameters.

Forcing Laravel response status code


How does one correctly avoid the avcodec_alloc_context3 leak from avformat_new_stream?

Question: This maddening thread describes the problem I'm having: a memory leak on shutdown due to some stuff allocated when avformat_new_stream is called.
Here's the valgrind stack trace from the leak:

1,447 (1,440 direct, 7 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 4
at 0x4C2FFC6: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x4C300D1: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x690DEFF: av_malloc (mem.c:87)
by 0x690E09D: av_mallocz (mem.c:224)
by 0x533D28A: init_context_defaults (options.c:128)
by 0x533D325: avcodec_alloc_context3 (options.c:164)
by 0x663D09E: avformat_new_stream (utils.c:4384)
by 0x4204B6: main (test_ffmpeg.cpp:918)

So clearly the problem is that when a AVFormatContext's stream's codec context's priv_data field is somehow not being freed.
My code frees the AVFormatContext with avformat_free_context. This calls ff_free_stream, which calls free_stream, which frees a few of the stream's codec context fields itself - but not the priv_data field!
Compared and contrast with the corresponding code in avcodec_close.
The suggested solution to the problem from the thread: "close the codec firstly before calling av_format_free_context". Presumably this refers to calling avcodec_free_context? - but I'm already doing this! Roughly following the structure in the muxing example, I have an encoder context created by my code, that's used to track the uncompressed input data. Then there's another encoder context created internally by avformat_new_stream (as above), which is used internally by FFmpeg. I close the former, because it was opened using avcodec_open2, but I don't close the latter, because it wasn't. I am following the maddening thread's advice, and yet here I am.
Furthermore, reading between the lines, using avcodec_free_context to free the AVStream's codec context is no good anyway, because when doing this (a) AVStream's codec field is deprecated, so this gives a bunch of warnings, and (b) there are no NULL checks in free_stream, so this crashes at runtime.
What I have done for now is drag in the appropriate bit of code from avcodec_close, and put it in my own code just ahead of the call to avformat_free_context:
#ifdef __GNUC__
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
#endif
    for(unsigned i=0;i<avf_context->nb_streams;++i) {
        AVStream *st=avf_context->streams[i];

        if(st->codec->priv_data&&
           st->codec->codec&&
           st->codec->codec->priv_class)
        {
            av_opt_free(st->codec->priv_data);
        }

        av_freep(&st->codec->priv_data);
    }
#ifdef __GNUC__
#pragma GCC diagnostic pop
#endif

So that fixes the leak, and it's clearly (to my eyes) a bug, and I need to file a bug report or something.
However the corresponding bug report is marked as fixed and closed... along with a link back to the maddening thread, and no further explanation. (This is why it is maddening!) So maybe I'm just using FFmpeg wrongly?
Can anybody confirm whether this is actually a bug in FFmpeg or not?
If it isn't a bug, what's the right sequence of calls to make?
(I'm using FFmpeg built locally from commit 03eb0515c12637dbd20c2e3ca8503d7b47cf583a. I had similar-looking problems with the one you get from the Ubuntu 16 package manager, which prompted me to build it myself with symbols and so on.)
Answer: I think you are using libav API incorrectly. As part of API control mechanism each library has a version.h file (for example ./libavformat/version.h) that defines a set of macros controlling public API of that library.
typedef struct AVStream {
    int index;    /**< stream index in AVFormatContext */
    /**
     * Format-specific stream ID.
     * decoding: set by libavformat
     * encoding: set by the user, replaced by libavformat if left unset
     */
    int id;
#if FF_API_LAVF_AVCTX
    /**
     * @deprecated use the codecpar struct instead
     */
    attribute_deprecated
    AVCodecContext *codec;
#endif
    void *priv_data;

#if FF_API_LAVF_FRAC
    /**
     * @deprecated this field is unused
     */
    attribute_deprecated
    struct AVFrac pts;
#endif

When you encounter deprecation warnings you should look for corresponding API control macro and compile libav and your code that uses libav with this macro set to 0 (with -DFF_API_LAVF_AVCTX=0 in case of codec field for example) overriding version.h content and ensuring that this deprecated API is not used anymore.
Comment: Thanks! - I know what the deprecation warning is for, and can guess what the ramifications of ignoring it might be ;) But if I take out the code that produces the deprecation warning, I get a memory leak. How do I fix the memory leak *without* this ugly workaround that I've had to come up with? That is my question.
Comment: After you turn this deprecated API off there will be no `st->codec` field and no code that initializes it and your additional cleanup code `av_opt_free(st->codec->priv_data);` will be unnecessary. Actually it looks like you are getting this leak _because_ you still have this deprecated stuff enabled. Typically devs don't really care about content of deprecated fields or correctness of deprecated methods. Suppressing deprecation warnings is wrong.
Comment: But it comes from the repo this way... so I guess this *is* a bug in FFmpeg then?
Comment: I would rather call this _API_ _deprecation_ _expenses_. Ideally it would be nice if old stuff got patched to work correctly, but who is going to update old stuff that shouldn't be used and that is going to be deleted soon anyway?
Comment: This assumes one can rebuild ffmpeg, how do you do when it's the version of e.g. a Linux distro which comes with all the toggles enabled?
Comment: @Jean-MichaëlCelerier I don't think it is possible to do anything in this case so you should just use ffmpeg as is, with all the toggles enabled, possibly adopting some workarounds in case if some problems pop up.


Convolution of difference distribution doesn't conserve expectation - faulty assumption?

Question: I have observations of a system which I hypothesize is defined as $T = A + G$ where $T,A, G$ are random variables. I've measured the empirical continuous distributions $T$ and $A$, and the question is, what is the distribution of $G$?
Assuming $A$ and $G$ are independent, this is just the convolution integral for $T-G$, that is, since $i-(i-k)=k$,
$$
P_G(G=k)=\int_{i=0}^\infty P_T(T=i)P_A(A=i-k)
$$
Because the data for $T$ and $A$ are empirical distributions, I computed discrete PMFs with various discretizations and the following results don't depend much on what resolution to discretize.
Inputting my data, I get a nice distribution for $G$, however, there are two things I don't understand.

While the discrete PMFs that approximate $A$ and $T$ sum to 1, the sum of $P_G =.543$. Do I just renormalize?
$E[G]=.78$ while $E[T]=1.52$ and $E[A]=1.32$. Mean of $G$ is not affected by (1). I know even for correlated variables, $E[A+B]=E[A]+E[B]$, so something must be wrong. The only assumption I've made is that $A$ and $G$ are not correlated (so the convolution applies). Is this result telling me $A$ and $G$ are correlated? What can I do to find out if they are, given what I have? I cannot observe $G$ without $A$.

Edit: changed "Assuming $A$ and $G$ are not correlated" to "Assuming $A$ and $G$ are independent" per Dilip since the terminology used was incorrect.
Comment: Can you assume that $A$ and $G$ are (at least approximately) independent (not just uncorrelated)? If so, then you want to *deconvolve* $T$ and $A$ (not $G$), which is a different formula--but relatively straightforward. [The mistake here lies in assuming that $G=(A+G)-A$ can be computed with convolutions. To see why not, consider a case where $G$ is constantly $0$ but $A$ varies. Then $A+G=A$, but your formula for $A-A$ is really a formula for $A_1-A_2$ where the $A_i$ are independent and have the same distribution as $A$: their difference definitely is not zero.]
Comment: I can assume that $A$ and $G$ are independent but I'm not 100% sure they are. It seems like the appropriate first step is to assume independence. I think I see several of the flaws in my logic. So the problem becomes solving $T=G \ast A$ for $G$?
Comment: @whuber Tried a bunch of things to solve that. First attempt: ifft(fft(T)/fft(A)). Did not work. This is a well studied problem. Seems like the most effective approach is solving the problem $T=Ag$ where T is a vector, A is a matrix A(k,i) filled with the values A(k,i)=$P_A(k-i)$ from the sum convolution, and g is the unknown probability distribution vector. Tried solving this with truncated SVD. Result is ugly, namely lots of negative values. Pointers?
Answer: Your statement

Assuming $A$ and $G$ are not correlated, this is just the convolution integral

is false. The distribution of the sum or difference of independent
random variables can be obtained via convolution-type formulas, but
this formula does not hold for uncorrelated random variables.
The key issue here is

I've measured the empirical continuous distributions $T$ and $A$

whereas what you need is the (empirical) joint distribution of $T$ and $A$;
the individual (a.k.a. marginal) distributions do not suffice.

The only assumption I've made is that A and G are not correlated (so the convolution applies). Is this result telling me A and G are correlated?

What the result is telling you is that $A$ and $G$ are dependent random variables. They might be correlated, and they might not be correlated;
it all depends on the joint distribution.  For example, if $(A,G)$
is uniformly distributed on the unit disc, then $A$ and $G$ are
uncorrelated but dependent random variables.
Applying your convolution formula to the sum/difference of $A$ and $G$
in this case will not give you the distribution of $A\pm G$.
Comment: Thanks for reminding me of the fundamental differences between independence and correlation, I was definitely loose with my terminology.
Comment: As I stated above, I don't have access to the joint distribution of T and A. I may have, however, the values of predictors for both. Still, I'd like to estimate $G$ using the simplest model first before integrating over dependent variables. It seems to me my problem is more that I've assumed $T=G+A$ means $G=T-A$ (false), as @whuber points out.


Sources of musical tension

Question: I understand that harmonic tension occurs when you hang on a chord which "wants" to resolve, like a chord with a leading tone or certain dissonant intervals.  Are there other sources of harmonic tension?
What are non-harmonic sources of tension?
Comment: You can also have tension when the listener isn't sure where the actual tonic chord of the music would be.  For example, the first section of the [Sense8 theme](https://www.youtube.com/watch?v=f9ZqGFiKwA8) jumps around a few potential keys, which (in my opinion) creates tension right away.
Comment: Whatever you used to be diong, do the opposite. Surprise the listener. Then go back to something you used to be doing to reslove the tension.
Answer: The pause is one that comes straight to mind - the rhythm stops, and everything hangs for some moments. Rubato has a similar effect, as do rallentando and accelerando.
Crescendo and decrescendo can also be used to produce tension - especially in the way Beethoven used it. Never felt certain that it wasn't his frustration showing through...
A key change is also effective, although I've never felt an effect lasting more than a bar or two.
EDIT: 5/4 in itself tends to create tension. There have been several themes for suspense type tv programmes that have used 5/4. As already mentioned, a change of time sig. for a single bar is a good ploy. Particularly in a piece with a good foot-tapping rhythm. The listener just gets disturbed. (As does the player who may not be expecting it !)
Change of instrumentation, most effective within an orchestral setting, will bring large mood changes to a piece.
And, of course, the other factor that makes music - silence. Unexpected tacet is truly effective - whether it's in an orchestra or a rock band.
Comment: @DontVoteMeDown - and there's Holst's Mars.
Comment: @DontVoteMeDown - Classic example of a "TV suspense" theme in 5/4 - Mission: Impossible
Comment: *"5/4 in itself tends to create tension. There have been several themes for suspense type"* true that, Halloween's theme can prove it.
Answer: In very general terms, I'd say that deliberately withholding some element of music that the listener knows (or thinks they know) is coming will always build tension.
It could be a harmonic a resolution that's a long time coming - for example, if you use Emin, Fmaj and Gmaj chords, then the listener might assume you're in the key of Cmaj/Amin, because that's traditional key that fits those in best.  If you hold off from actually using either of those chords, it creates tension.
In dubstep and related genres, there's "the drop", which is where the artist deliberately withholds the heavy bass beat for as long as possible, creating tension. (See this Lonely Island sketch.)
Or perhaps skipping a beat that the listener really feels you should play (like leaving the first beat of the bar silent).
Overall, figure out something obvious that your listener wants and expects to happen next, and see if you can delay it.
Answer: Syncopation is a very common form of non-harmonic tension.  From Wikipedia,

Syncopation involves a variety of rhythms which are in some way
  unexpected which make part or all of a tune or piece of music
  off-beat. More simply, syncopation is a general term for "a
  disturbance or interruption of the regular flow of rhythm": a
  "placement of rhythmic stresses or accents where they wouldn't
  normally occur." The correlation of at least two sets of time
  intervals. Also known as an "Uneven movement from bar to bar".

There are a variety of ways to achieve syncopation, including:

Carrying a beat over the bar line
Accenting the "weak" beat instead of the "strong" beat, for example in 4/4 accenting the 2 and the 4 instead of the 1 and the 3
Dropping the "strong" beat
Polyrhythms, or playing two different time signatures against one another.  A common trope is playing 2 against 3. How to play 2 against 3 polyrhythm evenly?

Playing multiple notes in the space for a differrent number of beats will also create tension.  If you play a triplet over 2 beats then that's a hemiola.
In addition, gradually increasing the tempo, accelerando, is an easy way to increase tension as is increasing the volumne.
Switching time signatures will increase tension as will creating melodies that are a little longer or a little shorter than the standard 32-bar length.  In American old-time these are called "crooked tunes" and will drop or add a half-bar to two bars of length to the A or B part of an AABB or AABA tune.
Big jumps up or down in the melody can create excitement.  The general rule is "Big step up, small steps down,"  or vice versa.
Finally: silence.  Silence can create enormous tension, especially in music that's "notey".
Comment: To go along with the accelerando...sometimes *not* increasing the tempo when the music really feels like it wants to take off will increase tension. This goes along with Tim's comment above regarding Beethoven's crescendos, as well. The 1st mvt of the Moonlight Sonata has a beautiful crescendo which creates musical tension as one wants so badly to give into the passion and run away with the tempo. As the tempo holds steady, however, the self restraint increases the tension in the music.
Comment: @HeatherS. - so true. Many many times crescendo is translated as cresc/accel. And it's the holding back of tempo that underlines its success.
Comment: Syncopation in pop type songs is so commonplace now, it's hard to find songs without 'pushed' notes over the barlines.
Comment: Listen to this version of the 15th c. Spanish song [_Un Sarao de la Chacona_](https://youtu.be/HlCz8MlQKU4). The opening 16 bars are highly syncopated, and when the second phrase arrives in simple 3 you can feel a distinct "resolution" of tension.
Comment: @Tim, yes, that is one of the most important things I learned as a performer was to keep the tempo steady despite changes in dynamics. And knowing that, now as a composer I find it is important to mark accelerandos/ritards only in very specific places in order to communicate that the tempo is to remain steady everywhere else.
Answer: Once you have established a major or minor tonality, almost anything without the tonic and third (combined) will sound unresolved. That works well enough to keep a lot of pop music going. This is accomplished through simple chords of stacked thirds (Em, D, etc.), or by using suspended notes. The 4th and tonic sus4 chords sound particularly unresolved.
Briefly stepping outside of the key works well in jazzier settings. For example, if you are in the key of C, moving directly to a Dbmaj7 chord then immediately back to Cmaj7 creates tension and release. This works particularly well with minor chords and chords that contain extensions. (For example, Cmaj7 Dm7 Ab9 G9 Cmaj7, which is actually a b5 substitution.)
Comment: @S. Burt, From the OP: "Are there other sources of harmonic tension?"
Comment: b5 substitution commonly called a tritone substitution.
Comment: Aren't these all examples of harmonic tension?  The OP is looking for non-harmonic examples.

Sources of musical tension


Python: writing function: how to count average sentence length in any text

Question: my task is to count average sentence length in a text(with many sentences). I tried two ways, first one like counting tokens in words, but here I don't know how to tell the program, that i want word count in the sentence, and not tokens.
def satzlänge(text):
    sentences = sent_tokenize(text)
    total = 0
    count = 0
    for x in sentences:
        length = len(x)
        total += length
        count +=1
    avg_length = total / count
    print(total)
    print(count)
    return(avg_length)

print(satzlänge('Hello, this is John. How are you? How is your mother?'))

second one I tried to tokenize the sentences in words, but here I get separate lists of words, and I can't add the len(words) together with sum to get the result I need. Error message: TypeError: 'int' object is not iterable
def satzlänge(text):
    sentences = sent_tokenize(text)
    for x in sentences:
        tokenizer = RegexpTokenizer(r'\w+')
        words = tokenizer.tokenize(x)
        print(words)
        print(len(words))
    avg = sum(len(words))/len(sentences)
    return(avg)

print(satzlänge('Hello, this is John. How are you? How is your mother?'))

Answer: if you have sentence and you want to get the number of word in the sentence you can use
length = len(sentence.split())

Comment: It says: AttributeError: 'list' object has no attribute 'split'.. thanks anyway
Comment: use this function on the variable x (a sentence), length = len(x.split())


Solving $\frac{x^{11}+x}{x^7+x^5}=\frac{205}{16}$

Question:
Blockquote
Find $x$ satisfy the equation$$\frac{x^{11}+x}{x^7+x^5}=\frac{205}{16}$$

Here is my work
$$\frac{x^{10}+1}{x^6+x^4}=\frac{205}{16},\quad\quad t:=x^2$$
$$\frac{t^5+1}{t^3+t^2}=\frac{205}{16}$$
$$16t^5-205t^3-205t^2+16=0$$
$t=-1$ is a root of the polynomial but not sure how to find the other roots.
Another approach I tried is dividing numerator and denominator of the original fraction by $x^6$,
$$\frac{x^5+\frac1{x^5}}{x+\frac1x}=\frac{205}{16}$$
$$x^4-x^2+1-\frac1{x^2}+\frac1{x^4}=\frac{205}{16}$$
Comment: Also if $x$ is a solution, then so is $\frac{1}{x}$. Because $\frac{x^{10}+1}{x^6+x^4}=\frac{x^5+\frac{1}{x^5}}{x+\frac{1}{x}}$. So $x=\pm\frac12$ are two more solutions. With $t=\frac14$, now your $t$-polynomial can be reduced to a quadratic.
Comment: Can you guess any solutions to the last equation? There are 4 values which are worth trying.
Comment: I see the $x^{10}+1$ in the numerator and wonder how it might relate to $205$. If $205$ comes from reducing a fraction, can we find some $10$th power (plus $1$) that is a multiple of $205$? The first $10$th power you might try is $2^{10}$. And it works: $\frac{2^{10}+1}{2^6+2^4}=\frac{205}{16}$. So by ad hoc thinking, one solution is $x=2$. Since the exponents are even, $x=-2$ works as well. And $t=4$ is one root, so you can reduce your 5th degree polynomial in $t$ to a 4th degree polynomial. And you also have $t=-1$, so further to a 3rd degree.
Answer: HINT
You are on the right track!
Notice that
\begin{align*}
x^{2} + \frac{1}{x^{2}} = \left(x + \frac{1}{x}\right)^{2} - 2
\end{align*}
Similarly, we do also have that
\begin{align*}
x^{4} + \frac{1}{x^{4}} & = \left(x^{2} + \frac{1}{x^{2}}\right)^{2} - 2\\\\
& = \left[\left(x + \frac{1}{x}\right)^{2} - 2\right]^{2} - 2
\end{align*}
Now you can make the substitution
\begin{align*}
u = x + \frac{1}{x}
\end{align*}
Then we get that
\begin{align*}
[(u^{2} - 2)^{2} - 2] - (u^{2} - 2) + 1 = \frac{205}{16} \Longleftrightarrow u^{4} - 5u^{2} + 5 = \frac{205}{16}
\end{align*}
which is a biquadratic equation.
Can you take it from here?
Answer: As an alternative from here we have
$$\frac{t^5+1}{t^3+t^2}=\frac{205}{16} \iff \frac{(t+1)(t^4-t^3+t^2-t+1)}{t^2(t+1)}=\frac{205}{16}$$
and since $t=x^2\ge 0$
$$\iff \frac{t^4-t^3+t^2-t+1}{t^2}=\frac{205}{16} \iff t^2-t+1-\frac1 t+\frac1{t^2}=\frac{205}{16}$$
$$\iff t^2+2+\frac1{t^2}-t-\frac1 t-\frac{221}{16}=0 \iff \left(t+\frac1t\right)^2-\left(t+\frac1t\right)-\frac{221}{16}=0$$
and by $u= t+\frac1t$ we obtain

$$u^2-u-\frac{221}{16}=0$$


Special Characters in C code

Question: I am a freshman in college taking an Electrical and Computer Engineering course using C code. We are working on the final project (coding a game of Old Maid) right now and I am trying to get these characters printed in the console:
 ██████╗ ██╗     ██████╗         ███╗   ███╗ █████╗ ██╗██████╗ ██╗
██╔═══██╗██║     ██╔══██╗        ████╗ ████║██╔══██╗██║██╔══██╗██║
██║   ██║██║     ██║  ██║        ██╔████╔██║███████║██║██║  ██║██║
██║   ██║██║     ██║  ██║        ██║╚██╔╝██║██╔══██║██║██║  ██║╚═╝
╚██████╔╝███████╗██████╔╝        ██║ ╚═╝ ██║██║  ██║██║██████╔╝██╗
 ╚═════╝ ╚══════╝╚═════╝         ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝╚═════╝ ╚═╝

But what comes out is:
 ??????? ???     ???????         ????   ???? ?????? ?????????? ???
????????????     ????????        ????? ???????????????????????????
???   ??????     ???  ???        ?????????????????????????  ??????
???   ??????     ???  ???        ?????????????????????????  ??????
?????????????????????????        ??? ??? ??????  ?????????????????
 ??????? ???????????????         ???     ??????  ????????????? ???

I even tried saving the .c file differently as well as reading the characters in from a text file as an array and printing that, but the same problems occurs. Are these characters just impossible to use?
I am working on Visual Studios 2015. Any help would be appreciated, I'm still kinda new to this stuff.
Comment: What is your console (console emulator) and what is [the codepage](https://en.wikipedia.org/wiki/Code_page) used by it? You should know which codepage is active (or how to change it to the needed one, probably some from old DOS times for [╔ and ╝ symbols](https://en.wikipedia.org/wiki/Box-drawing_character) like [437 code page](https://en.wikipedia.org/wiki/Code_page_437); or try to unicode?) and find codes of symbols used for formatting.
Comment: Possible duplicate of [Unicode characters in Windows command line - how?](http://stackoverflow.com/questions/388490/unicode-characters-in-windows-command-line-how)
Answer: You should consider the use of wide characters.
See the C standard library wchar.h.
For example:
#include <stdio.h>
#include <wchar.h>
#include <stdlib.h>

int main(int argc, char** argv) {
  wchar_t widechar = L'\x220E';

  wprintf(L"%lc\n", widechar);

  return EXIT_SUCESS;
}

The output:
∎

Special Characters in C code


IP and network config

Question: I have a question regarding IP and network config.
I have Ubuntu OS with KVM - Virt Manager and I installed two VM Guest OS(Ubuntu).I config the Host IP with 172.X.X.1(local IP) and try to access the two internal VM image with 172 .X.X. 2 and 172 .X.X. 3) with bridge connection but i cant able to access the two internal VM via external system.I tried but not giving successful result.
Its any thing i missed in the bridge configuration or any thing in internal VM image.
How to config this bridge connection and internal VM image?
172.x.x.1 ------ HOST - Ubuntu
|---- VM image 1 ---- > Ubuntu ---> 172.X.X.2
|---- VM Image 2 ---- > Ubunut ---> 172.X.X.3 ( I cant access the VM image via SSH and even in browser too)
But the Host System is works fine in most of the system - accessible via SSH and browser.
Comment: could you provide output of `ip link` , `ebtables -L` and `iptables -L -nv`. Im almost certain that its because of macvtap devices.
Answer: Have you enabled packaet forwarding? If not follow the instructions here

IP and network config


Explicit calculation to show a complex is exact

Question: I know that the image of the preceding map, when equaling the kernel of the next map, means the complex is exact. Furthermore, this operator, listed below, is linear, but I am having a hard time explicitly calculating it. $E_n(A)$ is the set of all maps $f$ and has an abelian structure where map addition is as $$f+g: G^n \to A :: x \mapsto f(x) + g(x)  .$$ The differntial map used is:
$$d_n: E_n(A) \to E_{n+1}(A)::  d_n(f)(g_0,\ldots,g_n) = g_0\cdot f(g_1,\ldots,g_n) + \sum_{i=1}^{n}\,(-1)^i f(g_0,\ldots,g_{i-2},(g_{i-1}\,g_i),g_{i+1},\ldots,g_n) + (-1)^{n+1}\,f(g_0,\ldots,g_{n-1})$$ where $$f: G^n \to A \in E_n(A)$$ and the elements $$g_0,\ldots,g_n\in G$$. If $n=0$, this is meant to reduce to the following: if $a \in E_0(A) = A$ then $d_0(a) : G \to A$ is the map $$d_0(f)(g) = g\,a - a.$$

So in the explicit calculation, I start by showing, $$d_{n+1}(f)(g_0,\ldots,g_{n+1}) = g_0\,f(g_1,\ldots,g_{n+1}) + \sum_{i=1}^{n+1}(-1)^i\,f(g_0,\ldots,g_{i-1}g_i,\ldots,g_{n+1}) + (-1)^{n+2}\,f(g_0,\ldots,g_n)$$ and then apply on each of these three terms, $d_n(f)$.
For instance, the first term,
$$(d_n \circ d_{n+1})(f)(g_0,\ldots,g_{n+1}) = d_n \circ \big( g_0\,f(g_1,\ldots,g_{n+1}) \big) = g_0\,d_n(f)(g_1,\ldots,g_{n+1}) = g_0\,g_1\,f(g_2,\ldots,g_{n+1}) + g_0\,\sum_{i=1}^n\,(-1)^i\,f(g_1,\ldots,(g_{i-1}g_i),\ldots,g_{n+1}) + g_0\,(-1)^{n+1}f(g_1,\ldots,g_n)$$
(with corrections)
On the last term, I get,
$$(d_n \circ d_{n+1})(f)(g_0,\ldots,g_{n+1}) = d_n \circ \big( (-1)^{n+2}\,f(g_0,\ldots,g_n)\big) = (-1)^{n+2}\,d_n(f)(g_0,\ldots,g_n) = (-1)^{n+2}\,(-1)^{n+1}\,f(g_0,\ldots,g_{n-1}) = (-1)^{n+2}\,g_0\,f(g_1,\ldots,g_{n}) + (-1)^{n+2}\,\sum_{i=1}^n\,(-1)^i f(g_0,\ldots,(g_{i-1}g_i),\ldots,g_n) + (-1)^{n+2}\,(-1)^{n+1}\,f(g_0,\ldots,g_{n-1}) = (-1)\,f(g_0,\ldots,g_{n-1})  .$$
(with corrections)
For the middle term I got,
$$ d_n(f) \circ \left( \sum_{i=1}^{n+1}\,(-1)^i\,f(g_0,\ldots,(g_{i-1}g_i),\ldots,g_{n+1}) \right) = \sum_{i=1}^{n+1}\,(-1)^i\,d_n(f)(g_0,\ldots,(g_{i-1}g_i),\ldots,g_{n+1}) = nightmare!   .$$
This is 3 terms for each $i$ - how does one do this in general?
I'm pretty sure the first and last terms will cancel out in the composition of these neighboring differentials and that the summations will cancel out, but I cannot even get the terms not involing the summation to cancel. If one could show this explicitly I would greatly appreciate.
P.S. I forgot to add... For $f: G^2 = G \times G \to A$, how do I show:
Let $Z^2(A)$ be the set of factor systems, denote the set of maps $f$ s.t. $$g\cdot f(g',g'') - f(g\,g',g'') + f(g,g'g'') - f(g,g') = 0$$ for the $g$'s in $G$. Show that every element in $ker\;d_2$ gives rise to a factor system.
"gives rise"? Thank you for any clarification on that part too!
Comment: I assume $G$ is a group, $A$ is an abelian group, $G$ operates o $A$ and that's it, or are there additional constraints? Also, do you really want to show tha tthe complex is *exact*, or just that it is a *complex* in the first place?
Comment: There are no other constraints than what you said, and I left out - A is a G-module. Umm. I guess to show it is a complex to begin with, i.e., that $d_{n+1} \circ d_n = 0$.
Answer: To see that this really is a complex, note that $d_n(f)$ has $n+1$ summands:

Summand 0 is $g_0\cdot f(g_1,\ldots,g_n)$,
for $1\le i\le n$ summand $i$ is $(-1)^if(g_0,\ldots, (g_{i-1}g_i),\ldots, g_n)$ and
finally the $n+1$st summand is $(-1)^{n+1}f(g_0,\ldots, g_{n-1})$

This can be written more uniformly:
Let
$$\tilde f(g_0,\ldots, g_{n+1})=g_0\cdot f(g_1,\ldots,g_n)$$
Then the $i$th summand, $0\le i\le n+1$ is just
$$(-1)^i\tilde f(g_{-1},\ldots, (g_{i-1}g_i),\ldots, g_{n+1})$$
with $g_{-1}=g_{n+1}=1$. That is, we now have only summands of the "middle type".
Now if we consider $d_{n+1}\circ d_n$, then the summands in the result are obtained by combining two pairs of consecutive arguments - first one pair, then the other. However, the same combinig of arguments can be obtained in the other order. The sign factors cause thse summands to cancel, i.e. one time we obtain $(-1)^i(-1)^j$, say, and with reverse order $(-1)^j(-1)^{i+1}$ or $(-1)^{j-1}(-1)^{i}$. In summary, $d_{n+1}\circ d_n=0$.

Regarding your gives rise question. The condition written there is precisely that $f\in\ker d_2$, i.e. the left hand side is simply $d_2(f)(g, g', g'')$.
Comment: Oh I see that now - it makes sense that $d_{n+1} \circ d_n = d_n \circ d_{n+1}$ - thank you!! Also, I see what the second part means - probably much clearer if I just saw it as a $d_2$ operation.


Drush Permissions Issue

Question: With the help of the support team of my dedicated server, I have setup Drush globally on my server. Now I can run multiple Drush versions like drush6 status, drush7 status, drush8 status with a little tweak in my .bashrc file
One problem I am unable to resolve is that when I download or update drupal it starts giving 500 Internal server error.
My Support Team Told me
When you use Drush to download Drupal, it gives files 664 permission. The PHP handler in the server is suPHP, which requires 644 file permission, so you need to run the file permission fixing script for the account after using Drush install/update commands.
Is there any way we can instruct Drush to download the files with 644 permission?
Answer: Drush does not handle platform-specific issues such as file permissions. To get rid of group write, just run chmod -R g-w /path/to/drupal. If you wanted to do this automatically, you could make a post-update hook for the pm-updatecode command.  See the example Drush pm_update hook file for one example post hook function.
In your case, you would want something like this:
function drush_myhook_post_pm_updatecode() {
  passthru('chmod -R g-w .');
}

Save that in a file named myhook.drush.inc, and put it somewhere that Drush will find it, such as DRUPAL_ROOT/drush.  Then run drush cc drush, and your hook should run the next time you call drush pm-update.
Answer: This isn't a Drush issue. It's about the configuration of your shell. The file creation mode is controlled by the umask. The correct solution is to set umask to 0022 (022) in the start up files of your shell. Or better, ask your hoster to add it to the global startup files which they already should have done if suPHP requires it ;-)

Drush Permissions Issue


Excel 2003 difference between .xlm behaviour

Question: I have two Excel 2003 .xlm files, each with one makro. The text in both macros is the same. I run macros from workmap (.xls file) with only one sheet. Cells in .xls file are preformatted, vor example Cell(1, 5) has Custom Format "MMMM".
The first thing the macro does is to put value in preformatted cell in .xls file, for example with the command: Cells(1, 5).Value = "01.11.2010".
If I do it with one .xlm file, the result is "November".
If I do it with another .xlm file, the result is "01.11.2010".
Other preformatted cells working differently too. If I put in cell the value "8,00", first .xlm put it as number, the second as a text. It goes even the both .xlm files are opened.
What could be the reason for this discrepancy?
Comment: Please consider posting a link to your workbooks (via DropBox or GoogleDocs). It will be easier to troubleshoot with all of these information.
Comment: the column format mask appears to be different.
Answer: Regional settings play a factor here so my setup may be a bit different than what you have.  However in A2 I keyed in 1/11/2012.  I set B2 formula = A2 and C2 formula = A2.
Notice how the display is different for each of the two results.  This is because I have formatted the columns B and C to be in different display (masks) formats.
It is possible you have a similar issue on the sheet with "NOVEMBER" being displayed.


Points of intersection of two functions

Question: Through the following steps I found the x-coordinates of the intersection points of two functions:
$(x)= -x^{2}+3x+1\:  and\: g(x)=3/x $
The numbers I found are x=(1, 2, 3)
But on the graph, one of the points has a negative x value, could you guys point me to anything I have missed in my calculations.
$-x^{2}+3x+1=3/x
\\x(-x^{2}+3x+1)=3
\\x=3
\\and
\\-x^{2}+3x+1=3
\\x^{2}-3x-1=-3
\\x^{2}-3x-1+3=-3+3
\\x^{2}-3x+2=0
\\(x-1)(x-2)=0
\\So
\\x=1 \:and\: x=2
$
Comment: $x(-x^{2}+3x+1)=3\quad$ doesn't implies $x=3$ or $-x^{2}+3x+1=3$. Instead of that we have
$$x(-x^{2}+3x+1)=3\iff -x^3+3x^2+x-3=0$$
Comment: your mistake is $-x^2+3x+1=3$ this is only true when $x=1$ you need to solve $f(x)=-x^3+3x^3+x-3=0$ knowing that $x=3$ is one solution. Divide $f(x)$ by $(x-3)$ to get a quadratic which you can solve to get the other 2 roots.
Answer: $x(-x^{2}+3x+1)=3\quad$ doesn't imply $x=3$ or $-x^{2}+3x+1=3$. Instead of that we have
$$x(-x^{2}+3x+1)=3\iff -x^3+3x^2+x-3=0\iff-(x+1)(x-1)(x-3)=0$$
Therefore the graphs of the functions mets at $x=-1$, $x=1$ and $x=3$.
Answer: The first equation is a cubic, $  -(x+1)(x-1) (x-3) = 0,  $
with roots $ x = \pm 1, 3 $ at points of intersection.

Points of intersection of two functions


Lower voice in Chopin's Nocturne Op 9 No 1 not adding up to 6/4 time signature

Question: I wanted to learn Chopin's Nocturne Op 9 No 1. In most cases, the note values add up to the time signature of 6/4. However, in measure 6 of the right hand part, I counted the note values of the lower voice (stems point down) and it does not fully occupy a bar of 6/4 time.

How am I supposed to play it? On the version provided by virtualsheetmusic.com, there is a dotted half rest to fill in the gap. MuseScore hides the dotted half rest. Why was MuseScore designed like that?
Comment: Don't really understand your question. Whoever used musescore to do the engraving must have chosen to hide the rest. Most editions exclude the rest, though some include it. You can browse many editions at https://chopinonline.ac.uk/ocve/. But question about how musescore was designed would have to be directed to musescore.
Comment: @phoog There is no autograph, but most editions, both on IMSLP and the Chopin Variorum, which is linked in my previous comment, omit the rest.
Comment: M6? I see nothing wrong. It all lines up and counts perfectly. Play it so the notes aligned vertically go together. Anacruces are usually not counted as m1, though. Do you mean 1st bar on line 3?
Comment: @Tim You're right that the anacrusis shouldn't be counted as measure 1, but in this score the measure numbers are shown at the start of each line.
Comment: @Aaron "Most editions exclude the rest": do you know whether there are any sources that show whether Chopin included the rest?  (I'd guess that the omission reflects his practice because normally one would expect it to be included, but I don't have time right now to look myself.)
Comment: @Aaron I had assumed from your not having mentioned an autograph that none probably exists.  So the next questions would be whether Chopin participated in the preparation of any of these editions and perhaps the relative timing of those editions that include the rest and those that omit it. (In other words, sources other than autograph manuscripts may help answer the question.) Also since musescore is open source it is probably fairly easy to find information about design decisions.
Comment: @phoog Have at it.
Comment: @Tim, Good notation practice would allow you to read the top staff without benefit of the bottom staff, left to right *without having to work backward from the end of the measure to know when the Db comes in*. Alignment should reinforce and agree with note and rest values, not substitute for them. (Measure numbers here are printed for the first measure of each line. The first measure is assigned 1, even if incomplete, for reference purposes)
Comment: I am talking about measure 6.
Comment: @DjinTonic - not in my, or most other muso's world. So call anacrucis bar 0. 1st full bar generally known as bar 1.
Comment: @Aaron as mentioned, no time just now.  I was hoping you'd noticed something relevant in your research.  As it seems you didn't, I'll continue to hope that someone else who reads this either knows or is curious enough to spend some time finding out.
Comment: @Tim I stand corrected. You are right  about measure numbering.
Answer: A dotted half rest is not printed for the lower voice in the upper staff at the start of that measure, because it is unnecessary, and the score is much neater without it.  The pianist can see when to play that D-flat because it's at the half-way point in the measure; it lines up with the beam group in the lower staff.

Use of Rests
To keep the stave as uncluttered as possible, use rests sparingly for additional parts: when both hands are already playing the pianist  does not want to read extra rests, except to clarify the placing of additional parts that have independent rhythm.

— Gould, Behind Bars - Keyboard / Part-writing, pp. 311–312
Comment: @phoog Perhaps the long note in the upper voice should be written as a dotted minim tied to a mimim tied to quaver?
Comment: I've played that many times, and it had never occurred to me to wonder when the second-to-top voice came in. There had _been_ no second-to-top voice until that moment. If I'd seen a half rest in that measure, I'd have wondered what it was about. :) For me, the vertical alignment in most editions of Chopin is what my eyes rely upon: how else to compare 11ths with anything else. :)
Comment: "except to clarify the placing of additional parts that have independent rhythm": to my eye, at least, that is exactly why the rest is better included than omitted.
Comment: @ElementsinSpace A **dotted** half note tied to a **regular** half note tied to a **regular** eighth note is the way that Frédéric Chopin did it himself: https://i.stack.imgur.com/sNtVK.png
Comment: @Vighnesh That does look neater for the top voice, doesn't it? But the rest at the start of the measure is still unnecessary clutter.  PS I'm not sure that really is Chopin's handwriting.
Comment: If you're a beginner, then learn from a version that _does_ include the dotted half rest to make it easier to play the first D♭ note in the lower voice.


Unable to enter a kubernetes pod. Error from server: error dialing backend: dial tcp: lookup (node hostname) on 220.127.116.11:53: no such host

Question: We have deployed a K8S cluster using ACS engine in an Azure public cloud.
We are able to create deployments and services but when we enter a pod using "kubectl exec -ti (pod name) (command)" we are receiving the below error,

Error from server: error dialing backend: dial tcp: lookup (node hostname) on 22.214.171.124:53: no such host

I looked all over the internet and performed all I could to fix this issue but no luck so far.
The OS is Ubuntu and 220.127.116.11 is a public IP from Azure used for DNS.(refer below link)

https://blogs.msdn.microsoft.com/mast/2015/05/18/what-is-the-ip-address-168-63-129-16/

I've already added host entries to /etc/hosts and entries into resolv.conf of the master/node server and nslookup resolves the same. I've also tested by adding --resolv-conf flag to the kubelet but still it fails. I'm hoping that someone from this community can help us fix this issue.
Comment: As far as I know, If you provide limited resourcesQuota to namespaces then you create the number of resources there, for example, deployments, pods, services, PVC etc .Therefore, all of the resourcesQuota are consumed by these k8s resources. Now if you are trying to consume more resource quota by using another api call or request. in your case, you are using "kubectl exec -ti (pod name) (command)" then it will be rejected by Admission Controller.  I would suggest you resolve resource quota limitations.  You can also dig up with Admission controller further. Perhaps there is a flag to ignore it
Comment: Can you run any other `kubectl` commands ?
Comment: @Rico yeah we are able to run other kubectl commands and they work fine.
Answer: Verify the node on which your pod is running can be resolved and reached from inside the API server container. If you added entries to /etc/resolv.conf on the master node verify they are visible in the APIserver container, if they are not, restarting the API server pod might be helpful
Answer: The problem was in VirtualBox layer
sudo ifconfig vboxnet0 up

Solution is taken from here https://github.com/kubernetes/minikube/issues/1224#issuecomment-316411907

Unable to enter a kubernetes pod. Error from server: error dialing backend: dial tcp: lookup (node hostname) on 168.63.129.16:53: no such host


conditioning check boxes and radio buttons in flask

Question: I have 3 basic html pages created in flask. I created a simple questionnaire having check boxes and radio buttons as shown in the image below.
I want to be able to add a condition such that if e.g(male + musical instruments + football + volleyball)then sports.html and entertainment.html should be suggested. Similarly if (female + music + volleyball) then science.html and entertainment.html should be displayed.
I just want to be able to recommend pages based on the questions answered. Thanks
Answer: Here is a Javascript/jQuery script that will alert "The best choices are sports.html and entertainment.html" if male + musical instruments + football + volleyball is selected and
"science.html and entertainment.html are the best choices" if female + music + volleyball are selected.

<input type="radio" id = "male" name ="sex" value="male">Male

<input type="radio" id = "female" name = "sex" value="female">Female

<input type="checkbox" id = "mi"> Musical Instruments

<input type="checkbox" id = "football"> Football

<input type="checkbox" id = "volleyball"> Volleyball

<input type="checkbox" id = "music"> Music

<button type="button" onclick = "suggest()">Submit</button>


Python subprocess does not work with complex linux commands

Question: I have a command in Linux which gives me following output:

1425463080: 0.0000000000e+00 0.0000000000e+00
  1425463085: 0.0000000000e+00 0.0000000000e+00
  1425463090: 0.0000000000e+00 0.0000000000e+00
  1425463095: 0.0000000000e+00 0.0000000000e+00
  ...

To make it more human readable I am using awk command as follows:
my command | awk 'NR > 2 {printf "%s %0.2f %0.2f\n", $1, $2, $3;}'

Now to remove extra :, I am using sed as follows:
my command | awk 'NR > 2 {printf "%s %0.2f %0.2f\n", $1, $2, $3;}' | sed 's/://g'

With this I get following output as expected:

1425463715 0.16 0.42
  1425463720 0.16 0.42
  1425463725 0.16 0.42
  ...

Now I want to run above command through Python. But I am facing so many problems while running it through Python subprocess module. For example I am facing following error:

('', 'awk: NR > 2 {printf "%s %0.2f %0.2f\nawk:                ^ unterminated string\n')

Even if I assign command as:
cmd = """ my command | awk 'NR > 2 {printf "%s %0.2f %0.2f\n", $1, $2, $3;} """
subprocess.Popen(cmd, ...)

... still it does not work.
Please let me know if my Linux command is correct. If yes then how can I execute it through Python subprocess.
Comment: Your sample `cmd` contains a stray `'` single quote before the first `|`. Also, the `\n` is not escaped.
Comment: And you are missing the trailing quote on the awk snippet
Comment: Did you set `shell=True`? Without that you are trying to run a single executable named `my command' | awk 'NR > 2 {...etc..`
Comment: Yes, tried with shell=True as well
Comment: You're missing the opening quote around `my command`
Comment: @Avinash True but I wanted to handle that on Linux cmd level
Comment: Oops, sorry that was typo error. Corrected extra \'
Answer: You need to escape the \n escape sequence; Python interprets that too; you are also missing a closing ' single quote in the awk command:
cmd = """ 'my command' | awk 'NR > 2 {printf "%s %0.2f %0.2f\\n", $1, $2, $3;}' """

or use a raw string:
cmd = r""" 'my command' | awk 'NR > 2 {printf "%s %0.2f %0.2f\n", $1, $2, $3;}' """

Note that Python is perfectly capable to do this work too, removing the need to use an extra shell binary in between to interpret the shell syntax.


need help, positioning values in rrdtool

Question: DEF:clients=cccam.rrd:kliensek:AVERAGE
DEF:activeclients=cccam_actclient.rrd:activeclients:AVERAGE
"LINE1:clients#0000FF:Connected   clients" COMMENT:" \n"
"LINE2:activeclients#99FF00:Active clients" COMMENT:" \n"
GPRINT:clients:LAST:'Current'\%5.0lf%s COMMENT:" \n"
GPRINT:clients:MIN:'Min: '\%5.0lf%s  COMMENT:" \n"
GPRINT:clients:MAX:"Max:\:%5.0lf%s\n"
GPRINT:activeclients:LAST:'Current'\%5.0lf%s COMMENT:" \n"
GPRINT:activeclients:MIN:'Min: '\%5.0lf%s COMMENT:" \n"
GPRINT:activeclients:MAX:"Max:\:%5.0lf%s\n"

How I can place values, in this form, like this:
http://d.pr/Okcr+
Number under numbers, caption under captions..
Answer: You need to make sure you set the relevant spaces correctly in order to make everything line up correctly.  For example:
COMMENT:'     Clients  Connected\n'
COMMENT:'LAST '
GPRINT:clients:LAST:'%8.0lf%s '
GPRINT:activeclients:LAST:'%8.0lf%s\n'
COMMENT:'MIN  '
GPRINT:clients:MIN:'%8.0lf%s '
GPRINT:activeclients:MIN:'%8.0lf%s\n'
COMMENT:'MAX  '
GPRINT:clients:MAX:'%8.0lf%s '
GPRINT:activeclients:MAX:'%8.0lf%s\n'


Best practice: how to handle data records with changing "schema" / "columns"

Question: This is a best-practice question.
Our set up is a hadoop cluster, storing (log) data in hdfs. We get the data in csv format, one file every day. Running MR jobs in hadoop on these files is fine, as long as the "schema" of the file, especially the number of columns, does not change.
However, we are facing the problem that the log records we want to analyze eventually change, in the sense that columns might be added or removed. I was wondering if some of you would be willing to share your best-practices for these type of situations. The best way we can think of at the moment is to store the data not as csv, but in json format. However, this will increase (at least double) the required storage space. We also came along Apache Avro and Apache Parquet, and just started looking into this.
Any ideas and comments on this issue are more than welcome.
Comment: Just found this mailing list entry, discussing advantages of json vs. avro: [http://mail-archives.apache.org/mod_mbox/avro-user/201208.mbox/thread](http://mail-archives.apache.org/mod_mbox/avro-user/201208.mbox/thread)
Comment: At the end we decided to go for Avro. Avro is able to handle schema changes, and has connectors to all major languages, incl. jaql (which is part of IBM's BigInsights).
Answer: use Thrift and use elephant bird (twitter lib )  for using the related file input/output format .


Pegar ID do elemento clicado

Question: Tenho uma DIV e essa DIV tem alguns checkbox com ID e eu quero saber como pego o id do checkbox clicado.

    <input type="checkbox" id="0" value="false" name="ckb0">
    TESTE

    <input type="checkbox" id="1" value="false" name="ckb1">
    Posso escolher essa

    <input type="checkbox" id="2" value="false" name="ckb2">
    Essa também

    <input type="checkbox" id="3" value="false" name="ckb3">
    E essa

    <input type="checkbox" id="4" value="false" name="ckb4">
    este tambem

Answer: Crie um event handler e use $(this).attr("id"). O código abaixo irá pegar os cliques em todos os checkbox dentro da div #resposta:

$(document).ready(function(){
   $("#resposta").find("input[type='checkbox']").click(function(){
      console.log($(this).attr("id"));
   });
});

      <input type="checkbox" id="0" value="false" name="ckb0">
      TESTE

      <input type="checkbox" id="1" value="false" name="ckb1">
      Posso escolher essa

      <input type="checkbox" id="2" value="false" name="ckb2">
      Essa também

      <input type="checkbox" id="3" value="false" name="ckb3">
      E essa

      <input type="checkbox" id="4" value="false" name="ckb4">
      este tambem

Documentação jQuery:
Função .attr()
Função .find()
Comment: Perfeito! Obrigado.

Pegar ID do elemento clicado


Not sure how to carry out this two-way repeated measures ANOVA

Question: I have a scenario where both factors are within-subject. One is time (5 levels) and one is condition (2 levels). The time point of 0 was only measured once in each subject, and is a 'resting value' that is to be used as the time point of 0 for both conditions. I cannot find a way of dealing with this accept for entering in the value twice into the data set as t=0 for both conditions. I feel like this is not a valid approach but I don't know what else there is to do. I could run two separate one way ANOVAs but then I couldn't see an interaction between the two conditions. I use SPSS.
UPDATE: Thanks for the responses. The nature of the methodology/study design includes assuming rest is the same for both conditions (to reduce tissue sampling required). Therefore, entering the rest measures twice in the two-way ANOVA is valid and is the nature of the methodology in this case, albeit a limitation.
Comment: Is subtracting the values at t=0 from your remaining conditions (4 time points x 2 conditions) an option? Then, the difference from rest could be the variables you put into your ANOVA.
Comment: @Amyunimus I really like your comment. If you have a binary repeated-measures variable, the ANOVA does nothing else but taking the difference so it clearly is a good idea. Why not post it as an answer? I would upvote it.
Answer: Try subtracting the "resting value" at t=0 from your remaining time points (4x) in both conditions and put the difference from rest as variables into the repeated-measures ANOVA.
Comment: This may be an option but I have never seen this method used in my field. Often a fold-change from rest is used but I don't agree with not showing the variability which occurs at rest.
A problem I see with this is not being able to show significance in the change in the measure from rest as there would be no resting measure, and then again not being able to show the variance at rest.
Answer: This is a bit of a rough fix, but you might consider recoding your categories into a 9 level variable. So, instead of having two variables, (0,1,2,3,4) and (A,B), you would have one variable (0,1A,1B,2A,2B,3A,3B,4A,4B). Then you could try and run a one-way ANOVA using that variable as your grouping variable.
Again, this isn't exactly the best way to handle it, as this won't let you test for either main or interaction effects directly, but you can use post-hoc tests and visual inspection to get a good impression of what is going on in your data.
Comment: Thanks for the suggestion. I ran this analysis and saw similar effects as the original analysis (an overall effect - which in running the one-way can be indirectly understood as an interaction).
Answer: I like Amyunimus's answer.  But as an alternative, how about fitting the model for t = 1 through 4 and both conditions, then conducting post-hoc tests for each of the fitted values against the value of the response variable at t=0?


Error: Expected all tensors to be on the same device, While all are on same device

Question: I don't know why I am getting this error, while my all tensors are at the same device "cuda".
Here is code
def train(self):
    self.model.train()
    self.optimizer.zero_grad()  # Clear gradients.
    print("X is",self.XT.get_device())
    print("edge_index_ is", self.edge_index_.get_device())
    print("R_train_mask is", self.R_train_mask.get_device())

    print("datas.y is", self.datas.y.get_device())
    print("edge_index_ is", self.edge_index_.get_device())
    out = self.model(self.XT, self.edge_index_)  # Perform a single forward pass.
    loss = self.criterion(out[self.R_train_mask],
                          self.datas.y[self.R_train_mask])

Here is error
X is 0
edge_index_ is 0
R_train_mask is 0
datas.y is 0
edge_index_ is 0
Traceback (most recent call last):
  File "/home/adnan/GNNPaperCodes/PyGCL/examples/GCAPaperAd.py", line 126, in <module>
    main()
  File "/home/adnan/GNNPaperCodes/PyGCL/examples/GCAPaperAd.py", line 91, in main
    topfeatures=tf.TopFeaturesFind()
  File "/home/adnan/GNNPaperCodes/PyGCL/examples/PatchFinder.py", line 64, in TopFeaturesFind
    loss = self.train()
  File "/home/adnan/GNNPaperCodes/PyGCL/examples/PatchFinder.py", line 154, in train
    out = self.model(self.XT, self.edge_index_)  # Perform a single forward pass.
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/adnan/GNNPaperCodes/PyGCL/examples/PatchFinder.py", line 211, in forward
    x = self.conv1(x, edge_index_)
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch_geometric/nn/conv/gcn_conv.py", line 181, in forward
    x = self.lin(x)
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch_geometric/nn/dense/linear.py", line 103, in forward
    return F.linear(x, self.weight, self.bias)
  File "/home/adnan/.conda/envs/forGit/lib/python3.9/site-packages/torch/nn/functional.py", line 1848, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_mm)

Obvioulsy, Everything is ok if i change device to 'cpu'
Answer: You forgot to transfer self.model to the device. Indeed, the model has weights that must be transferred to the correct device, since they interact with your input.
You can do it with self.model.to("cuda").
Comment: Thank you, it worked, I was going crazy. I really need to study fundamentals haha.


falling star css animation with before/after

Question: I am trying to make fallin star using css animation, so i created star with using pseudo element but i unable to set animation in pseudo element
 i{
     position: relative;
   width: 0;
   height: 0;
   border-left: 12px solid transparent;
   border-right: 12px solid transparent;
   border-bottom: 20px solid red;
}
 i:after {
  content:"";
   border-left: 12px solid transparent;
   border-right: 12px solid transparent;
   border-top: 20px solid red;
   position: absolute;
   width: 0;
   height: 0;
   margin: 28px 0 0 -12px;
}

http://jsfiddle.net/m7bpp9za/
see the above fiddle which is falling "after pseudo element" but i need both i & i:after to get falling star
any suggestion would be great,
 Thanks
Comment: just a suggestion make i:after {margin: 23px 0 0 -12px;} for better star
Answer: You were nearly there, you just need to change this line :
i, i:after

to
i

DEMO
Explanation :
As the i:after pseudo element is positioned absolutely in the i element, it moves relatively to it so it "follows" it during the animation. Therefore it doesn't need to be animated.
Comment: Thanks for info but the fiddle is not falling
Comment: @parthiban as you set an animation delay you need to wait 2 seconds for the animation to start.
Comment: yes i wait more than that but still i couldn't find exactly what i miss. do you see anything falling on that demo. Or i mess up with position? Even i tried to change the position to absolute, but no help :(
Comment: @parthiban yes, the demo I linked is working for me, the stars are falling and rotating. try this one http://jsfiddle.net/m7bpp9za/9/
Comment: i tried with firefox in ubuntu is not working but for chrome is working any idea why it happen?
Comment: @parthiban this is because you didn't include the `-moz-` vendor prefixes in the CSS. Please accept answer if it help.


How do I save iptables config?

Question: I have just installed Webmin on a Ubuntu server. According to the docs you need to open up port 10000 (which is what Webmin runs on), with this:
iptables -A INPUT -p tcp -m tcp --dport 10000 -j ACCEPT

It then says you need to apply the firewall configuration with:
/etc/init.d/iptables restart

However, this command doesn't work for me. Is there a different command that saves the changes on Ubuntu?
Answer: there is a package that contains a system service to restore
sudo apt-get install iptables-persistent

put rules in /etc/iptables/rules.
one way to do this is:
sudo bash -c "iptables-save > /etc/iptables/rules"

Answer: Your iptables rule is active once you execute it. Issuing a restart merely restores your iptables config to however it was last saved. There is no need to do the restart.
Comment: and it will go away on the next reboot
Answer: On many Linux distributions, you can save changes made to iptables config by:
service iptables save

or alternately
/etc/init.d/iptables save

But this doesn't activate changes, it merely makes them permanent.  The moment you executed the iptables command, the change was active.  This command makes changes in real time.
Comment: there is no such service in the current version of Ubuntu 10.04
Answer: iptables-save
iptables-restore

will save rules in a plain text file, and usually restored on system sturup by the init.d script.  see iptables.conf e iptables man for details
Comment: these commands dont exist.
Comment: Simply forgotten a dash. Fixed

How do I save iptables config?


If strings are nullable by default, why do i get Unable to cast object of type 'System.DBNull' to type 'System.String'.?

Question: I have seen similar threads like this where the solution is a tertiary if.
My question is, why even get such an error if Strings are nullable?
I am reading a value of a text column in access using ado.net.
Whenever there is a row with an empty text column i get that error.
Culprit:
while (dr.Read())
{
      UserList.Add(new UserInfo()
      {
        .
          DestributionGroup = (string)dr["Destribution Group"]
        .
      }
}
class UserInfo
{
.
    public string DestributionGroup;
.
}

Edit:
So in other words I have to convert all of my strings that I am reading from the DB into a line similar to this?
return (accountNumber == DBNull.Value) ? string.Empty : accountNumber.ToString ()

No other way around it?
Comment: Because `null != System.DBNull`
Answer: Because null != DBNull.Value.
But you can check if it the value in the DataReader is null with the IsDbNull method:
DestributionGroup = dr.IsDbNull("Destribution Group") ? "" : dr.GetString("Destribution Group");

Answer: You can use this if you don't mind DestributionGroup being set to null when dr["Destribution Group"] is DBNull.Value :
DestributionGroup = dr["Destribution Group"] as string;


Random Forest signal lagged?

Question: I would like to expand the original question posted here First steps learning to predict financial timeseries using machine learning
with another question of my own:
Let's fit an overly basic RF algorithm in R following the example:
getSymbols("GOOG")

fit <- RF(lag(GOOG.Close,1), GOOG.Close, data=GOOG[1:(NROW(GOOG)-20)])

prediction <- predict(fit,GOOG[(NROW(GOOG)-19):NROW(GOOG)])

sig = ifelse(prediction >0, 1,-1)

ret = diff(log(GOOG$GOOG.Close))

pnl = ret * sig

I'm not lagging the signal sig because I already lagged the explanatory variable.
Is that correct or I should lag the signal again such as:
sig1 = lag(sig,1)

pnl = ret * sig1

What is the correct procedure ?
Thank you!
Answer: For these problems it helps to think about how you plan to score your model in its real application.
For predicting stock price, your input will be all the close prices up to and including yesterday's (or for your example: just yesterday's). You will try to predict today. Your measure of success will be the prediction made today versus today's true close price.
Fitting today's price as a function of yesterday's price means lag.
Comparing today's predicted price to today's actual price means no lag.
Comment: Hi Dex Groves, when you say "Your measure of success will be the prediction made today versus today's true close price.", you mean : your measure of success will be the prediction made yesterday for today's price versus today's true close price ? I'm stressing this out because I'm making my prediction yesterday  for today's price.
Comment: You know the prior day's close price when you make your prediction right? Then it's the same logic: you make a decision with the most up to date record, one record into the future. If you have to predict two records out, you should lag your variable by two.

Random Forest signal lagged?


How can I integrate $\ln \left( x+\sqrt{1+x^2} \right) $?

Question:
How can I integrate $$\ln \left( x+\sqrt{1+x^2} \right) $$
  The problem is from IA Maron Calculus, Problem 4.3.17. The problem is under the heading 'Integration by parts'. I have tried it, but it just leads to 2 unsolvable integrals
Comment: is it $$\int \ln(1+\sqrt{1+x^4})dx$$?
Comment: According to the version of the textbook on [Internet Archive](https://archive.org/details/ProblemsInCalculusOfOneVariableI.A.Maron), the integral should be $$ \int \ln \left(x+\sqrt{1+x^2} \right) \, dx = \int \operatorname{arsinh}(x) \, dx$$
Comment: @Dr.SonnhardGraubner $\ln(x+\dots)$...
Comment: A rapid look at your integral and I can state that there shall necessarily be an error in your textbook. That integration is not trivial at all, and surely by parts method does not make things easier.
Answer: Assuming the correct question is the edited one. With integration by parts:
$$\int \ln\left(x+\sqrt{1+x^{2}}\right) \,\mbox{d}x
= x\ln\left(x+\sqrt{1+x^{2}}\right)-\color{blue}{\int \frac{x}{\sqrt{1+x^2}}\,\mbox{d}x}$$
Since:
$$\left(\ln\left(x+\sqrt{1+x^{2}}\right) \right)' =\left( \mbox{arcsinh}\,x \right)' = \frac{1}{\sqrt{1+x^2}}$$
For the blue integral, you can substitute $u=1+x^2$ and end up with:
$$\int \ln\left(x+\sqrt{1+x^{2}}\right) \,\mbox{d}x
= x\ln\left(x+\sqrt{1+x^{2}}\right)-\color{blue}{\sqrt{1+x^2}}+C$$
Answer: As I wrote in the comment, the integral is not trivial. There has to be some error in the command.
But for the sake of completeness, here is the solution.
$$\log \left(x+\sqrt{x^4+1}\right) x-2 x+\frac{1}{2} \left(\sinh ^{-1}\left(x^2\right)-\frac{i \left(-3 i+\sqrt{3}\right) \tan ^{-1}\left(\frac{\left(3+i \sqrt{3}\right) x^8-2 i \sqrt{6-6 i \sqrt{3}} \sqrt{x^4+1} x^6+2 \sqrt{3} i \left(\sqrt{2-2 i \sqrt{3}} \sqrt{x^4+1}+2\right) x^4-2 i \sqrt{6-6 i \sqrt{3}} \sqrt{x^4+1} x^2+\sqrt{3} i-3}{\left(-7 i+\sqrt{3}\right) x^8+2 \left(i+\sqrt{3}\right) x^6+2 \left(-3 i+\sqrt{3}\right) x^4+2 \left(i+\sqrt{3}\right) x^2-i+\sqrt{3}}\right)}{\sqrt{6-6 i \sqrt{3}}}+\frac{\left(3-i \sqrt{3}\right) \tan ^{-1}\left(\frac{\left(3 i+\sqrt{3}\right) i x^8-2 i \sqrt{6+6 i \sqrt{3}} \sqrt{x^4+1} x^6+2 \sqrt{3} i \left(\sqrt{2+2 i \sqrt{3}} \sqrt{x^4+1}+2\right) x^4-2 i \sqrt{6+6 i \sqrt{3}} \sqrt{x^4+1} x^2+\sqrt{3} i+3}{\left(7 i+\sqrt{3}\right) x^8+2 \left(-i+\sqrt{3}\right) x^6+2 \left(3 i+\sqrt{3}\right) x^4+2 \left(-i+\sqrt{3}\right) x^2+i+\sqrt{3}}\right)}{\sqrt{6+6 i \sqrt{3}}}+\frac{\left(3 i+\sqrt{3}\right) \log \left(16 \left(x^4-x^2+1\right)^2\right)}{2 \sqrt{6+6 i \sqrt{3}}}+\frac{\left(-3 i+\sqrt{3}\right) \log \left(16 \left(x^4-x^2+1\right)^2\right)}{2 \sqrt{6-6 i \sqrt{3}}}-\frac{\left(3 i+\sqrt{3}\right) \log \left(\left(x^4-x^2+1\right) \left(\left(-3 i+\sqrt{3}\right) x^4-i \left(\sqrt{2+2 i \sqrt{3}} \sqrt{x^4+1}+2\right) x^2-3 i+\sqrt{3}-2 i \sqrt{2+2 i \sqrt{3}} \sqrt{x^4+1}\right)\right)}{2 \sqrt{6+6 i \sqrt{3}}}-\frac{\left(-3 i+\sqrt{3}\right) \log \left(\left(x^4-x^2+1\right) \left(\left(3 i+\sqrt{3}\right) x^4+i \left(\sqrt{2-2 i \sqrt{3}} \sqrt{x^4+1}+2\right) x^2+3 i+\sqrt{3}+2 \sqrt{2-2 i \sqrt{3}} i \sqrt{x^4+1}\right)\right)}{2 \sqrt{6-6 i \sqrt{3}}}\right)-\frac{\left(-3 i+\sqrt{3}\right) \tan ^{-1}\left(\frac{1}{2} \left(1-i \sqrt{3}\right) x\right)}{\sqrt{-6+6 i \sqrt{3}}}-\frac{\left(3 i+\sqrt{3}\right) \tan ^{-1}\left(\frac{1}{2} \left(1+i \sqrt{3}\right) x\right)}{\sqrt{-6-6 i \sqrt{3}}}$$
Comment: XD this definitely shows that the OP's integral is a bit off
Answer: Notice the identity
$$\sqrt{\sinh^2(t)+1}=\cosh t$$ which hints the change of variable $x=\sinh t$ and $dx=\cosh t\,dt$.
The integral becomes
$$\int\log(\sinh t+\cosh t)\cosh t\,dt=\int t\cosh t\,dt.$$
Then by parts
$$t\,\sinh t-\int\sinh t\,dt=t\,\sinh t-\cosh t=x\text{ arsinh }x-\sqrt{x^2+1}.$$


How to determine the inputs for the MA terms when forecasting with an ARIMA-GARCH model?

Question: I am having difficulty determining how to forecast values for an ARIMA-GARCH model manually (by hand).
I understand that for an ARIMA model, the inputs for the MA terms are the residuals (i.e. the fitted value - true value). However, since in an ARIMA - GARCH, the GARCH model is fit on the residuals produced by the ARIMA model, how would I now interpret the MA terms? Are the inputs for the MA terms now the residuals produced by the GARCH model? Or is it still the residuals from the ARIMA model?
To give a hypothetical example, say the ARIMA model is a (0,0,1), and the first two residuals (εt and εt-1) from the ARIMA model are 0.62 and 0.52 respectively. A GARCH model is then fit on the residuals and the first two residuals of the GARCH model are 0.23 and 0.25. When I am predicting the Yt+1 of the ARIMA model, would it be 0.62+θ0.52 or 0.23+θ0.25?
If it is the former (i.e. 0.62+θ0.52), would that imply that the GARCH model has no effect on the first forecast of the ARIMA(0,0,1)?
Answer: Consider an ARMA($p,q$)-GARCH($r,s$) model
\begin{aligned}
r_t &= \mu_t + u_t, \\
\mu_t &= \varphi_1 r_{t-1}+\dots+\varphi_p r_{t-p} + \theta_1 u_{t-1}+\dots+\theta_q u_{t-q}, \\
u_t &= \sigma_t \varepsilon_t, \\
\sigma_t^2 &= \omega + \alpha_1 u_{t-1}^2 +\dots + \alpha_s u_{t-s}^2 + \beta_1 \sigma_{t-1}^2 + \dots + \beta_r \sigma_{t-r}^2, \\
\varepsilon_t &\sim i.i.d.(0,1).
\end{aligned}
Suppose you have estimated the model. The conditional mean equation is the ARMA equation for $\mu_t$: $\mathbb{E}(r_t|I_{t-1})=\mu_t$ where $I_t$ is the information up to and including time $t$. If the estimated conditional mean suits you as a point forecast (this would be the case under square loss, for example), you can ignore the other equations and only use the ARMA equation. Where the other equations play a role is the estimation stage, not the forecasting stage.
ARIMA-GARCH is mostly the same, one just has to take a cumulative sum of the corresponding first-difference series which is modelled by an ARMA-GARCH model.

Now to address your questions specifically,

When I am predicting the Yt+1 of the ARIMA model, would it be 0.62+θ0.52 or 0.23+θ0.25?

Neither of them. It would be $\hat\theta_1\cdot 0.62$.

If it is the former (i.e. 0.62+θ0.52), would that imply that the GARCH model has no effect on the first forecast of the ARIMA(0,0,1)?

If your point forecast is the conditional mean, the conditional variance equation only plays a role in the estimation stage, not the forecasting stage. But if your point forecast is something else than the conditional mean, the conditional variance equation and the distributional assumption plays a role also in the forecasting stage.


Problemas com "or" em C++

Question: Preciso fazer um algoritmo que recebe 3 números diferentes, e, se receber números repetidos, informa uma mensagem de erro.
Meu programa está todo funcionando corretamente, mas quando vou colocar a tal linha para avisar do erro tento usar Or e não dá certo. Segue o que estou digitando
if (z==x) or (y==z) or (x==y);
    cout << "\nErro. Os valores devem ser diferentes. Tente novamente: \n" << endl;

Mesmo que tenha um modo mais fácil do que or, vocês poderiam, por gentileza me dizer como funciona a inserção do or em C++ ?
Segundo o Code::Blocks, o programa espera uma expressão primária antes do or.
Comment: na verdade, seu problema não é exatamente com o operador OR; e sim, como você escreveu o if. Se fizesse if (z==x or y == z or x == y) cout.... nao iria ter problemas.
Comment: Você não precisa fazer isto agora ou se não achar que deve mas se a resposta lhe atendeu bem você pode aceitá-la como correto. E pode votar em todos os posts do site que ajudaram de alguma forma. Veja o [tour].
Answer: A sintaxe preferencial do or em C++ é || apesar de serem sinônimos.
Além disto tinha um ; que estava encerrando o if e não executando da forma como parece ser seu desejo. A mensagem não estava fazendo parte do bloco condicional.
Também há problema com os parênteses. Um único par deve conter toda a condição. Você até poderia colocar parênteses extras nas sub-expressões mas eles são totalmente desnecessários.
Então sua linha ficaria:
if (z==x || y==z || x==y)
    cout << "\nErro. Os valores devem ser diferentes. Tente novamente: \n" << endl;

Coloquei no GitHub para referência futura.
Chaves poderiam ser recomendadas para encapsular o bloco mesmo tendo apenas uma linha. Isto evita alguns erros por descuido. Mesmo que opte por não usar chaves seria interessante colocar o bloco que deverá ser executado condicionalmente em uma nova linha com indentação como demostrado acima.
Comment: Nota: `or` e `||` são sinônimos.
Comment: +1 por apresentar sobre as chaves
Answer: O operador lógico OR no C++ é o ||.
if(z == x || y == z || x == y) {
  cout << "\nErro. Os valores devem ser diferentes. Tente novamente: \n" << endl;
}

Você também está adicionando um ; depois do if. Isto vai fazer com que esta linha seja encerrada ali e o cout irá sempre imprimir a mensagem, independente do resultado da avaliação do if.
Comment: `or` também funciona.
Answer: Se for usar C++ 11/14, você poderá usar o or( || ) e o and( && ) normalmente se quiser.
Comment: Dá uma olhada às outras respostas que já há aqui. Se achares que vale a pena submeter mais uma resposta ainda explica por favor melhor então. Até já. (ah, e bemvindo!)

Problemas com "or" em C++


Find the rational points on $1 + 18 x + 81 x^2 + 44 x^3 = y^2$ with Sage

Question: I'm trying to use Sage on-line,but I meet some trouble with the code of it.
I want to find the rational points on an ellipse curve,such as $$1 + 18 x + 81 x^2 + 44 x^3 = y^2,\tag1$$
I know that $(x,y)=(0,1)(1,\pm12)(-\frac{1}{11},0)$ are on the curve,but I don't know how to find more,so I try to solve it with the math-software Sage.
What should I input?Thanks in advance!
Edit: Multiply by $44^2$,we can difine the ellipse curve by this way,
E = EllipticCurve([0,81,0,44*18,44^2])
Then how to find the rational points on it?
Comment: http://math.stackexchange.com/questions/203484/how-to-compute-rational-or-integer-points-on-elliptic-curves  has links to some examples .
Comment: Since this question has been linked to elliptic curves, should it be better to tag it as such? Regards.
Comment: You could always try asking [here](http://ask.sagemath.org/questions/).
Comment: BTW, `E.rational_points(bound=1000)` took several *days* to return the output `[(-4 : 0 : 1), (0 : -44 : 1), (0 : 1 : 0), (0 : 44 : 1), (44 : -528 : 1), (44 : 528 : 1)]` -- so knowing the mathematics in the answers below certainly helps. :-)
Answer: By the Mordell-Weil theorem, the group of rational points $E(\mathbb{Q})$ is finitely generated and therefore isomorphic to a group of the form $T\oplus \mathbb{Z}^R$, where $T$ is finite abelian formed by those points of finite order, and $R\geq 0$ is the rank of the elliptic curve, and it counts how many $\mathbb{Z}$-linearly independent points of infinite order there are.
With Sage, define your curve by

E=EllipticCurve([0,81,0,44*18,44^2]);

Now you can ask Sage to calculate the rank $R$ with

E.rank();

and the answer is $0$. So there are no points of infinite order, only finite order points. Let us calculate the torsion subgroup as follows

T=E.torsion_subgroup();

if you type ``T;'' and evaluate, Sage will tell you that $T\cong \mathbb{Z}/6\mathbb{Z}$. Finally, if you type

T.0;

Sage returns a generator of the torsion subgroup, namely $P=(44,528)$, which is a point of exact order $6$. Hence, $E(\mathbb{Q})=\langle P \rangle$ or
$$E(\mathbb{Q})=\langle P \rangle = \{ \mathcal{O}, (44,528),(0,44), (-4,0), (0,-44), (44,-528)\}.$$
Comment: Thanks!May I ask what should I do when E.rank()$\ge$1 ?For example,E = EllipticCurve([0,121,0,22*52,52^2]);then E.rank()=1,now how can I find the point which rank is infinte?
Comment: Just like Jesper Petersen explains in another answer.
Answer: In case a curve has positive rank, you'd like to know its generators by E.gens().
In the case
E = EllipticCurve([0,121,0,22*52,52^2]);

the rank is 1
sage: E.rank()
1

and its generator is
sage: E.gens()
[(-104 : 260 : 1)]

In this case you can construct infinitely many points on E, by adding a torsion point and some multiple of the generator, for instance
sage: T = E(0,52)
sage: P = E(-104,260)
sage: T + 2*P
(480/49 : 55796/343 : 1)
sage: T + 3*P
(-12662312/485809 : -65467251044/338608873 : 1)
sage: T - 5*P
(-70265016926482573/11385999920450401 : 5045808098009295785013222/1214945410097644049235601 : 1)

and so on.


Hover on Image not working Javascript / JQuery

Question: I am working on website which uses Jquery and CSS3.
I have image and on hover i want to scale image. Hover works with CSS but i want it to via JQuery following code i have tried till now. Please help.
HTML
<!DOCTYPE html>

      <link rel="stylesheet" type="text/css" href="css/app.css" />

CSS
.scaleout {
    transform: scale(1, 0.80);
   -ms-transform: scale(1, 0.80);
   -webkit-transform: scale(1, 0.80);
}

I also tried hover() method instead of mouseenter and mouseleave in above script code
$(document).load(function(){
                $('#thumbnail').hover(function(){
                    $(this).toggleClass('scaleout');
                },
                function(){
                    $(this).removeClass('scaleout');
                });
            });

Please help why hover not working.
Comment: `,'img');` to `);` in first method and not `$(document).load(` it's `$(document).ready(` in second method
Comment: Put the img into a div and declare a transition to its class.
Answer: I've created a fiddle that works using code pretty similar to your second solution (use document.ready and no need for the removeClass function). https://jsfiddle.net/L5bj38me/

Update OP's issue is due to Angular see - Run jQuery code after AngularJS completes rendering HTML
Comment: i tried same code but dont know why its not working when i put i css using :hover selector it works. My images gets rendered using angular js framework
Comment: @virendrao Make sure you don't have multiple images all with the same ID. Does my Fiddle work for you?
Comment: it works but when use in my script doesnt work though works through css
Comment: @virendrao your issue is due to Angular, see this post for a fix http://stackoverflow.com/questions/16935766/run-jquery-code-after-angularjs-completes-rendering-html
Answer: fiddle
    <!DOCTYPE html>

              <link rel="stylesheet" type="text/css" href="css/app.css" />

Answer: Remove 'img' from event handler,
Its work for me

Comment: i tried same code but dont know why its not working when i put i css using :hover selector it works. My images gets rendered using angular js framework
Answer: There are many ways to do it, one solution is -
$(document).ready(function(){
    $('#thumbnail').on('mouseenter', function (e) {
        $(this).toggleClass('scaleout');
    });
});

Here's another one -
 $("#thumbnail").mouseenter(function (e) {
        $(this).toggleClass('scaleout');
    });

Few more -
$( "#thumbnail" )
  .mouseover(function() {
    $(this).toggleClass('scaleout');
  })
  .mouseout(function() {
    $(this).toggleClass('scaleout');
  });

Same output but with different approach -
$( "#thumbnail" )
  .mouseenter(function() {
     $(this).toggleClass('scaleout');
  })
  .mouseleave(function() {
    $(this).toggleClass('scaleout');
  });


Objective-C const NSString * vs NSString * const

Question: I'm trying to a NSString constant in my .h file to be defined in my .m. I understand that
extern NSString * const variableName; in the .h and
NSString * const variableName = @"variableValue"; is the way to do this.  Examining C tutorials I see that const is supposed to go before variable definitions.  My question is why is it not declared as extern const NSString * variableName; in the .h and const NSString * variableName = @"variableValue"; in the .m.  I know this doesn't work because I encounter compiler warnings which say 'Passing argument 1 of methodName: discards qualifiers from pointer target type'.  What does this mean?
Answer: It's not the same. The const modifier can be applied to the value, or the pointer to the value.

int * const

A constant pointer (not modifiable) to an integer (its value can be modified)

const int *

A modifiable pointer to a constant integer (its value can't be modified)

So you can imagine:
const int * const;

Comment: To expand on that, if you're not dealing with a pointer, `const int` and `int const` are exactly the same. `NSString`s are already immutable, so the const only needs to be applied to the pointer (which can be a little confusing since Objective-C objects are always accessed via pointers).
Comment: @Wevah: "NSStrings are already immutable" but an NSString pointer can still point to an NSMutableString
Comment: @Macmade why didn't you explain this in terms of NSString since the syntax is different anyways with the pointer?
Comment: To expand on that, `const int *` and `int const *` are the same, but `int * const` is different.
Answer: Constant pointer is NOT a pointer to constant. Constant pointer means the pointer is constant. E.g. constant pointer int * const ptr2; indicates that ptr2 is a pointer which is constant. This means that ptr2 cannot be made to point to another integer. However, the integer pointed by ptr2 can be changed.
Whereas a pointer to constant const int * ptr1; indicates that ptr1 is a pointer that points to a constant integer. The integer is constant and cannot be changed. However, the pointer ptr1 can be made to point to some other integer.