Are parent scales always major?

Question: As a classical musician, the notion of "parent scales" isn't something I encountered until I started visiting this website.
From context, I understand the concept just fine. But it seems that answers exclusively use major scales as the parent scale. Even an Internet search of "parent scale" results in several hits that specify "parent major scale."
Does this mean that parent scales are always major? Or are there perhaps different contexts used for the term "parent scale"? (Online usage of the term is...ambiguous at best.) I'm wondering if parent scales for diatonic modes are always major, but then parent scales for chords (or modes other than the diatonic church modes) are allowed to be something else.
Comment: That said, we call the one main scale (in minor) that begets its modes a parent scale, don't we?
Comment: I think 'parent scale' is the datum point - the start point from which everything emanates. We consider a minor scale, for example, to have 'changed notes' from its parallel major. (Rather than vice versa). But major intervals are so called not because they're more important, but because they're bigger. Although the chords for minor keys come from those minor scales... Good question!+1.
Comment: @Tim Exactly. I can think of several instances where a major scale is unable to generate a scale or chord, which would suggest that some parent scales are not major. But I've only ever seen discussions where the parent scale is major.
Comment: I've always heard it in terms of the modes. E.g. the parent scale of D Dorian is C major, or the parent scale of C Phrygian dominant is F harmonic minor, that kind of thing.
Answer: It might depend on context and it might apply more to one instrument than another.  So you could get a variety of answers.  In short given a 12TET tuning and all the other caveats we have 7 diatonic modes, Ionian, Dorian, Phrygian, Lydian, Mixolydian, Aeloian, Locrian.  Each being distinct in character each is contained in the Ionian or Major scale.  So, playing in D Dorian puts you in the key of C major for example.  And F Aeolian is in the key of Ab Major.  In guitar blogs and online lessons we frequently talk about the parent major scale as a way of knowing where the key signature tonal centers are regardless of what position you are in on the neck.  Why major and not parent Lydian?  I have no answer to that except historical context.  We have music written in Major and Minor keys, where the minor compositions would typically use melodic or harmonic minor.  Indeed we can, and should, and do use the concept of a parent melidic minor scale when discussing the melodic minor modes used in Jazz.  It is rare to talk about the parent Dorian but I think it makes sense.  Take for example So What by Miles Davis.  The tune is written in D Dorian, the Key of C.  But no one refers to it as being in C.  It is so strongly modal that saying it's in C (major) throws everything out of context.  And I can tell you from playing it 1000s of times over the decades in my mind I default to "Parent Dorian Patterns" when moving through the change.  I might be in the Major scale but I am hearing and feeling Dorian, otherwise it wouldn't come together quite right.  I think that what we call Parent Scales depends on the application.
In the context described above I see the concept as picking a frame of reference for the ear and in my case (as a guitarist) the hand.
The fact that any mode in 12TET can be embedded in the Chromatic scale doesn't really make the Chromatic a parent in the above context.  It is ambiguous in that there is no pattern, every note is a viable starting note and it always has a Chromatic character.
Comment: Surely D Dorian is *another* minor - if only 'cos it has m3 from the root.That's about all the other 'minors' have in common.D Dorian and D mel. min. both also have M6.Sure enough, we don't think 'in key C' even though the notes are from the *parent key* (Dare I use the term..?). We label 'C' as C maj., probably the default flavour - bit like '7th' is always dominant 7th in reality. So What wouldn't be considered to be in key C, but may well be construed by some to be in 'key Dm'. In fact, I've seen it written with no key sig.(C or D Dorian - correct), Dm (one flat) and even D maj. two sharps!
Comment: "_playing in D Dorian puts you in the key of C major_" -- I would think that a piece focused around D Dorian should be in D minor. But I don't think of keys as related to scales so much as tonal centers; maybe key-feeling would be a better phrase for what I mean.
Comment: Absolutely not.  D Dorian is NOT D minor, just because the differ by one note.
Comment: D natural minor, D harmonic minor, and D melodic minor are all D minor, yet differ in detail.
Comment: But those details are more meaningful.  Harmonic shifts the 7th degree so that you have a leading tone.  Melodic shifts the 6th up so that the scale is comprised of steps.  None of this applied to comparing Dorian and Phygian to minor.
Comment: @exnihilo, another thing I think you are missing is that D dorian is literally in the key of C maj, They are the exact same set of note, just played in a different order.  This is true in any key, play X Maj from 2 to 9 and you have Dorian.
Comment: The fact that D dorian is a minor scale doesn't change the fact that you are in fact in the key of C maj.
Comment: The way my mind works says that Parent is the source and really is only partially responsible for  the character of the newly derived scale, mode, or what have you. I would expect the formula involved would be equally responsible for that character.
Answer: I have only seen "parent" scale and chords in relation to the jazz "chord scale system." My impression is the standard major and minor scales presented in textbooks (major, natural minor, harmonic minor, and melodic minor ascending) are the common "parent scales." Sometimes the term "parent" is used, sometimes "nth mode of a scale" is the wording used. Either way the concept is the same. Probably no one will use natural minor as a parent scale, because it's just a mode of the major scale.
Some minor mode examples are the altered scale which is the seventh mode of the parent melodic minor scale, or the Lydian dominant scale which is the fourth mode of the parent melodic minor scale. Harmonic minor can be a parent, like the Freygish scale is the fifth mode of harmonic minor. Charts abound on the internet of the all modes of the parent harmonic and melodic minor scales.

While I read "parent scale" a lot, I've not seen a common term like "derived scale" or "synthetic scale" for all the "scales" you get from a parent.
Some scales derived from parents are culturally significant, like the Freygish scale. Others like the super Locrian double flat seventh are not.
The parent scale idea seems to cause a lot of confusion about mode, key, and key signature. Also, I've never seen any of these sources make a distinction between a mere collection of tones - like the modes of harmonic minor - and the modes of real musical cultures - like the modes of Jewish music. The same goes for the modes of major versus the Medieval modes. The former suggests you can substitute any number of (sort of) similar scales over certain chord types, the later are cultural expressions and tonalities for entire compositions.
Regardless of my misgivings, this stuff is all over the internet.
Answer: No, parent scales are not always major.
First, what is a parent scale, actually?
I think the usage of the term "parent (major) scale" is mainly due to how we have historically been taught the diatonic scales - I'd wager that most people start by learning the Ionian (major) scale first, next it's usually Aeolian (minor), then the rest.
To remember how to construct the rest of the diatonic scales (in 12TET), we learn that these are derived from a specific order of half-steps and their cyclic permutations; meaning that if we start from the major scale: [2, 2, 1, 2, 2, 2, 1], we can derive the dorian scale: [2, 1, 2, 2, 2, 1, 2] by moving the initial whole step to the end of the list.
This commonly taught way of constructing/deriving the scales means that we usually have to think of a parent scale first, which is — out of habit-- commonly the major scale.
What's the alternative?
We can enumerate the 12TET notes from 0 to 11 and arrive at the definition of a pitch class set:

Here, the "initial" whole step that gets "rotated" around is highlighted in green, and the flattened/sharpened notes (from the perspective of the major scale) are given in bold.
According to the works of Forte and Rahn, we now examine these pitch class sets, and stipulate that one of them is the "prime form", while the others (the modes) can be derived purely by rotations. The prime (or "normal") form is chosen to be the one which is most compact, in this case the Locrian mode.
In this context, therefore, we might call the Locrian mode the "parent scale" of the diatonic scales, which evidently is not a major scale. Note however, that due to the way we're taught, most musicians instinctively rather think of modes of the "major scale".
Further reading
Here, I'd strongly suggest to check out Ian Ring's great website, which lists all 2^11=2048 scales with a lot of analysis and clickable midi examples. Most importantly, he found that by representing a scale in binary form major scale: 101010110101 (picture this as if each note on a 12-tone keyboard being on or off, reading from right-to-left!)
we can directly derive a decimal representation major scale: 101010110101 == 2741 and if we do this for all the modes, we can find the "most compact set" is the one with the lowest decimal number (the locrian scale).

So what does this tell me about any non-major parent scales?
Using Ian's scale finder, it's extremely easy to find the modes of any scale; take for example the Ultralocrian scale with its modes: Locrian Natural 6, Major Augmented, Lydian Diminished, Phrygian Dominant, Aeolian Harmonic, Harmonic minor scale, all of which are clearly not the major (i.e. Ionian) scale.
Comment: Interesting, I've never encountered that binary notation before! But is your binary notation of major actually the binary notation of Lydian?
Comment: @Richard in binary notation you need to read from right to left, i.e. `000000000001` would mean only the *first note* (for example "C") is "turned on", `000000010101` then translates to e.g. "[C, D, E]", `000000110101` is "[C, D, E, F]" and so on. Lydian hence is  10101**10**10101, not 10101**01**10101. Each bit (counting from n=zero, right to left) is "worth" `2^n`, hence the Lydian decimal value is `2^5 = 32` "higher" than that of the Ionian mode.
Answer: The answer would be no. I would like to offer a particular example of parent scale, namely the melodic minor scale. The reason why I think it's specially relevant to this discussion is because its modes are a very important part of what's called chord-scale theory, which is of interest to jazz students, etc. In the context of C-S, these modes are a natural way to complement those of the major scale, as the two parent scales (major and melodic minor) share some characteristics of harmonic and melodic interest. So, in a similar way to when you think of dorian (2nd mode of major) as related to the "m7" chord, you think of altered (7th mode of melodic minor) as related to the "7alt" family of chords (that's a "7" chord with its 5th and 9th raised or lowered).
Answer: In western music, I  would think that the Chromatic scale might be considered the Parent Scale, although when we discuss chord formula and modes, we usually are discussing The Major Scale, but I think that has to do with narrowing the focus of the discussion in order to understand a particular topic more clearly. Perhaps the Chromatic Scale could be referred to as the GrandParent scale. I've only studied music theory on my own, but I've been doing it pretty regularly for a few years now and it seems clear to me that the usage of the word parent is mainly descriptive of the place of origin for whatever concept, idea, scale, mode, etc. mostly intended to help us understand structure.

Are parent scales always major?


JavaScript - checking for any lowercase letters in a string

Question: Consider a JavaScript method that needs to check whether a given string is in all uppercase letters. The input strings are people's names.
The current algorithm is to check for any lowercase letters.
var check1 = "Jack Spratt";
var check2 = "BARBARA FOO-BAR";
var check3 = "JASON D'WIDGET";

var isUpper1 = HasLowercaseCharacters(check1);
var isUpper2 = HasLowercaseCharacters(check2);
var isUpper3 = HasLowercaseCharacters(check3);

function HasLowercaseCharacters(string input)
{
    //pattern for finding whether any lowercase alpha characters exist
    var allLowercase;

    return allLowercase.test(input);
}

Is a regex the best way to go here?
What pattern would you use to determine whether a string has any lower case alpha characters?
Comment: Just a-z. Not à or à for example?
Comment: The title says "any lowercase", the introduction says "all uppercase". That's not the same. Decide for one and then adjust the function and variable names.
Answer:

function hasLowerCase(str) {
    return str.toUpperCase() != str;
}

console.log("HeLLO: ", hasLowerCase("HeLLO"));
console.log("HELLO: ", hasLowerCase("HELLO"));
Comment: @DipenduPaul No, the function would not fail. It would return `false`, which is correct, since an all-digit string does not have any lowercase characters.
Comment: it would fail, if string is all digits or non-letters
Comment: Why `!=` instead of the type-safe `!==`?
Comment: Nice. Nowadays it would be: `const hasLowerCaseLetter = (str) => str.toUpperCase() !== str;`
Answer: also:
function hasLowerCase(str) {
    return (/[a-z]/.test(str));
}

Comment: That's a very English-centric view of what a lowercase letter is. Is "à" not lower case? I'm mentioning this so long after this answer was accepted because this question has been referenced by [this newer question](http://stackoverflow.com/questions/3816905). The `!= toUpperCase` answer below is much more inclusive.
Comment: This won't work for lowercase, you need to check for `undefined` as well.
`
function hasLowerCase(str) {
  return str ? (/[a-z]/.test(str)) : false;
}`
Comment: Won't work for chars with diacritics. Fot that use `/\p{Ll}/u.test(value)`.
Comment: what if you want to find out like 2 lowercase char on an string ?
Answer: function hasLowerCase(str) {
    return str.toUpperCase() != str;
}

or
function hasLowerCase(str) {
    for(x=0;x<str.length;x++)
        if(str.charAt(x) >= 'a' && str.charAt(x) <= 'z')
            return true;
    return false;
}

Comment: The second one works with any language that does not have Diacritics or special chars, which is why the first one is prefered.
Answer: Another solution only match regex to a-z
function nameHere(str) {
    return str.match(/[a-z]/);
}

or
 function nameHere(str) {
        return /[a-z]/g.test(str);
    }

Comment: Welcome to StackOverflow. At StackOverflow, it is expected that you read everyone else's answers before posting your own answer. Both of your solutions work, but they are nearly word-word duplicates of Ariel's answer at the top of this page. While you may have come up with your solution on your own, you should never post your solution an answer if someone else has already beaten you to it. P.S. There isn't really any "or." Both of your solutions function identical in all cases, except the former solution is better because it takes up one less character (in the minified file).
Comment: this answer could be improved by discussing the difference between `.test` and `.match`, and when to use each one, or at least referencing an answer that does, like this one: https://stackoverflow.com/questions/10940137/regex-test-v-s-string-match-to-know-if-a-string-matches-a-regular-expression/10940138#10940138


Omega Theme: how to change the stacking order of each region for each layout?

Question: I´m starting my first theme with Omega. I´ve read the documentation, and managed to understand how it works.
My problem here is that I don´t know how to stack regions in different order for each layout.
I know that I can hide some regions with css and the different css files provided by the omega subtheme, but how to change the order?
Let me explain with an example:
I have this in my wide and tablet (normal) layout:

HEADER SECTION:
User Bar First -- User Bar Second -- Branding
CONTENT SECTION: ...
FOOTER SECTION: ...

For my mobile version I have this:

HEADER SECTION:
User Bar First
User Bar Second
Branding
CONTENT SECTION: ...
FOOTER SECTION: ...

And I want it to be:

HEADER SECTION:
User Bar First
CONTENT SECTION: ...
User Bar Second
Branding
FOOTER SECTION: ...

Any idea on how to do that?
Thanks!
Answer: The browser, not the server, knows the width of the layout. So you cannot change the stacking order on the server side and move the regions or zones up and down without having the browser talk back to the server, which Omega doesn't do.
You could use Javascript to move things around, which is painful. You could also duplicate the content you want to be in the different places, and then hide it or display it with display: none and display: block. But then you are outputting multiple copies of the content and lengthening page load time.
Here's a similar discussion about (not) changing stacking order in Omega: http://drupal.org/node/1343798.
Comment: Omega does do media queries, though... how is that different from the user agent talking back to the server?
Comment: I should have been less hand-wavy. A media query is part of a  tag that gets sent with the page, and controls whether a CSS file gets loaded. But by itself (without Javascript) it can't control the actual content loading.
Answer: The following is an example of selective region/zone display as show on this comment from Dan Halbert to how to change the stacking order for mobile layouts. It shows how to use the CSS display attribute to hide the header zone in the mobile layout.

Put this in global.css:
.zone-header { display: none; }

Put this in mytheme-alpha-default.css:
.zone-header { display: block; }

The header zone will be visible in all but the mobile layout. You can imagine variations on that: If you wanted to hide in the narrow layout, but not in normal and wide, you could put the display: block in mytheme-alpha-default-normal.css and mytheme-alpha-wide.css.

See also Additive content / display:none problem.
There are some good comments on this on Display problem.
Comment: The first part of this answer is a direct unattributed quote from something I wrote here: http://drupal.org/node/1343798#comment-5267222. I don't mind your using what I wrote to help answer a question, but please say where you got it from, and adapt to the question at hand.
Comment: Thanks Vic!! But... how do I rearrange those zones? I want to display the header in the mobile version, but at the bottom of the layout...
Comment: Hi Dan, Ofc, I was actually in highlighting your quote and linking to it, But Kiamlaluno beat me to it, by 59 seconds
Answer: The answer can be found here: http://drupal.org/node/1343798#comment-7180142
Comment: That post has already been listed in both answers, and link only answers aren't considered good ones here. Can you summarize how this is different.


How should I transform non-negative data including zeros?

Question: If I have highly skewed positive data I often take logs. But what should I do with highly skewed non-negative data that include zeros? I have seen two transformations used:

$\log(x+1)$ which has the neat feature that 0 maps to 0.
$\log(x+c)$ where c is either estimated or set to be some very small positive value.

Are there any other approaches? Are there any good reasons to prefer one approach over the others?
Comment: I've summarized some of the answers plus some other material at http://robjhyndman.com/researchtips/transformations/
Comment: excellent way to transform and promote stat.stackoverflow !
Comment: It seems strange to ask about how to transform without having stated the purpose of transforming in the first place. What is the situation? Why is it necessary to transform? If we don't know what you're trying to achieve, how can one reasonably suggest *anything*? (Clearly one cannot hope to transform to normality, because the existence of a (non-zero) probability of exact zeros implies a spike in the distribution at zero, which spike no transformation will remove -- it can only move it around.)
Comment: Yes, I agree @robingirard (I just arrived here now because of Rob's blog post)!
Comment: Also see http://stats.stackexchange.com/questions/39042/how-should-i-handle-a-left-censored-predictor-variable-in-multiple-regression for an application to left-censored data (which can be characterized, up to a shift of location, exactly as in the present question).
Comment: This Q is also relevant to this discussion: [How small a quantity should be added to x to avoid taking the log of zero](http://stats.stackexchange.com/questions/30728/).
Comment: I've been playing around with log(x+d)-c, where c=floor(log(smallest-non-zero-value)), and d=exp(c).  This also has the feature that 0 maps to 0.

However, I've been trying to interpret the coefficients resulting from this model, and I've been having trouble- In particular, the feature of a log-linear model on a continuous predictor X where a 1-unit change in X results in the same percent change in Y, regardless of choice of X and X+1 -- this no longer seems to hold due to the small adjustment.  For that reason, I'm starting to suspect that the alternate transformations below are superior.
Answer: No-one mentioned the inverse hyperbolic sine transformation. So for completeness I'm adding it here.
This is an alternative to the Box-Cox transformations and is defined by
\begin{equation}
f(y,\theta) = \text{sinh}^{-1}(\theta y)/\theta = \log[\theta y + (\theta^2y^2+1)^{1/2}]/\theta,
\end{equation}
where $\theta>0$. For any value of $\theta$, zero maps to zero. There is also a two parameter version allowing a shift, just as with the two-parameter BC transformation. Burbidge, Magee and Robb (1988) discuss the IHS transformation including estimation of $\theta$.
The IHS transformation works with data defined on the whole real line including negative values and zeros. For large values of $y$ it behaves like a log transformation, regardless of the value of $\theta$ (except 0). The limiting case as $\theta\rightarrow0$ gives $f(y,\theta)\rightarrow y$.
It looks to me like the IHS transformation should be a lot better known than it is.
Comment: That paper is about the inverse sine transformation, not the inverse hyperbolic sine.
Comment: About the IHS some seems to disagree:  http://onlinelibrary.wiley.com/doi/10.1890/10-0340.1/abstract
Comment: Looks like a good alternative to $tanh$/logistic transformations
Comment: Other notations often met -- either in mathematics or in programming languages -- are asinh, arsinh, arcsinh. The latter is common but should be deprecated as this function does not refer to arcs, but to areas. See https://www.jstor.org/stable/i326521
Answer: It seems to me that the most appropriate choice of transformation is contingent on the model and the context.
The '0' point can arise from several different reasons each of which may have to be treated differently:

Truncation (as in Robin's example): Use appropriate models (e.g., mixtures, survival models etc)
Missing data: Impute data / Drop observations if appropriate.
Natural zero point (e.g., income levels; an unemployed person has zero income): Transform as needed
Sensitivity of measuring instrument: Perhaps, add a small amount to data?

I am not really offering an answer as I suspect there is no universal, 'correct' transformation when you have zeros.
Comment: Every answer to my question has provided useful information and I've up-voted them all. But I can only select one answer and Srikant's provides the best overview IMO.
Comment: Also note that there are zero-inflated models (extra zeroes and you care about some zeroes: a mixture model), and hurdle models (zeroes and you care about non-zeroes: a two-stage model with an initial censored model).
Answer: A useful approach when the variable is used as an independent factor in regression is to replace it by two variables: one is a binary indicator of whether it is zero and the other is the value of the original variable or a re-expression of it, such as its logarithm.  This technique is discussed in Hosmer & Lemeshow's book on logistic regression (and in other places, I'm sure).  Truncated probability plots of the positive part of the original variable are useful for identifying an appropriate re-expression.  (See the analysis at https://stats.stackexchange.com/a/30749/919 for examples.)
When the variable is the dependent one in a linear model, censored regression (like Tobit) can be useful, again obviating the need to produce a started logarithm.  This technique is common among econometricians.
Comment: @David, although it seems similar, it's not, because the ZIP is a model of the *dependent* variable, not the independent variable.
Comment: Is modeling data as a zero-inflated Poisson a special case of this approach?
Comment: @landroni H&L was fresh in my mind back then, so I feel confident there's *something* in that book related to this topic. (I had consulted with it to develop some very large regression models and many of the independent variables had to be treated in this way.) However, I can't find the reference at this point, either. I have written about this technique in subsequent posts, in case you're looking for details.  Two that show up in a site search are at http://stats.stackexchange.com/questions/6563 and http://stats.stackexchange.com/questions/4831.
Comment: @whuber *This technique is discussed in Hosmer & Lemeshow's book on logistic regression* By any chance would you happen to know in which chapter they discuss this technique? I'm looking at their book, but can't seem to find the right page...
Comment: @landroni Yes, they are equivalent, in the same way that all numerical encodings of any binary variable are equivalent. Choose whichever one you find most convenient to interpret.
Comment: @whuber *two variables: one is a binary indicator of whether it is zero and the other is the value of the original variable* I'm slightly confused as to the proper parametrization in this technique. Here (as well as in [this answer](http://stats.stackexchange.com/a/4833/36515)) you suggest that the dummy should "equal 1 when $x=0$ and 0 otherwise". While in [this other answer](http://stats.stackexchange.com/a/6565/36515), the dummy takes 1 when $x>0$ and 0 when $x=0$. Are the two formulations equivalent?
Answer: The log transforms with shifts are special cases of the Box-Cox transformations:
$y(\lambda_{1}, \lambda_{2}) =
\begin{cases}
 \frac {(y+\lambda_{2})^{\lambda_1} - 1} {\lambda_{1}} & \mbox{when } \lambda_{1} \neq 0 \\ \log (y + \lambda_{2}) & \mbox{when } \lambda_{1} = 0
\end{cases}$
These are the extended form for negative values, but also applicable to data containing zeros.  Box and Cox (1964) presents an algorithm to find appropriate values for the $\lambda$'s using maximum likelihood.  This gives you the ultimate transformation.
A reason to prefer Box-Cox transformations is that they're developed to ensure assumptions for the linear model.  There's some work done to show that even if your data cannot be transformed to normality, then the estimated $\lambda$ still lead to a symmetric distribution.
I'm not sure how well this addresses your data, since it could be that $\lambda = (0, 1)$ which is just the log transform you mentioned, but it may be worth estimating the requried $\lambda$'s to see if another transformation is appropriate.
In R, the boxcox.fit function in package geoR will compute the parameters for you.
Comment: For anyone who reads this wondering what happened to this function, it is now called `boxcoxfit`.
Comment: @Rob: Oh, sorry.  Diggle's geoR is the way to go -- but specify `lambda2=TRUE` in the arguments to `boxcox.fit`.  (Also updated the answer.)
Comment: @ars. I think the `boxcox` function in MASS only estimates $\lambda_1$ and assumes $\lambda_2=0$.
Answer: I'm presuming that zero != missing data, as that's an entirely different question.
When thinking about how to handle zeros in multiple linear regression, I tend to consider how many zeros do we actually have?
Only a couple of zeros
If I have a single zero in a reasonably large data set, I tend to:

Remove the point, take logs and fit the model
Add a small $c$ to the point, take logs and fit the model

Does the model fit change? What about the parameter values? If the model is fairly robust to the removal of the point, I'll go for quick and dirty approach of adding $c$.
You could make this procedure a bit less crude and use the boxcox method with shifts described in ars' answer.
Large number of zeros
If my data set contains a large number of zeros, then this suggests that simple linear regression isn't the best tool for the job. Instead I would use something like mixture modelling (as suggested by Srikant and Robin).
Answer: If you want something quick and dirty why not use the square root?
Comment: And frequently the cube root transformation works well, and allows zeros and negatives.  I've found cube root to particularly work well when, for example, the measurement is a volume or a count of particles per unit volume.  Cube root would convert it to a linear dimension.  A more flexible approach is to fit a restricted cubic spline (natural spline) on the cube root or square root, allowing for a little departure from the assumed form.
Comment: +1. For a little article on cube roots, see http://www.stata-journal.com/article.html?article=st0223 (This will be a free .pdf from first quarter 2014.)
Comment: A square root of zero, is zero, so only the non-zeroes values are transformed. This does nothing to deal with the spike, if zero inflated, and can cause serious problems if, in groups, each has a different amount of zeroes. In other words, if some groups have many zeroes and others have few, this transformation can affect many things in a negative way. For the group with the largest variance (also had the least zeroes), almost all values are being transformed. In contrast, those with the most zeroes, not much of the values are transformed. This can change which group has the largest variance.
Comment: No transformation will maintain the variance in the case described by @D_Williams. Mixture models (mentioned elsewhere in this thread) would probably be a good approach in that case.
Answer: Comparing the answer provided in by @RobHyndman to a log-plus-one transformation extended to negative values with the form:
$$T(x) = \text{sign}(x) \cdot \log{\left(|x|+1\right)} $$
(As Nick Cox pointed out in the comments, this is known as the 'neglog' transformation)
r = -1000:1000

l = sign(r)*log1p(abs(r))
l = l/max(l)
plot(r, l, type = "l", xlab = "Original", ylab = "Transformed", col = adjustcolor("red", alpha = 0.5), lwd = 3)

#We scale both to fit (-1,1)
for(i in exp(seq(-10, 100, 10))){
  s = asinh(i*r)

  s = s / max(s)
  lines(r, s, col = adjustcolor("blue", alpha = 0.2), lwd = 3)
}
legend("topleft", c("asinh(x)", "sign(x) log(abs(x)+1)"), col = c("blue", "red"), lty = 1)

As you can see, as $\theta$ increases more the transform looks like a step function. With $\theta \approx 1$ it looks a lot like the log-plus-one transformation. And when $\theta \rightarrow  0$ it approaches a line.

EDIT: Keep in mind the log transform can be similarly altered to arbitrary scale, with similar results. I just wanted to show what $\theta$ gives similar results based on the previous answer. The biggest difference between both approaches is the region near $x=0$, as we can see by their derivatives.
Comment: This transformation has been dubbed the neglog. In my view that is an ugly name, but it reflects the principle that useful transformations tend to acquire names as well having formulas. https://www.jstor.org/stable/3592674
Comment: @NickCox interesting, thanks for the reference!
Answer: I assume you have continuous data.
If the data include zeros this means you have a spike on zero which may be due to some particular aspect of your data. It appears for example in wind energy, wind below 2 m/s produce zero power (it is called cut in) and wind over (something around) 25 m/s also produce zero power (for security reason, it is called cut off).  While the distribution of produced wind energy seems continuous there is a spike in zero.
My solution:  In this case, I suggest to treat the zeros separately by working with a mixture of the spike in zero and the model you planned to use for the part of the distribution that is continuous (wrt Lebesgue).
Answer: Since the two-parameter fit Box-Cox has been proposed, here's some R to fit input data, run an arbitrary function on it (e.g. time series forecasting), and then return the inverted output:
# Two-parameter Box-Cox function
boxcox.f <- function(x, lambda1, lambda2) {
  if (lambda1!=0) {
    return(((x + lambda2) ^ lambda1 - 1) / lambda1)
  } else {
    return(log(x + lambda2))
  }
}

# Two-parameter inverse Box-Cox function
boxcox.inv <- function(x, lambda1, lambda2) {
  if (lambda1!=0) {
    return((lambda1 * x + 1) ^ (1 / lambda1) - lambda2)
  } else {
    return(exp(x) - lambda2)
  }
}

# Function to Box-Cox transform x, apply function g,
# and return inverted Box-Cox output y
boxcox.fit.apply <- function(x, g) {
  require(geoR)
  require(plyr)

  # Fit lambdas
  t <- try(lambda.pair <- boxcoxfit(x, lambda2=T)$lambda)

  # Estimating both lambdas sometimes fails; if so, estimate lambda1 only
  if (inherits(t, "try-error")) {
    lambda1 <- boxcoxfit(x)$lambda
    lambda2 <- 0
  } else {
    lambda1 <- lambda.pair[1]
    lambda2 <- lambda.pair[2]
  }
  x.boxcox <- boxcox.f(x, lambda1, lambda2)

  # Apply function g to x.boxcox. This should return data similar to x (e.g. ts)
  y <- aaply(x.boxcox, 1, g)

  return(boxcox.inv(y, lambda1, lambda2))
}

Answer: The Yeo-Johnson power transformation discussed here has excellent properties designed to handle zeros and negatives while building on the strengths of Box Cox power transformation. This is what I typically go to when I am dealing with zeros or negative data.
Here is a summary of transformations with pros/cons to illustrate why Yeo-Johnson is preferable.
Log
Pros: Does well with positive data.
Cons: Does not handle zeros.
> log(0)
[1] -Inf

Log Plus 1
Pros: The plus 1 offset adds the ability to handle zeros in addition to positive data.
Cons: Fails with negative data
> log1p(-1)
[1] -Inf
> log1p(-2)
[1] NaN
Warning message:
In log1p(-2) : NaNs produced

Square Root
Pros: Uses a power transformation that can handle zeros and positive data.
Cons: Fails with negative data
> sqrt(-1)
[1] NaN
Warning message:
In sqrt(-1) : NaNs produced

Box Cox
R Code:
box_cox <- function(x, lambda) {

    eps <- 0.00001
    if (abs(lambda) < eps)
        log(x)
    else
        (x ^ lambda - 1) / lambda

}

Pros: Enables scaled power transformations
Cons: Suffers from issues with zeros and negatives (i.e. can only handle positive data.
> box_cox(0, lambda = 0)
[1] -Inf
> box_cox(0, lambda = -0.5)
[1] -Inf
> box_cox(-1, lambda = 0.5)
[1] NaN

Yeo Johnson
R Code:
yeo_johnson <- function(x, lambda) {

    eps <- .000001
    not_neg <- which(x >= 0)
    is_neg  <- which(x < 0)

    not_neg_trans <- function(x, lambda) {
        if (abs(lambda) < eps) log(x + 1)
        else ((x + 1) ^ lambda - 1) / lambda
    }

    neg_trans <- function(x, lambda) {
        if (abs(lambda - 2) < eps) - log(-x + 1)
        else - ((-x + 1) ^ (2 - lambda) - 1) / (2 - lambda)
    }

    x[not_neg] <- not_neg_trans(x[not_neg], lambda)

    x[is_neg] <- neg_trans(x[is_neg], lambda)

    return(x)

}

Pros: Can handle positive, zero, and negative data.
Cons: None that I can think of. Properties are very similar to Box-Cox but can handle zero and negative data.
> yeo_johnson(0, lambda = 0)
[1] 0
> yeo_johnson(0, lambda = -0.5)
[1] 0
> yeo_johnson(-1, lambda = 0.5)
[1] -1.218951

Comment: Cons for Yeo–Johnson: complex, separate transformation for positives and negatives and for values on either side of lambda, magical tuning value (epsilon; and what is lambda?). No readily apparent advantage compared to the simpler negative-extended log transformation shown in Firebug's answer, unless you require scaled power transformations (as in Box–Cox).
Comment: Cons for Log(x+1): it is arbitrary and rarely is the best choice.  See https://stats.stackexchange.com/questions/30728.
Comment: Inverse hyperbolic sine (IHS) transformation, as described in the OP's own answer and blog post, is a simple expression and it works perfectly across the real line. There's still an arbitrary scaling parameter.
Answer: Suppose Y is the amount of money each American spends on a new car in a given year (total purchase price).  Y will spike at 0; will have no values at all between 0 and about 12,000; and will take other values mostly in the teens, twenties and thirties of thousands.  Predictors would be proxies for the level of need and/or interest in making such a purchase.  Need or interest could hardly be said to be zero for individuals who made no purchase; on these scales non-purchasers would be much closer to purchasers than Y or even the log of Y would suggest.  In a case much like this but in health care, I found that the most accurate predictions, judged by test-set/training-set crossvalidation, were obtained by, in increasing order,

Logistic regression on a binary version of Y,
OLS on Y,
Ordinal regression (PLUM) on Y binned into 5 categories (so as to divide purchasers into 4 equal-size groups),
Multinomial logistic regression on Y binned into 5 categories,
OLS on the log(10) of Y (I didn't think of trying the cube root), and
OLS on Y binned into 5 categories.

Some will recoil at this categorization of a continuous dependent variable.  But although it sacrifices some information, categorizing seems to help by restoring an important underlying aspect of the situation -- again, that the "zeroes" are much more similar to the rest than Y would indicate.
Comment: You could also split it into two models: the probability of buying a car (binary response), and the value of the car given a purchase. This is the standard practice in many fields, eg insurance, credit risk, etc.
Comment: @HongOoi - can you suggest any readings on when this approach is and isn't applicable?
Answer: To clarify how to deal with the log of zero in regression models, we have written a pedagogical paper explaining the best solution and the common mistakes people make in practice. We also came out with a new solution to tackle this issue.
You can find the paper by clicking here:  https://ssrn.com/abstract=3444996
First, we think that ones should wonder why using a log transformation. In regression models, a log-log relationship leads to the identification of an elasticity. Indeed, if $\log(y) = \beta \log(x) + \varepsilon$, then $\beta$ corresponds to the elasticity of $y$ to $x$. The log can also linearize a theoretical model. It can also be used to reduce heteroskedasticity. However, in practice, it often occurs that the variable taken in log contains non-positive values.
A solution that is often proposed consists in adding a positive constant c to all observations $Y$ so that $Y + c > 0$. However, contrary to linear regressions, log-linear
regressions are not robust to linear transformation of the dependent variable. This
is due to the non-linear nature of the log function. Log transformation expands low
values and squeezes high values. Therefore, adding a constant will distort the (linear)
relationship between zeros and other observations in the data. The magnitude of the
bias generated by the constant actually depends on the range of observations in the
data. For that reason, adding the smallest possible constant is not necessarily the best
worst solution.
In our article, we actually provide an example where adding very small constants is actually providing the highest bias. We provide derive an expression of the bias.
Actually, Poisson Pseudo Maximum Likelihood (PPML) can be considered as a good solution to this issue. One has to consider the following process:
$y_i = a_i \exp(\alpha + x_i' \beta)$ with $E(a_i | x_i) = 1$
This process is motivated by several features. First, it provides the same interpretation
to $\beta$ as a semi-log model. Second, this data generating process provides a logical
rationalization of zero values in the dependent variable. This situation can arise when
the multiplicative error term, $a_i$ , is equal to zero. Third, estimating this model with PPML does not encounter the computational difficulty when $y_i = 0$. Under the assumption that $E(a_i|x_i) = 1$, we have $E( y_i - \exp(\alpha + x_i' \beta) | x_i) = 0$. We want to minimize the quadratic error of this moment, leading to the following first-order conditions:
$\sum_{i=1}^N ( y_i - \exp(\alpha + x_i' \beta) )x_i' = 0$
These conditions are defined even when $y_i = 0$. These first-order conditions are numerically equivalent to those of a Poisson model, so it can be estimated with any standard statistical software.
Finally, we propose a new solution that is also easy to implement and that provides unbiased estimator of $\beta$. One simply need to estimate:
$\log( y_i + \exp (\alpha + x_i' \beta)) =  x_i' \beta + \eta_i $
We show that this estimator is unbiased and that it can simply be estimated with GMM with any standard statistical software. For instance, it can be estimated by executing just one line of code with Stata.
We hope that this article can help and we'd love to get feedback from you.
Christophe Bellégo and Louis-Daniel Pape
CREST - Ecole Polytechnique - ENSAE
Answer: I had the same problem with data and no transformation would give reasonable distribution. I came up with the following idea. I would appreciate if someone decide whether it is worth  utilising as I am not a statistitian.
We may adopt the assumption that 0 is not equal to 0. There is a hidden continuous value which we observe as zeros but, the low sensitivity of the test gives any values more than 0 only after reaching the treshold. So what we observe is more like half-normal distribution where all the left side of normal distribution is shown as one rectangle (x=0) in histogram. So maybe we can just perform following steps:

Transform the variable to dychotomic values (0 are still zeros, and >0 we code as 1)
We search for another continuous variable with high Spearman correlation coefficent with our original variable.
We perform logistic regression which predicts 1. Dependant variable - dychotomic, independant - highly correlated variable
We look at predicted values for observed zeros in logistic regression.
We recode zeros in original variable for predicted in logistic regression. We leave original values higher than 0 intact (however they must be higher than 1)
We rank the original variable with recoded zeros.
We normalize the ranked variable with Blom - f(r) = vnormal((r+3/8)/(n+1/4); 0;1) where r is a rank; n - number of cases, or Tukey transformation.
Answer: Depending on the problem's context, it may be useful to apply quantile transformations.
The idea itself is simple*, given a sample $x_1, \dots, x_n$, compute for each $i \in \{1, \dots, n\}$ the respective empirical cumulative density function values $F(x_i) = c_i$, then map $c_i$ to another distribution via the quantile function $Q$ of that distribution, i.e., $Q(c_i)$.
*Assuming you don't apply any interpolation and bounding logic.
With the method out of the way, there are several caveats, features, and notes which I will list below (mostly caveats). Details can be found in the references at the end.

These methods are lacking in well-studied statistical properties

Does not necessarily maintain type 1 error, and can reduce statistical power.

Interpretation is difficult
Requires a large number of samples
Not easily translated to multivariate data
Typically applied to marginal distributions. It may be tempting to think this transformation helps satisfy linear regression models' assumptions, but the normality assumption for linear regression is for the conditional distribution.
Correlations not preserved
In the case of Gaussians, the median of your data is transformed to zero.
Non-parametric
Expensive to compute
Quantiles depend on your sample
Is a monotone and invertible transformation

Related questions and references:

Quantile Transformation with Gaussian Distribution - Sklearn Implementation
Quantile transform vs Power transformation to get normal distribution
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2921808/


cannot echoing result from table

Question: I can echo $row_count but I am unable to print $first.
Account table contain fname and lname.
Can anyone fix this problem?
public function getAccountInfoAll(){
        $acc_info = $this->con->query("SELECT * FROM account");
        $results = $acc_info->fetchAll();
        $row_count = $acc_info->rowCount();
        echo $row_count.' rows selected';
        while ($row = $acc_info->fetchAll())
          {
          $first = $row['fname'];
          echo $first;
          }
        }

Answer: I'm guessing your using PDO, so here's what you did wrong :
$acc_info = $this->con->query("SELECT * FROM account");
$acc_info->execute();
$results = $acc_info->fetchAll(PDO::FETCH_ASSOC);
$row_count = $acc_info->rowCount();
echo $row_count.' rows selected';
foreach($results AS $row) {
   $first = $row['fname'];
   echo $first.'
';
}

Answer: Something like this:
public function getAccountInfoAll(){
    $acc_info = $this->con->prepare("SELECT * FROM account");
    $acc_info->execute();

    $row_count = $acc_info->rowCount();
    echo $row_count.' rows selected';

    while ($row = $acc_info->fetch(PDO::FETCH_ASSOC)) {
      $first = $row['fname'];
      echo $first;
    }
}

More information on prepared statements.

cannot echoing result from table


how to create foreign reference key for composition primary key

Question: Suppose we have the following two tables
Create table Bank (
   Bank_ID Numeric Not Null,
   Bank_Card Numeric Not Null,
   Primary Key(Bank_ID, Bank_Card)
)

Create table Customer (
   Customer_ID Numeric Not Null,
   Name varchar(30) Not Null,
   Primary key(Customer_ID)
)

Where Customer_ID is generated by concatenating Bank_ID and Bank_Card. How can I set foreign key Customer_ID to reference Bank_ID and Bank_Card
Comment: Short answer: You can't. Long answer would be to redesign the tables or add a generated column.
Answer: What you want is a constraint, but it's not a FK (foreign key) constraint.(A FK constraint says that values for a column list appear elsewhere as PK/UNIQUE.) To enforce it declaratively you could add redundant generated (computed/calculated) column Customer_ID to Bank and a FK to it in Customer. To enforce it without adding redundant columns you need triggers. But smart keys are a bad idea.


Working with the equation: $25^{\log_{10}(x)}=5+4x^{\log_{10}(5)}$

Question: The answer should be $10$, but I don't know how to get it.
I've tried many different ways and I still can't get it.
I tried this :
 $$25^{\log_{10}(x)}=5+4x^{\log_{10}(5)}\implies \log_{10}(25^{\log_{10}(x)} - 5) = \log_{10}(4x) \cdot \log_{10}(5).$$
Comment: This is close to unreadable. Please use the MathJax typesetting.
Answer: You have
$$25^{\log_{10}(x)} = 5^{2 \log_{10}(x)} = e^{2 \frac{\ln(x)}{\ln(10)} \ln(5)} = x^{2 \log_{10}(5)}$$
So, let's define $X = x^{\log_{10}(5)}$. Your equation becomes
$$X^2 = 5 + 4X$$
i.e. $X=5$ or $X=-1$. Of course $X=-1$ is impossible because by definition $X>0$. So $X=5$, i.e.
$$x^{\log_{10}(5)} = 5$$
You obtain finally
$$x = 5^{\frac{1}{\log_{10}(5)}} = 5^{\frac{\ln(10)}{\ln(5)}} = e^{\frac{\ln(10)}{\ln(5)} \ln(5)} = e^{\ln(10)} = 10$$
Answer: Hint: make a change: $x=10^t$.


Erro ao conectar no PostgreSQL pelo pgAdmin 4

Question: Instalei o pgAdmin 4 no meu note com Windows 10 e não estou conseguindo acessar o banco de dados.

Host name/address = /22.214.171.124
  Porta 5432
  Maintenance Database = xxxxxx
  Username = cesar_r
  Password = ******
  SSL mode = Prefer

O nº do IP está configurado no pg_hba.conf da seguinte forma:

host all all 126.96.36.199/32 password

No endereço coloco o nº do IP precedido de / (conforme consta na documentação).
O banco está instalado num servidor Linux (Ubuntu).
OBS: Estou tentando conectar com um usuário simples com permissão somente de leitura.
Comment: Que erro que ele acusa?
Comment: No pgAdmin 4 é necessário indicar o diretório dos binários, você fez isso? como já disse que é Windows 10 é provável que seja `C:\Program Files\PostgreSQL\*versao*\bin`. Precisamos saber qual é o erro.
Answer: Deu um trabalhinho pra rodar... mas é o seguinte, após instalar o pgAdmin4
Num prompt cmd (não serve do git bash) acesse o diretório do postgres
cd c:\Program Files\PostgreSQL\SUA_VERSAO\bin

Rode o init
initdb -D "c:/Program Files/PostgreSQL/9.6/data"

Inicie o pg  definindo um diretório para o log
pg_ctl -D "c:/Program Files/PostgreSQL/9.6/data" -l "C:/Users/SEU_USUARIO/logfile.log" start

Pronto. O usuário que você usou na instalação deve ser usado para criar o servidor no pgAdmin4
caso esteja na dúvida sobre o user cadastrado, digite ainda no prompt:
createuser

Ele vai informar o nome já existente.


Correlation between spot and futures

Question: I am Airliner.I want to protect my business from price volatility of jet fuel cost.Jet fuel is not traded in futures market but Crude oil is traded in futures market. I have daily spot prices of jet fuel and contract prices of crude oil.
e.g. on 19th April,spot price(Jet fuel) is 100.April contract price (crude oil) = 103,May contract price = 110,June Contract price =118 and July contract price = 109 and August price = 125. Spot and contract prices change on daily basis as per demand-supply dynamics.
Should I find correlation between spot price on particular date and contract prices on that date for all contracts? How should I go about it?
Please help me with approach.
Comment: This stupid app won't let me post a comment, so this will go in the answer section. Could you be more specific? This sounds like a question from a textbook. If so, probably would help to know the text, then I could pinpoint what they are looking for. My guess is if you're an airline, you're not actively trading futures: so time series analysis is out the window. The most simple approach would simply be to buy futures in the months you think oil would rise, at a ratio defined by the slope of the linear correlation between jet fuel and spot oil.
Comment: 1) No.It's not from any textbook.
2) We can assume that airlines trade Crude future contracts in the market.
3) I have daily data available for spot and future prices.Should I use,say,19th April's spot price and future prices of april,may,june,July and August contracts on that day? How should I establish correlation?
Answer: Well, because I'm new to this site, I am not allowed to comment. I will answer and have a moderator move it to the comments section (again).
Honestly, your question is way too broad. What do you mean by "daily data"? Tick data? Daily closing prices? Both? The literature on futures hedging is endless, so you need to be more specific in what context you need this answered, then I can be of more help.
Regressing futures prices against the underlying spot price is sort of circular. There are well known futures pricing models (based off spot, risk-free rates, etc... See Options, Futures and Other Derivatives by John C. Hull, for starters) that should theoretically GIVE you the futures price based off of spot.
Another approach is using spot prices and the yield curve to bootstrap an implied futures/spot relationship extending to the deferred months. This lets you see what the market is implying about futures prices of the deferred months, and where things are actually trading (due to liquidity issues, etc... This is common in products like soybeans, eg).
EDIT: Just to clarify, the reason you wouldn't want to regress the futures price against the underlying is that we already know there is a very direct correlation between the two assets. The spot asset is used in deriving the fair-value futures price. To use a concrete example, there are well known formulas to price out US Treasury futures based off of cash treasuries. What you're suggesting is to regress Treasury futures against treasury cash, and that is just silly. What you should be doing is regressing the Treasury BASIS (essentially buying the cash vs selling future and vice versa) against some other, external stimuli (like the Fed Funds rate, or the temperature outside - to use a silly example).
Comment: By daily data,I mean closing price.Thanks for response. I have collected data(historic prices).Let me tell you something about prices.April contract opens in 10th December previous year and expires on 20th April.So 20th April spot price can be put into equation using future prices of April,May,June,July and August contract on that day.But 21st April will have one variable because of April contract expiry.Again on 10th May,we will have additional variable September contract prices.When I tried to solve it using multiple regression in Excel,I got error since some of the fields were blank.conti..
Comment: e.g. April Future contract price on 21st April.I don't know how to proceed further.Should I split my data in various intervals and then apply multiple regression?In that case,how should I collate the data in the end? Please let me know if my approach is wrong.Thanks for reading.
Comment: Well, one thing you want to do is take note of the spot month, or the month that exhibits the highest volume. Typically, that is the closest month to expirty.
Comment: HOWEVER, there is something called the "roll" where traders roll their futures from the expiring month into the next month closest month, or the next serial month, whichever has more volume. That contract then becomes the "top" contract, named so because the location pit traders used to stand in order to trade it (the top of the steps). Free information is available to see which contract exhibits the most volume and when there is a switch. Perhaps this is free too, but subscription software will actually produce a smoothed curve to account for these gaps that you undoubtedly are now aware of.

Correlation between spot and futures


Find $ \lambda$ such that f(x) has no point of local maxima or minima in $|x|<\frac{π}{2}$

Question: Let $$f(x)=(sin(x))^3+ \lambda(sin(x))^2$$ for every $x$ in domain of $(\frac{-π}{2},\frac{π}{2})$ has no point of local maxima or minima, then find the value of $\lambda$ for the given condition to follow.
My process:
(1): found $f'(x)$ and equated with zero to find that it only matters by $(3sinx + 2\lambda)$ $ $  and $sinx$ to have change in the sign of derivative around zero.So the $\lambda$ becomes 0 and 0 becomes the point of inflection.
Is my process wrong?
Answer: Solution
We have
$$f'(x)=3\sin^2 x\cos x+2\lambda\sin x \cos x=\sin x \cos x(3\sin x+2\lambda),$$ and $$f''(x)=2\cos^2 x(3\sin x+\lambda)-\sin^2 x(2\lambda+3\sin x).$$
Notice that, whatever $\lambda$ is, $f'(0)=0.$ Thus, we at least need $f''(0)=0.$ Otherwise, $f(x)$ reaches its local extremum value at $x=0$. But $f''(0)=2\lambda.$ Hence $\lambda=0$, which has only one possible value. Now, we may verify that $f(x)=\sin^3 x$ could satisfy the conditions we supposed.
Answer: Yes, your process is correct, the only $\lambda$ for which there is no maxima or minima in the interval $(-\pi/2,\pi/2)$ is $\lambda = 0$.


ios PWA reduce innerheight after return back from external link

Question: aftre return back from payment gateway in PWA app in ios, window.innerHeight will be decreased and there will be a free space at bottom page.

i tried to reload app but it didn't work.
i need a solution to force safari to recalculate innerHeight or anything else.
thanks for your help.
Answer: This problem occurs because of the automatic innerHeight change by the Safari browser. so that after opening the external link and returning to pwa, it decreases the height value. You can save the innerHeight value in local storage before opening the link. And after returning to your app, set the min height value of the body equal to the innerHeight stored in the local storage.
The important thing when setting the height value for a body is that the values must be in pixels.
// before open link
window.localStorage.setItem('bodyHeight, `${window.innerHeight}px`);

// after return to pwa app
document.getElementsByTagName('body')[0].style.minHeight = window.localStorage.getItem('bodyHeight');


What's the Italian musical term for "in minuet steps"?

Question: I am writing down some choir music in which, beginning at a certain point, the bass starts to sing continuous eighths in a particular manner: the first eighth is tied to the second one, but the second and third ones are sung staccato.
If I recall correctly, this technique is called a "minuet step". I am looking for adding a comment in Italian to the notes so that I don't have to write the legatos and staccatos explicitly all over the place (as this part of the piece is quite long). My question is: how does one say "The bass in minuet steps" in Italian?
I don't speak Italian, but after searching for similar expressions and machine-translating "minuet steps" in Italian, I've come up with

Passi di minuetto il basso

but I'm not anywhere near sure this is correct.
Comment: Dont skimp on the use of staccato marks. It is poor not to indicate exactly what notes you want played detached,.
Answer:
I am looking for adding a comment in Italian to the notes so that I
  don't have to write the legatos and staccatos explicitly all over the
  place (as this part of the piece is quite long).

Although I know generally what a minuet is musically, I don't know what "minuet steps" means. I wouldn't associate the phrase with any particular articulation pattern, and could only guess how to interpret that phrase if I saw it in music. Unless your choir is well-versed in Baroque dance steps, its likely they won't understand it either.
Rather than use a term that may or may not be understood by the performers, it may be better to write out the desired articulation for the first few bars, and then use the very common term simile, which means to continue playing "in the same way." This would prevent you from having to mark the same articulations throughout the section, and will be generally understood by most musicians.
In addition, there is a somewhat common tempo marking, a tempo di menuetto, which means to play in the tempo and style of a minuet. You could use this to indicate to the performer that this section should sound like a graceful dance, but don't necessarily expect it to indicate a precise articulation pattern.
Comment: Interesting… maybe it's a national peculiarity then. In Hungary, almost everyone calls this kind of articulation a "minuet step". Go figure.


How much does a tailpiece affect the overall sound quality of an electric guitar?

Question: On a typical stoptail bridge such as the one pictured below, how much does the quality of the tailpice (or "stopbar") affect the overall sound quality of the guitar? What kind of advantages can a guitarist see by upgrading this part of the bridge setup?
For reference, here is a typical two-piece stoptail setup (from Wikipedia):
Comment: While I have a lot of signal processors and effects for my guitars, I'd have to respectfully disagree Carl. Getting the sound right before you start playing with it is essential. If you start off with poor sound there is only so much you can do with effects. Which is why so many good guitarists have many guitars - they sound different.
Comment: @DrMayhem I accept your response without reservation, but would like to point out that, on almost any instrument, the quality of the instrument affects how hard the musician has to work to produce a good sound (as opposed to whether a good sound can be produced).  I've seen pro violinists get amazing sound out of a 1/2-size child's rental, but they weren't happy about the effort involved :-)
Comment: At the risk of sounding like a grumpy old musician (I'm at least 2 out of 3 there :-) ),  electric guitars are really nothing more than frequency generators. The overtone series can be manipulated from here to Orion's Belt with a few simple electric circuits (see 'stompbox' and 'effects processor').  The source overtone series matters rather less than most people would like to believe.
Comment: That is also very true Carl, the effort is a factor i hadn't even mentioned.
Answer: The bridge is the key point of connection between the strings and the body, but the tailpiece is an essential part of this. The main effect of upgrading this will be to change sustain and tone.
You may also notice an improvement in tuning stability - if you are upgrading from a poor quality bridge/tailpiece.
The actual effect varies a lot between an acoustic, semi-acoustic and solid bodied guitar, and also between through-neck and bolt-on guitars, however unless there is a specific sound you are after, I wouldn't see this upgrade as that useful - it is very minor compared to most of the other modifications you could make.
My suggestion: don't do this unless you have a broken or poor quality tailpiece, or want to change the finish. You'd be better off saving for another guitar if you want a different sound.
Comment: Thanks. I picked up a cheap, beaten-up guitar from a friend that I'm using as a project guitar to get some practice making some less-than-routine modifications (already had to ream out the holes on the headstock to install better tuners, and I'll probably have to sand and refinish the body). I've been upgrading the hardware to make it sound/play a bit better, but I didn't want to dump *too* much money into it. I figured a new nut, tuning machines, and bridge would do plenty to brighten up the tone and give it some sustain (it sounds "muddy" right now), I just wasn't sure about the tailpiece.


Excel parse produces 40802 for date 16-Sep-11

Question: I found a PHP script that parses excel files, but there seems to be some kind of numbering system or encoding that is associated with dates.  For example, the original .xls  spreadseets contains this date in this format:
16-Sep-11

However, the information that is parsed by the script shows the following integer to represent this date:
40802

If someone could help me by identifying what representation is being used for the date, then I think I can figure out how to do the conversion.
Comment: Number of days since jan 1 1900.  Or 1904 if you've set that option.
Answer: Depending on the properties of the file in question, Excel uses one of two incompatible formats to store dates, the "1900" system and the "1904" system. By default, a file that originated on Windows will use the "1900" system, while a file that originated on a Mac will use the "1904" system.  In the 1900 system, the number stored is the number of days elapsed since 1 Jan 1900, while in the 1904 system, the number stored is the number of days elapsed since 1 Jan 1904. There are various historical reasons (or is that hysterical raisins?) why Excel uses the two formats. Times are also stored in a similar manner, with the integer part being the date and the fractional part being the portion of the day that has passed.
Here is a Microsoft Knowlegebase article with more information: http://support.microsoft.com/kb/214330
Answer: Excel dates are based on the number of days from Jan 1, 1900. This page provides good detailed information on the quesiton.


Can I authenticate a user in react without making a backend api?

Question: I am building an app for a single user, and i don't want to build backend for it. How can I protect a route with help of authentication using state?
i dont want it to be secure. wondering if there is any way to prevent a route from loading based on a username and password
Comment: You cannot. Anything on the front end can be directly modified.
Comment: You can use a flag variable and can decide what you want to render and what you don't want but this is not how authentication works. For proper authentication you have to use some backend API.
Answer: Simply, you can't.
Authentication requires a backend API because it initiates what is called a session between the user and the API they are trying to access.
The frontend can always be manipulated, even if you code it in react and the people that use your website get some jibberish code, they can still manipulate it and change the state.
What you might be trying to do is protecting the website with a password. That, you can do, but as I mentioned above, it can be manipulated and people with a bit of knowledge can easily get the password from the state or wherever you save it.
Comment: i dont want it to be secure. wondering if there is any way to prevent a router from loading based on a username and password
Answer: store your user details in local or session storage while login. then re-use it whenever you want. But this is not a good way its an a temporary way.


Do I need Exchange Server for SMTP only

Question: I have a Web based application, through which i intend to send an email notifications to the users who singns in.
I have put a SMTP server on Windows 2008 R2 Standard edition for the same.
Do i need anything else apart from the SMTP server, like Lotus or Exchange server.
Again for clarification I only need to send Emails to the users and I am not suppose to recieve emails through it.
Looking for help soon.
Thanks in advance.
Answer: Nope, so long as you have an SMTP injection point, you're good to go.
There are a few things you should take into account when setting something like that up to reduce the chances of your outgoing mail getting flagged as spam (SPF, etc...). Consider SPF records and ensuring your DNS is properly configured and all that.
Comment: Hi McJeff,
Many thanks for ur reply.
Thanks for the info, this is really usefull.
A Little explanation: If I have a website and user creates an account on it, after this process, an email wud b sent to the user.The srvrs invloved in this scneraio r (web,App,DB,AD & SMTP) all r on seperate physical machines.Now, the user creates an account,this would b done with the help of WebMethods srvr which is seperate frm above mentioned srvrs n is integrated. the webmethods hav to invoke the mail request to send email via SMTP srvr.
Now,how this wud b achieved n what do i need to do on SMTP srvr.
Comment: I'm not 100% sure what you're asking? How do you get the message from code to the SMTP system?
Comment: you got it now McJeff, This is my real question, what do i need to do to enable sending emails from my SMTP server. do i need to write some commands in the code or some forwarding mechanism?
Comment: The short answer is "yes, you need to handle it in code."  Your specific framework will have a mechanism for configuring an outbound SMTP connection. That's framework dependent. Perhaps XML, perhaps YAML, perhaps in the actual method that sends the mail. You'll need to set that up so that it uses your SMTP machine as a relay point.

How? That I really don't know. Depends on your framework, version, code, and so on. If you're wondering how to do it programmatically, might be more of a stackoverflow.com question.
Answer: No, you don't need anything else. The SMTP server built into IIS can send email to the internet successfully.
Answer: Standard Windows SMTP is perfectly fine for this.  You should look at locking it down so that only the web app can use it to send, and so that it doesn't accept incoming email.  I'm not certain how much of that is possible with Windows SMTP, but as much of it as you can do is advisable.

Do I need Exchange Server for SMTP only


product variation switcher

Question: Hy Drupal Masters!
My question is about commerce module.
I managed to do the followings:

create taxonomies (for creating menu and categorizing products - type - , AND for classifying products in other context - package)
create product types
create the appropiate content types to product types
creating product variations(variations based on 2 fields in content types: "type" and "package" )

For example
I have 3 menu-category with products: prod1(products of category: prod1-1, prod1-2, prod1-3), prod2(prod2-1, prod2-2, prod2-3), prod3(prod3-1, prod3-2, prod3-3)
Each product in one category has 3 variation: small, medium, large
At the moment, if I click one of the categories, the product list appear with the good products, and each product has only one variation.
(eg. I click prod1, I'll see the products, like: prod1-1small, prod1-2small, prod1-3small)
How I expected...but:
I wold like to place a selector(select list or checkboxes, I don't mind) to the top of the product list under each category that is containing small, medium, large elements. Than if I select "medium", the product list needed to change according to the selection.(I would like to see the: prod1-1medium, prod1-2medium, prod1-3medium)
In a product details page I could do this, but in a product list page, I couldn't.
I am using drupal 7
Please somebody help me!
Comment: The "type" term-reference-field is defined at content types, and the "package" term-reference-field is defined at product types, beacuse of the variations.
Answer: I have a solution for this.
First: I have just created one product type, with the field 'somethingTermReference'(term reference), wich will be the base of the production switcher.
Second: I created content types for every product category, for displaying them. Each content type has a type field with distinct values.
Third: I created views for every product category and there are 3 filter criterias for each view: published content, content type(actual product category), and one exposed filter on a product reference 'somethingTermReference' field.

product variation switcher


How to determine which MySQL database has been selected?

Question:
Possible Duplicate:
MySQL: determine which database is selected?

I'm trying to debug some problems with a project from someone else. It looks like I get data from an other database, but I'm connected to the right one.  So I thought, let's look at what database I'm connected to.
How can I look at what database I'm connected at runtime?
Answer: Run the following query:
SELECT DATABASE();

Answer: You can use mysql_get_host_info();.
This will return the hostname of the server you are connected to.
You also have mysql_get_server_info() and mysql_get_proto_info() which will return the server version number and communication protocol version number respectively.
Comment: None of this information will help determine which database is selected.  It only gives information about the MySQL server itself.
Comment: Given that your request was "what database your are CONNECTED to" I naturally assumed you meant network connection. To determine with database is SELECTED use SELECT DATABASE()
Answer: Use mysql_db_name().
Comment: `mysql_list_dbs()` is deprecated in 5.4.  Since this function relies on that you should not use this to determine the currently selected database.
Comment: You can't find out the currently selected database with this anyway, since [`mysql_list_dbs()`](http://php.net/mysql_list_dbs) does nothing but list all the available databases.
Comment: I posed the link to the function 'mysql_db_name()', I don't know why is that problem??
Comment: @luka, the only problem is that it does nothing to answer the OP's question.


Tokens in blocks configuration for List only pages

Question: I want to set a block to visible only for a certain path, with the logged in user id, like my-path/[user:uid].  I feel like I should be able to just do that.
How can I do this?
Comment: Or, if anyone can show me how to do this in preprocess or a hook, that would be great too.
Comment: First thing, you did not mention how you created the block or want to create the block. Second, by enabling php filter, you can set the path explicitly! Make sure the path 'my-path/[User:id]' exist on your site!
Comment: The block is a Superfish menu block, so I can't enable a code filter.  Also, the my-path/[User:id] does exist, but when the user is logged in, the path becomes my-path, so I need to cruch some logic to figure out if I can display my block.
Comment: detail blog: http://goo.gl/77zEds
Answer: 1) Enable the core PHP filter module
2) Use php in the visibility setting of your block. Notice that the path 'mypath' must be registered somehow before using this code. I mean that if the path 'mypath' does not exist at all any code here will not work.
<?php
global $user;
$uid = $user->uid; // Get the logged in user id
// If the path is 'mypath/myarg' then arg(0) is 'mypath' and arg(1) is 'myarg'.
$show = (arg(0) == 'mypath') && (arg(1) == $uid); // It must return both TRUE

return $show;
?>

Alternative, use the Context module.
Comment: I'm trying not to use the PHP filter.  Any ideas through a module or preprocess?
Comment: Why not use php? Template preprocess are using php also. If this block is made with views module you should use a user id argument for the block and then just use a path permission of 'mypath/*'.
Comment: PHP isn't the problem.  I shouldn't have to grab args off the URL for something that _should_ be simple.  I feel like there is a more appropriate way to do this.
Answer:
Add contextual filter: Global: Null
provide default value: raw value from URL
Path component: 1
Add contextual filter: User: Uid
When the filter value IS available or a default is provided
Specify validation criteria -> Validator -> user

Path validation (contextual filter: Global: Null)

You can validate the first path component in two ways:
#1
use contextual filter settings:
When the filter value IS available or a default is provided:
Specify validation criteria -> Validator -> PHP Code
return ($argument == 'my-path') ? TRUE : FALSE;

#2
Alternatively, if you don't want to use PHP code validation in the contextual filter settings, you can use preprocess function in your template.php file. Not sure whether it's the right way or not, but works:
/**
 * Preprocess the primary theme implementation for a view.
 */
function YOURTHEME_preprocess_views_view(&$vars) {
  $view = $vars['view'];
  switch ($view->name) {
    case 'YOUR-VIEW-NAME':
      if ($view->args[0] != 'my-path') $view->result = array();
      break;
  }
}

Assuming 'my-path' it's the View or some other module custom path, in this case it has to be validated with the php. Of course if it would be an alias of some content type, then php validation wouldn't be needed.
Comment: Thanks, but it's not a view.
Answer: You can do that with using hook_block_info_alter().
For Example:
function YOURMODULE_block_info_alter(&$blocks, $theme, $code_blocks) {
  global $user;
  $blocks['THE_MODULE']['THE_BLOCK']['pages'] = my-path/$user->uid;
}

Where THE_MODULE is the module defining the block you want to alter, and THE_BLOCK is the 'delta' of this block.


How to (true by pass) or switch on off your effects pedal board BOss GT100 to get original guitar amp sound

Question: I want to be able to go directly to my guitar amp sound which is a Roland JC120 and or a Fender Bassbreaker guitar amp. I use a BOSS GT100 Pedal effect board.
This GT100 when switched correctly does give you a similar amp sound but it is not my true guitar sound.
When stage performing, what device/switch pedal should I use so I can turn off my Boss GT 100 sound and then go directly to my original guitar amp sound and then turn off the amp sound to go to back to my GT100 effects board in between a song.
I have tried several bypass switches to no avail. They give me a similar sound but not the same. Is it that our technology is not there yet and I am just spinning my wheels to get that crisp original sound from my guitar amp?
Comment: I have tried to edit for clarity, but still am not sure what you are asking - a bypass will not give you any 'sound' - it simply switches between the effect and clean. So if your bypass switch has GT100 on the one side and your amp on the other, you will get whatever sound your amp gives when you switch directly to it. Can you please edit your question to explain what your problem is.
Comment: @DrMayhem I think he's saying that the 'clean' mode of the bypass still doesn't sound as good as going straight in to the amp - is that right Eduardo?
Comment: That's what I'm trying to understand. If that is the case, the answer is simple - buy a true bypass pedal :-)
Answer: What you require is an A B true bypass pedal
Comment: A true bypass pedal (or true bypass looper) and an A/B pedal are different things. Usually a true bypass looper can be used *as* an A/B pedal - but a simple A/B pedal (as illustrated) can't be used as a bypass pedal.


$(Ax,x)>0 \forall x$ implies $A$ selfadjoint?

Question: I read in Reed & Simons's Functional Analysis (Vol.1, pg. 194) that a positive operator, i.e, an operator $A$ such that $(Ax,x)>0 \forall x$ on a complex Hilbert space is selfadjoint, but this doesn't hold for real Hilbert spaces. The proof uses the fact that if $(Ax,x)$ is real for all x then $(Ax,x)=(x,Ax)$ for all x and then they use the polarization identity and say that in the complex case we can recover $(Ax,y)$ by knowing $(Ax,x)$ for all $x$, but that this is not possible in the real case.
I actually don't understand how can we recover $(Ax,y)$ by knowing $(Ax,x)$ for all $x$ nor why this holds only in the complex case. Can anyone explain this to me?
Comment: Consider the case with $x+y$ and $x+iy$ you will recover $(Ax,y)$. Without $i$ we will lose $1$ condition.
Comment: Note that in the real case the statement is wrong. See e.g. $A=\begin{pmatrix} 1 & 1 \\ -1 & 1\end{pmatrix}$. Then $x^TAx = \|x\|^2$, but $A$ is not symmetric.
Answer: In the complex case:
$(A(x+y),x+y)=(Ax,x)+(Ay,y)+(Ay,x)+(Ax,y)$
$(A(x+iy),x+iy)=(Ax,x)+(Ay,y)+i(Ay,x)-i(Ax,y)$
Then you have two equations for two variables $(Ax,y)$ and $(Ay,x)$
In the real case, you will lose 1 equation because there is no $i$.
Comment: Ok yes! Now everything is clear :) thanks!
Answer: If $A$ is self-adjoint (to include the real case) then since you know $(Az,z)$ for all $z$ then you compute:
$$(A(x+y),x+y)=(Ax,x)+(Ay,y)+2(Ax,y),$$ from where
$$(Ax,y)=\frac{(A(x+y),x+y)-(Ax,x)-(Ay,y)}{2}.$$
Alternatively,
$$(A(x+y),x+y)=(Ax,x)+(Ay,y)+2(Ax,y),$$ and
$$(A(x-y),x-y)=(Ax,x)+(Ay,y)-2(Ax,y),$$ from where
$$(Ax,y)=\frac{(A(x+y),x+y)-(A(x-y),x-y)}{4}.$$
Comment: This means that the result is also valid in the real case, isn't it? Is Reed & Simons wrong in this case?
Comment: He just assume that $A$ is self-adjoint, which is not true in general. See my post.
Comment: Note that, as @Mr.T has said, I am assuming self-adjointness. In other case, the result doesn't hold.
Answer: Let $\{x_k\}_{k \in \alpha}$ be an orthonormal basis for $H$.  Consider setting $x = e_k$, $x = e_j + e_k$, and $x = e_j + ie_k$.
This allows us to go from $(Ax,x) = (x,Ax)$ to $(A e_j,e_k) = (e_j,A e_k)$ for all $j,k \in \alpha$, so that $(A x,y) = (x,A y)$ for all $x,y \in H$.
As for the real case: consider the situation in which $A^* = -A$.


Polynomial of $11^{th}$ degree

Question: Let $f(x)$ be a polynomial of degree $11$ such that $f(x)=\frac{1}{x+1}$,for $x=0,1,2,3.......,11$.Then what is the value of $f(12)?$
My attempt at this is:
Let $f(x)=a_0+a_1x+a_2x^2+a_3x^3+......+a_{11}x^{11}$
$f(0)=\frac{1}{0+1}=1=a_0$
$f(1)=\frac{1}{1+1}=\frac{1}{2}=a_0+a_1+a_2+a_3+......+a_{11} $
$f(2)=\frac{1}{2+1}=\frac{1}{3}=a_0+2a_1+4a_2+8a_3+......+2^{11}a_{11} $
.
.
.
$f(11)=\frac{1}{11+1}=\frac{1}{12}=a_0+11a_1+11^2a_2+11^3a_3+......+11^{11}a_{11} $
for calculating $f(12)$, I need to calculate $a_0,a_1,a_2,....,a_11$ but I could solve further.Is my approach right,how can I solve further or there is another right way to solve it.
$(A)\frac{1}{13}$
 $(B)\frac{1}{12}$
 $(C)0 $
$(D)\frac{1}{7}$
which one is correct answer?
Answer: HINT:
Let $(x+1)f(x)=1+A\prod_{r=0}^{11}(x-r)$ where $A$ is an arbitrary constant
Comment: A is not an arbitrary constant. It is a fixed constant whose value we happen not to know.
Comment: But f is a fixed, given polynomial. Your equation can only hold for exactly one scalar A.
Comment: @MarianoSuárez-Alvarez, See any arbitrary finite constant will satisfy the given condition, right?
Comment: Solving for $f(x)$ from your equation will in general yield a *rational function* - there is a unique choice of $A$ for which it simplifies to a polynomial. Can you find that value for $A$?
Comment: @MarianoSuárez-Alvarez, I was pointing to the given condition. But, from that  $A$ can assume any arbitrary finite scalar value, right?
Comment: @anon, What's that unique choice?
Comment: Hint: The RHS is divisible by $x+1\iff $ RHS $=0$ when $x=-1$. :-)
Comment: what should be the f(12), 0 or 1/12?
Comment: I think I get it, we need 1+x to divide the RHS, We can do that by evaluating the A*product term with x=-1 and then figuring out what value A needs to be so the RHS vanishes, I make that A=-1/12! . Now, finale, f(12) would be 0 - choice C.  Any confirmation ? Good one!
Answer: Somewhat less vague...
$(x+1)f(x)-1$ is a polynomial of degree 12 with roots at every integer in $[0,11]$, so could be $$(x+1)f(x)-1 = A \prod_{c=0}^{11} x-c$$
for some/any (nonzero) constant $A$.
When $x=-1$, we have $(0)f(-1)-1 = A (-1)^{12} 12!$, or $-1 = A \, 12!$ and discover only $A = \frac{-1}{12!}$ is consistent with the givens.  (Why $-1$?  Because it is the only choice we haven't already used (we have used the integers 0, ... 11) that makes some expression containing $x$s zero.)
Hence, $13 f(12) - 1 = \frac{-1}{12!} 12!$ and $f(12) = \frac{-1+1}{13} = 0$.
Answer: For convenience, let us shift the variable: $g(y)=\dfrac1y$ for $y=1,2,\cdots12$.
Then $$\frac{A(y-1)(y-2)\cdots(y-12)+1}y$$ is a polynomial of degree $11$ and coincides with $\dfrac1y$ at the given points, provided that the numerator has no independent term, i.e. $12!A+1=0$.
From this,
$$f(12)=g(13)=\frac{A(13-1)(13-2)\cdots(13-12)+1}{13}=\frac{12!A+1}{13}=0.$$

Polynomial of $11^{th}$ degree


serve all requests via proxy *except* a specific one

Question: I have the following in my httpd.conf
<VirtualHost *:80>
    ServerName foo.org
    ServerAlias www.foo.org

    <Proxy *>
        Options FollowSymLinks MultiViews
        Order allow,deny
        Allow from all
        AllowOverride All
    </Proxy>

    ProxyPass        / http://127.0.0.1:5012/
    ProxyPassReverse / http://127.0.0.1:5012/
</VirtualHost>

This works well as all requests for http://foo.org get served from port 5012 using a different web server. However, I want to serve just http://foo.org/lib via Apache from port 80. How do I do that? Adding the following lines to the conf doesn't help
Alias /lib /path/to/lib

<Directory "/path/to/lib">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

Answer: You can avoid proxying for a specific location by telling mod_proxy to ignore it with a ! for a destination:
ProxyPass        /lib !
ProxyPass        / http://127.0.0.1:5012/
ProxyPassReverse / http://127.0.0.1:5012/

Comment: awesome! it was that easy! I tried `NoProxy` and `ProxyMatch with various regexps` to no avail. Thanks.


Filtering exposed filters with views_get_view_result() function

Question: I have built a module so that I can display PDFs with phpwkhtmltopdf. I am currently calling my view data using:
$view = views_get_view_result('course_catalog');

My question is, is there a way to have filter my exposed filters with this function? If not, what is the best way to go about this? --I suppose I could manually filter through the results with PHP, but that hardly seems efficient.
Answer: It seems views_get_view_result does not allow setting filters: https://www.drupal.org/node/1428696
Instead of using views_get_view_result, you might try with other functions of the Views API:
$view = views_get_view('course_catalog');
$view->set_exposed_input($my_exposed_input);
$view->init_display();
$view->pre_execute();
$view->execute();
$result = $view->result;

You can check the documentation for the set_exposed_input method, and also this related Drupal Answers question, for alternatives: How to change programmatically exposed filters in views
Comment: As the documentation is a bit short, I'd add that `$my_exposed_input` should be of the form `array("my_filter_label_1" => some_value, etc.)`.


Overfitting in recommender systems

Question: So I want to know whether or not my models are overfitting or the difference between train and validation errors are decent.
$L$: is the number of neighbors
The first column is the train error
The second column is the validation error

Okay. Actually I got those values from 5-fold cross validation. So the column error on the right is the averaged test error over all folds and the errors on the column on the left are the averaged train error over all folds. From what i know you are supposed to know if a model is overfitting or not by adding more data to the training set? In this example all I did was to sweep the values of the $L$ neighbors.
Comment: There is something weird going on, since the validation error remains pretty much constant throughout the L's while the training error actually increases. If you were overfitting you would expect this to decrease not increase.
Comment: Still, the results are telling you that you are getting worse results on your training set if you increase L, while the validation results tell you that you are not really learning anything new if you increase L, you are getting very similar results across the L's.
Comment: What metric exactly are we looking at here?
Comment: We are looking at RMSE
Comment: Okay. Actually I got those values from 5-fold cross validation. So the column error on the right is the averaged test error over all folds and the errors on the column on the left are the averaged train error over all folds. From what i know you are supposed to know if a model is overfitting or not by adding more data to the training set? In this example all I did was to sweep the values of the L neighbors.
Comment: Yes, actually the first bold row is the model I chose, which has the lowest error
Comment: What you might wanna do is decrease $L$.
Comment: @Art What's the difference between train cross validation error and test cross validation error?
Answer: Not sure if this will answer your question, but look at it this way:
In OLS, the more regressors ("features") you put into your model, the more likely you are to overfit the model.
But in the case of k-nearest neighbors (as I assume you're doing), increasing the number of neighbors ($L$) would actually lead to underfitting (see here: Does k-NN with k=1 always implies overfitting?)
So this is why I suggested lowering $L$. You should see train error go down, and test error go (generally) up. You could then decide what your $L$ should be, based on where the errors are "balanced".
Visually, you're on the left side of this graph. You need to explore further to the right (in the direction of decreasing bias) and this could be done by lowering $L$.
Comment: Thanks for the answer! It helps me a lot! However, my question is sort of tricky because those train and validation errors were obtained from 5 fold cross validation. Whenever you apply that technique, you always get sticked to the lowest cross validation error and forget about the training one. I'm using cross validation for tunning hyperparameters purposes. Once the lowest cross validation error is obtained, throw the best parameters' combination, I retrain the model on the entire dataset and test on the test set. From there on, I think I could apply your approach.

Overfitting in recommender systems


Remove extensive whitespace with a new line

Question: Currently my app saves a workout log like so.
Start time: 8:49:41 PM                                                     Objective: 17 pushups a minute for 1 hour                                                                      Total: 1020 pushups                                                     59 minutes have gone by                                                     0 pushups remaining                                                     0 minutes remaining                                                     Times Paused: 0                                                     Finished at: 9:49:41 PM

I'm trying to remove the extensive whitespace and new lines by using....
string.trim().replace(/[\s]/g," ").toString()

The objective is to save the file removing the extensive whitespace with a new line so it's easy to read like so...
Start time: 8:49:41 PM
Objective: 17 pushups a minute for 1 hour
Total: 1020 pushups
59 minutes have gone by
0 pushups remaining
0 minutes remaining
Times Paused: 0
Finished at: 9:49:41 PM

Thus my question is simple.
How can I remove this extensive whitespace with a new line?

var string = output.textContent.toString();

console.log(string.trim().replace(/[\s]/g," ").toString())

                  Start time: 8:49:41 PM

                  Objective: 17 pushups a minute for 1 hour

                  Total: 1020 pushups

                  59 minutes have gone by

                  pushups remaining

                  0 minutes remaining

                  Times Paused: 0

                  Finished at: 9:49:41 PM

Problem solved, Thanks to Emma's wonderful help...
string.trim().replace(/\s{2,}/gm,"\n").toString()

Answer: Maybe, \s{2,} might work, or if not, you can try \s{3,} or \s{4,} being replaced with a new line:

const regex = /\s{2,}/gm;
const str = `Start time: 8:49:41 PM                                                     Objective: 17 pushups a minute for 1 hour                                                                      Total: 1020 pushups                                                     59 minutes have gone by                                                     0 pushups remaining                                                     0 minutes remaining                                                     Times Paused: 0                                                     Finished at: 9:49:41 PM
`;
const subst = `\n`;

// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);

console.log(result);

If you wish to explore/simplify/modify the expression, it's been
  explained on the top right panel of
  regex101.com. If you'd like, you
  can also watch in this
  link, how it would match
  against some sample inputs.
Answer: Without crazy regular expressions. A split, map, trim, and filter

var string = output.textContent.split(/\n+/).map(s => s.trim()).filter(s => s).join("\n")
console.log(string)

                  Start time: 8:49:41 PM

                  Objective: 17 pushups a minute for 1 hour

                  Total: 1020 pushups

                  59 minutes have gone by

                  pushups remaining

                  0 minutes remaining

                  Times Paused: 0

                  Finished at: 9:49:41 PM

A reg exp

var string = output.textContent.trim().replace(/\s*\n\s*/g, '\n').replace(/\n+/,'\n')
console.log(string)

                  Start time: 8:49:41 PM

                  Objective: 17 pushups a minute for 1 hour

                  Total: 1020 pushups

                  59 minutes have gone by

                  pushups remaining

                  0 minutes remaining

                  Times Paused: 0

                  Finished at: 9:49:41 PM


How to improve Left Hand Discipline when drumming (right Handedly)

Question: I am right cross handed drummer, this will give you an idea of the drum layout (Hi-Hat on the left). Now I have been trying to swap hands to Left open handed and I have realized an obvious lapse in stick control and technique, are there any exercises that I can carry out that:
A: Brings my Left hand technique up to scratch with my right hand, (allowing similar speed and muscle memory.)
B: Does not allow my right hand to slip out of configuration (for want of a better word)
I have a practice pad which I use occasionally but I was wondering if there are specific rudiments or techniques that hone in on the left or weak hand discipline?
Answer: In my opinion, only playing rudiments is not the complete answer. The path to a stronger left hand is manifold. Rudiments are the possible combinations of sticking on the drumset, yes. But it's a repetition exercise. What I mean is that you can repeat something for a while with improper technique and be able to play the rudiment perfectly and still have a weaker hand.
A couple of years ago, I switched to a fully ambidextrous drumset (meaning I can lead both as a full lefty or righty). When I did, I noticed a huge difference in the way I held the stick between my right and left hand. The difference was especially apparent when I played doubles. It made sense since playing when we cross hands, we treat both hands differently in most scenarios. So that's the first thing I changed : I played those rudiments while paying attention to the left hand, adjusting my grip and the way I played to mirror my right.
The two most important exercises I used to do to improve (and still do) were quite a bit more simple than rudiments. The first is just playing linear patterns between the hands and the feet (like both hands, both feet, right and left foot, left hand and left foot, etc.). This will practice your stamina which the left side always lack as well as making it a coordination exercise by putting it in relation to your other members. The other exercise is just playing unison of the same combinations. Again, you will notice and feel a HUGE difference between your left side and right side (the left side will start to flam very quickly). Practice at different levels of volume for long periods. And of course, start slow!
The last step is quite dramatic but optional : play left handed a bit! Leading with your left will confuse your brain and again, will make you more aware of what you are doing. It will also train your left hand to move around the drumset. This is important because while rudiments will help you to move your left hand, they are usually done on a single surface (the snare drum) which has a very different feel than other drums or cymbals. By playing on those other surfaces, you will train slightly different groups of muscles (especially in your hand) that aren't as strong as your right. Also, listen carefully to the sound produced by your left hand (and foot).
Needless to say that it is a pretty long journey : most of us start playing cross handed and never really question this way of playing until we are confronted with those problems or situation. And when they happen, we must catch on to all those years of playing that way. But I tell you, after a while, you will feel that much more confident and free in your playing. It's all worth it!
Cheers and good luck!
Answer: I think achieving full symmetry is really hard because for most people there are differences between their right and left hands, both physically and mentally. That being said, my best results for keeping both hands at similar technical levels has definitely been working through the classic rudiments and alternating which is the "lead" hand in the rudiment. If your left hand is really far behind, then isolation exercises have helped me with that kind of thing in the past. Try playing rudiments with the left hand only, either playing both parts with the left hand or playing only the left hand part and treating the right hand as rests. If it sounds hard then that's a good thing in terms of training.
Comment: Nice thanks for the tips :) could you add a couple of pointers for me, such as which rudiments I should start with (difficulty for intermediate) or even a list of them to research and figure out how to perform?
Comment: http://www.pas.org/pdf/rudiment.pdf
I would practice these and switch up the R and L.
Comment: Don't hide behind being right or left-handed.  A "right-handed" guitarist does all the dextrous stuff with his left hand (I've never seen the point of playing left-handed guitar?) as does a violinist.  Just practice the usual rudiments and exercises using your new configuration.  It's different and unfamiliar.  What did you expect?
Comment: @LaurencePayne Does your comment make more sense for the question/OP as opposed to being a comment on this answer? As a person who plays all three instruments, I would say comparing handedness on drums to handedness on guitar is not as good a comparison as with piano.
Comment: Eh?  Is this some arcane point of netiquette (which seems to be an obsession round here - binary answews are so BORING :-)
Piano is such an obvious example, I left it alone.  Anyway, my point was that "right-handed" musicians often do much HARDER stuff with their "wrong" hand.  "Handedness" doesn't need to be an issue.
Comment: My answer says nothing about handedness and the original question does talk about handedness, which is the only reason why I thought your comments seemed to be about the question and not my answer. Think of it this way: would your comments still apply to this question if my answer were deleted?
Comment: @LaurencePayne do you play drums at all? drums are a lot different in technique to 99% of other instruments... you speak as if I am unaware of how my left hand is performing... I have trailed and tested experience of my left hand being weaker. I asked for techniques to strengthen it and Todd give me a perfectly reasonable answer...
Comment: Good.  So practice rudiments the same way you should ALWAYS have been practicing them, leading with either hand.

Todd - yes I commented rather than replied. That's how it works, doesn't it?
Comment: I was suggesting that you comment on the **question** instead of commenting on my **answer**. I think commenting was the right choice, just where you put the comment seemed a little off to me.
Comment: @LaurencePayne the comments you have made say the exact same thing is what the answer does... plus not everyone practices the same way there is no right or wrong way to practice music you have a very narrow minded view if you think everyone has to stick to a certain set of practice tasks...
Answer: I think all drumers with a dominant hand experiences this.
The way I've been best able to get my hands even were 3 steps (for me anyway )
First, play all your rudiments with a click. On all 3 go for longer periods of time, but make sure everything is clean before you crank up the speed.
2nd I'll play 16th noted 4 bars of 16ths 9n each hand, and throw 8n a flam before switching.
3rd (what helped me most) match your 16ths.  Not flams, but match em. The weaker hand will begin to play what the dominant hand is playing (technique wise) steadily speed up once you have control (control is key)
After that, there are pleanty of exercises to move em to different voices. Do this same thing with your feet. If you can do something with your dominant hand, or foot, make sure you can do it with all limbs.
Hope this helps. Try it, you will improve much faster than ya think