Statistical inference under model misspecification

Question: I have a general methodological question. It might have been answered before, but I am not able to locate the relevant thread. I will appreciate pointers to possible duplicates.
(Here is an excellent one, but with no answer. This is also similar in spirit, even with an answer, but the latter is too specific from my perspective. This is also close, discovered after posting the question.)

The theme is, how to do valid statistical inference when the model formulated before seeing the data fails to adequately describe the data generating process. The question is very general, but I will offer a particular example to illustrate the point. However, I expect the answers to focus on the general methodological question rather than nitpicking on the details of the particular example.

Consider a concrete example: in a time series setting, I assume the data generating process to be
$$
y_t=\beta_0 + \beta_1 x_t+u_t \tag{1}
$$
with $u_t \sim i.i.N(0,\sigma_u^2)$. I aim to test the subject-matter hypothesis that $\frac{dy}{dx}=1$. I cast this in terms of model $(1)$ to obtain a workable statistical counterpart of my subject-matter hypothesis, and this is
$$
H_0\colon \ \beta_1=1.
$$
So far, so good. But when I observe the data, I discover that the model does not adequately describe the data. Let us say, there is a linear trend, so that the true data generating process is
$$
y_t=\gamma_0 + \gamma_1 x_t+\gamma_2 t + v_t \tag{2}
$$
with $v_t \sim i.i.N(0,\sigma_v^2)$.
How can I do valid statistical inference on my subject-matter hypothesis $\frac{dy}{dx}=1$?

If I use the original model, its assumptions are violated and the estimator of $\beta_1$ does not have the nice distribution it otherwise would. Therefore, I cannot test the hypothesis using the $t$-test.
If, having seen the data, I switch from model $(1)$ to $(2)$ and change my statistical hypothesis from $H_0\colon \ \beta_1=1$ to $H'_0\colon \ \gamma_1=1$, model assumptions are satisfied and I get a well-behaved estimator of $\gamma_1$ and can test $H'_0$ with no difficulty using the $t$-test.
However, the switch from $(1)$ to $(2)$ is informed by the data set on which I wish to test the hypothesis. This makes the estimator distribution (and thus also inference) conditional on the change in the underlying model, which is due to the observed data. Clearly, the introduction of such conditioning is not satisfactory.

Is there a good way out? (If not frequentist, then maybe some Bayesian alternative?)
Comment: Your discomfort is endemic to classic approaches to awarding PhDs: careful hypothesis specification, followed by an empirical test and ending with descriptive causal inference. In this world, the short answer is, "no," there is no way out. However, the world is evolving away from that strict paradigm. For instance, in a paper in the *AER* last year titled *Prediction Policy Problems* by Kleinberg, et al, they make the case for data mining and prediction as useful tools in economic policy making, citing instances where "causal inference is not central, or even necessary." It's worth a look.
Comment: If I understand correctly, you are collecting data, then selecting a model and then testing hypotheses. I may be wrong, but it seems to me that the [selective inference](http://statweb.stanford.edu/~tibs/ftp/nips2015.pdf) paradigm investigated by Taylor and Tibshirani (among others) could be related to your problem. Otherwise, comments, answers and linked answers  [to this question](http://stats.stackexchange.com/questions/254314/is-mle-estimation-asymptotically-normal-efficient-even-if-the-model-is-not-tru) might be of interest.
Comment: @DeltaIV, that is, when doing inference, I am not interested in the *least false* parameters as under P-consistency, but rather I am interested in the *true* ones (the true partial derivative of $y$ w.r.t. $x$).
Comment: @RichardHardy, sure, despite being a stats grad student I don't really believe in inference anymore. It's a house of cards so fragile that it's unclear whether it's meaningful at all except in very strict and controlled circumstances. What is funny is that everyone knows this, but no one (well) cares.
Comment: In my view, the direct answer would have to be there is no way out. Otherwise, you would be guilty of the worst sort of data mining -- recasting the hypotheses to fit the data -- a capital offence in a strict, paradigmatic world.
Comment: @DeltaIV, no, of course you did not say it was a duplicate. I said that as a preemptive strike :) But your links are really interesting and helpful! And I am well aware of Gelman, he really helped me understand statistical methodology better (although there is still a looong way to go!).
Comment: Leeb and Pötscher have studied this extensively. The distribution of the estimator is typically highly non-standard and you are perfectly right that inference is usually highly flawed because of this. This applies to any model selection procedure, be it AIC, OLS post lasso, pretesting etc. There was a paper in the Annals of Statistics in 2013 by Berk etc although which supposedly allows for valid inference. If you want to search further, just google post-selection inference. Hjort and Claeskens 2003 model averaging paper in JASA is a good read too.
Comment: @hejseb, thanks! This issue is incredibly disturbing when I think about causal inference in economics (except, I guess, for experimental economics)... Looks like we could scrape most of the published results. Painful...
Comment: @DJohnson, Do I understand you correctly that the direct answer would be, there is no way out? If model $(1)$ does not happen to be the true data generating process but model $(2)$ is, we will not be able to test the subject-matter hypothesis using the data set at hand. Is that right?
Comment: @DJohnson, In terms of inference, I think this is not the worst but perhaps the lightest sort of data mining. I could, for example, test whether $\gamma_2=0$ and make it support my theory in some creative way :) That would probably be the worst. But that is beyong the point.
Comment: Good question. Have you heard of "indirect inference"? Sounds very similar to what you're describing http://bactra.org/notebooks/indirect-inference.html
Comment: @Delta, you understood me correctly. But basically it is not "my method", it is how 90% (?) of research in economics gets done (almost everything except for experimental economics)... which makes me strongly doubt the validity of any and all of their inference... Looking at your link, I see that there the question and the answers address something like [P-consistency rather than T-consistency](http://stats.stackexchange.com/questions/265739/t-consistency-vs-p-consistency), which I am not interested in for this question here. So the link is interesting, but the questions are not duplicates.
Comment: I never said that your question was a duplicate of mine, just that it might help, but evidently it doesn't. What about the other link, the pitch by Rob Tibshirani? On page 21, he explicitly mentions being able to control *selective type I error*, a sort of type I error when the hypothesis being tested is random (in the sense that it depends on the sample data). This seems to me more related to T-consistency than P-consistency (in your terminology), as no loss function is assumed in the definition of selective type I error. But, again, I am no expert in this kind of stuff and I may be wrong.
Comment: ps just one last comment on the unreliability of research done the way you described. You probably know it already, but Gelman's [garden of forking paths](http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf) does come to mind.
Comment: You write "This makes the estimator distribution (and thus also inference) conditional on the change in the underlying model, which is due to the observed data. Clearly, the introduction of such conditioning is not satisfactory.". Why the last sentence?
Comment: @AlecosPapadopoulos, These two sentences are sloppy as I still have not found a good way to express myself (evidently I do not understand the phenomenon perfectly). What I mean is that we are interested in inference as stated at the beginning, not some modification of it.
Comment: @DJohnson Relevant lit differentiating **prediction and explanation**:$$\phantom{0}$$ Prediction, explanation and the epistemology of future studies. *Futures*, 35(10):1027–1040. $$\phantom{0}$$
Hanson, N. R. (1959). On the symmetry between explanation and prediction. The *Philosophical Review*, 68(3):349–358.$$\phantom{0}$$
Rescher, N. (1958). On prediction and explanation. *The British Journal for the Philosophy of Science*, 8(32):281–290.$$\phantom{0}$$
Scheffler, I. (1957). Explanation, prediction, and abstraction. *The British Journal for the Philosophy of Science*, 7(28):293–309.
Comment: @hejseb, hmm, probably not everyone knows this. People around you may be statistically literate, but there is some selection bias.
Comment: For what it's worth, this *AER* article *Prediction Policy Problems* by Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer specifically addresses concerns with inference vs prediction, "We argue that there are also many policy applications where causal inference is not central, or even necessary." (https://www.cs.cornell.edu/home/kleinber/aer15-prediction.pdf)
Answer: The way out is literally out of sample test, a true one. Not the one where you split sample into training and hold out like in crossvalidation, but the true prediction. This works very well in natural sciences. In fact it's the only way it works. You build a theory on some data, then you're expected to come up with a prediction of something that was not observed yet. Obviously, this doesn't work in most social (so called) sciences such as economics.
In the industry this works as in sciences. For instance, if the trading algorithm doesn't work, you're going to lose money, eventually, and then you abandon it. Cross validation and training data sets are used extensively in development and making a decision to deploy the algorithm, but after it's in production it's all about making money or losing. Very simple out of sample test.
Comment: I sympathize with your view. But since sample splitting into "old" and "new" is equivalent to collecting new data, I do not understand where you see a big difference between the two.
Comment: Does that help estimate $\frac{\partial y}{\partial x}$?
Comment: @RichardHardy, yes, you test the same hypothesis on the new data. If it holds then you're good. If your model is misspecifyied then it should eventually fail, I mean other diagnostics too. You should see that the model is not working with new data.
Comment: OK, then it sounds like the good old prescription of splitting the sample into a subsample for model building and another for hypothesis testing. I should have included that consideration already in the OP. In any case, that seems like a sound strategy. The problem with macroeconomics, for example, would be that the same model would almost never fit unseen data well (as the data generating process is changing over time), so the exact same problem that we begin with would persist. But that is an example where basically any method fails, so it is not a fair criticism.
Comment: Meanwhile, in microeconomics in cross-sectional data setting it could work. +1 for now. On the other hand, once a model has been fit to all available data, this solution will not work. I guess that is what I was thinking when I wrote the question, and I am looking for answers that address the title question: inference from misspecified model.
Comment: @RichardHardy, no splitting the sample into pieces is not the same, because then you end up using out of sample in a very similar way to in-sample. I'm saying that you do best you could do with what you have, then make a prediction out of sample and collect new data. That's the only way
Comment: I did not mean the kind of splitting you mention. It is easy to realize that you can split the sample in exactly the same way as if there was "old" and "new" data. That is probably the only *real* way out as a macroeconomist cannot wait another 5 years to collect another 20 quarterly data points if a decision has to be made at the current moment. Also, in other branches of economics it may rarely be realistic that a researcher will go out and gather more data. Thus sample splitting into "old" and "new" data achieves the goal, without the extra complications of gathering new data.
Comment: @RichardHardy, in social sciences sample splitting is the way we go, because we can't do anything else, but it's not the way out. That's why economics is stuck where it is now without a single solid theory, a *soft* science. For instance, there is not a single fundamental constant, such as electron charge in physics, in the whole field of economics. In the industry (or applied economics) it's a little better because we can often collect new data, such as sales or income of other stores.
Answer: You could define a "combined procedure" and investigate its characteristics. Let's say you start from a simple model and allow for one, two or three more complex (or nonparametric) models to be fitted in case that the simple model doesn't fit. You need to specify a formal rule according to which you decide not to fit the simple model but one of the others (and which one). You also need to have tests for your hypothesis of interest to be applied under all the involved models (parametric or nonparametric).
With such a setup you can simulate the characteristics, i.e., with what percentage your null hypothesis is finally rejected in case it is true, and in case of several deviations  of interest. Also you can simulate from all involved models, and look at things such as conditional level and conditional power given that data came from model X, Y, or Z, or given that the model misspecification test procedure selected model X, Y, or Z.
You may find that model selection doesn't do much harm in the sense that the achieved level is still very close to the level you were after, and the power is OK if not excellent. Or you may find that data-dependent model selection really screws things up; it'll depend on the details (if your model selection procedure is very reliable, chances are level and power won't be affected very strongly).
Now this isn't quite the same as specifying one model and then looking at the data and deciding "oh, I need another", but it's probably as close as you can get to investigating what would be the characteristics of such an approach. It's not trivial because you need to make a number of choices to get this going.
General remark: I think it is misleading to classify applied statistical methodology binarily into "valid" and "invalid". Nothing is ever 100% valid because model assumptions never hold precisely in practice. On the other hand, although you may find valid (!) reasons for calling something "invalid", if one investigates the characteristics of the supposedly invalid approach in depth, one may find out that it still works fairly well.
Comment: I wonder if this is realistic in practice aside from the simplest of problems. Computational cost of simulations would quickly exceed our capabilities in most of the cases, don't you think so? Your comment on validity is of course logical. However, without this simple yet useful (in aiding our reasoning) notion we would be even more lost than we are with it - that is my perspective.
Comment: I'm not saying that this should e done every time such a situation is met in practice. It's rather a research project; however one take away message is that in my opinion, for the reasons given, data dependent model selection doesn't exactly invalidate inference that would have been valid otherwise. Such combined procedures may work rather well in many situations, although this is currently not properly investigated.
Comment: I guess if this was feasible, it would already be in use. The main problem might be infeasibility due to the large amount of modelling choices that are data dependent (back to my first comment). Or do you not see a problem there?
Comment: There's the odd simulation in the literature exploring misspecification test/model selection first and then parametric inference conditional on the outcome of that. Results are mixed as far as I know. A "classical" example is here: https://www.tandfonline.com/doi/abs/10.1080/00949657808810243?journalCode=gscs20
Comment: But you're right; modelling the full process with all kinds of possible modelling options would require lots of choices. I still think it'd be a worthwhile project, although not something that one could demand whenever models are selected from the same data to which they're fitted. Aris Spanos by the way argues against the idea that misspecification testing or model check on the data makes inference invalid. https://onlinelibrary.wiley.com/doi/abs/10.1111/joes.12200
Comment: To which [I respond](https://stats.stackexchange.com/questions/303887/effects-of-model-selection-and-misspecification-testing-on-inference-probabilis): I do not understand the argument. A commenter adds: *this stuff reads mostly like expounding philosophical principles and then jumping to conclusions (for which there is no empirical or mathematical proof) by analogies and hand-waving*.
Comment: I agree that Spanos denies that there is a problem where both of us think there is one. I think something like what I proposed would be needed to investigate how big the problem actually is. Apart from his general argument, Spanos makes a few remarks why the effect of model misspecification can often be expected to be low and is non-existing in some situations. Actually you may think it is somewhat schizophrenic that Spanos argues both a) that model misspecification isn't a problem for conceptual/philosophical reasons and b) that it isn't a big problem, conceptual/philosophical reasons aside.
Comment: I find his writings amusing and have a lot of respect for him, but I really do have a problem understanding the point we are discussing now. Personal communication has not helped all that much as his response to my request used roughly the same arguments and similar wording to the cited texts. I think  first of all, we are just having some communication problems.


Problems with holding Barre Chords

Question: I'm a bass player, and I just started to learn to play guitar.
I decided to learn barre chords because I find it the most intuitive.
However, I  have a few problems with holding the chords.
First problem is that my 2nd and/or 3rd finger goes over the D strings, unintentionally muting it.
I try solving it by bending my 2nd and 3rd fingers more and pressing harder. But this causes my first finger to bend and I can't press down on the strings.
I can't seem to play without some form of muting or buzzing.
I also do not know where to position my thumb. Some player seem to hold their thumb almost parallel to the neck, while some hold it perpendicular.
Any advice will be helpful. Thanks.
Comment: I would learn open chords first and just work on F until you have it, THEN move on. Starting with barre chords seems like a bad move.
Comment: Which barre shapes in particular? There are several, and it's not clear from the wording.
Comment: I hold my thumb between where I barre the chord and the other fingers.
Comment: There is a ton of coverage for this on YouTube, which has the advantage of visual communication. Ben Eller's style may or may not be to your liking, but he has a lot of goodness on hand positioning and such: https://www.youtube.com/watch?v=DrlF4Tc8qC8 .
Answer: You're not being overly specific (could you perhaps post a photo of your left hand in some of the problematic fingerings, preferably as viwed from the top? That could make it easy for others to point out what's wrong.), so I can just give some general advice:

I think that starting with barre's is a poor idea. Try to get the left-hand position right on some simple 2- or 3-finger shapes with lots of empty strings. It's possible even to accompany a whole lot of songs with them (since all songs are in fact Pachelbel's canon in D :—)).
(The main thing:) Whenever you put any left hand fingers onto the strings, they should be always perpendicular* to the neck. All of them. You seem to be having problems with this one, and it's definitely far easier to get this sorted out on the easy chords.
If you, by any chance, have nails on the left hand, this is the time to get rid of them (except for thumb, the thumbnail can remain however long you want), because, of course, your fingers won't be perpendicular to the neck if the nails are in the way.
Maybe the muting and buzzing you describe is caused by your index finger not pressing hard enough? (See 1, please take the time to sort out the issues one by one! It's far easier and less frustrating than trying to tackle them all at once. Barre's are a big chapter all for themselves, and not an easy one.)

*: As correctly pointed out in a comment, horizontal slanting of fingers is harmless and often done. What makes the buzzes and mutes is vertical slanting, which should be avoided.
Comment: @RockinCowboy — That is certainly true, thanks for pointing it out. I must admit that the "left hand fingers always perpendicular" is something like a classical guitar mantra which I just repeated verbatim :—).
Comment: Fretting fingers do need to be close to  "perpendicular" to the fingerboard on the floor to ceiling axis.  But in most chord formations some fingers will slant towards the headstock.  It's the slanting towards the floor that causes muting.  The fingernail trim is a good point.
Comment: In getting fingers 'perpendicular to the fingerboard', there's a great tendency to move the whole hand out, thus putting the thumb is a not so good position. Just checked my fingers, and for most chords, they're not far off 45 degrees - nowhere near perpendicular! Except for the barring finger, which I can't get anything other than flattish!
Comment: @Tim — I guess it's different for everybody. I just checked too, and my fingers are indeed quite perpendicular to the neck most of the time, let's say > 80 °. No problems with thumb position though. And the barring finger is an obvious exception :—). I think that the rule can help you out in the start even if you drop it later on.
Answer: Many beginning players find barre chords difficult to play cleanly in the beginning.  You must build enough grip strength to cleanly fret all of the barred chords.
It also helps to have a properly set up guitar with lower action and lighter strings.
Depending on the anatomy and length of your fingers, you must discover the optimal way to position your barring finger for each type barre chord.
I play some barre chord formations with my thumb perpendicular to the neck while others require me to place my thumb closer to parallel to the neck.  With my short fingers, I must rotate my wrist towards the headstock such that the outside edge of my fretting finger is contacting the strings and my thumb is slanted as well.
Any chord formation that is played in open position can also be played as a barre chord. Barre chords are basically open chord formations using the barre finger like a capo to change the pitch of the chord.  The most common barre chords are the "A shaped" and "E shaped" barre chords.  I would strive to learn those first.  Between those two chords you can cover the entire range of chords from A to G.
I find the E shaped barre chord the easiest to play.  When I play it my thumb is pointing towards the headstock more than towards the ceiling (parallel to neck).
To play the A shaped Barre chord you must form a "mini barre" with your middle finger to fret the d, g and b string.  To Play a B chord or B flat chord, you will use the A shaped barre chord if you are playing in first position.  When I play an A shaped major barre chord, my thumb is pointed towards the ceiling (perpendicular).
There are some excellent answers to describe effective ways to form the "A Shaped" Barre Chord under This Question
After you master the basic E and A shaped barre chord formations you can try the "C Shape" and the "D shape".  The A and C and D shape involve playing only 4 or 5 strings - just like the corresponding open chord.  If you find it easier - you can extend the barre finger all the way to the top of the fingerboard, but only strum the appropriate strings or slightly lift the barre finger so that the low E is muted if it is not a part of the chord.
You will find that playing a D shaped chord as a barre will end up being a 4 string version of the C shaped barre chord for the same chord.  For example if you play an F major by barring the 5th fret and playing a C shape with the non barring fingers the top four strings (e,b,g and d) are fretted in the same place (but with different fingers) as if you play an F major by barring the 3rd fret and playing a D shape with the non barring fingers.  The version that you use might depend on what you intend to do either immediately before or after playing the chord. Often the transition works better with one version or the other.
Barre chords are very useful for guitar players and it's worth putting in the practice time and strength building exercises to be able to effectively play at least the E and A shaped versions.  Keep working at it and you will find what works effectively for you.
Comment: Disagree with 'grip strength'. When one can play barre chords properly, one doesn't even need to have the thumb touching behind the neck - try it. Pulling in with the fretting arm, against the other upper arm on the guitar body is better. Squeezing the neck isn't necessary for any chord, once the fingers are in the correct position.   And, don't forget the lovely sounding G shape barre chord.
Comment: @Tim - Yes I've heard the argument that you don't need grip strength and I've tried the opposing force see saw guitar trick.  I know in the beginning for me it seemed that I just couldn't get the strings pushed down and it seemed that after gaining some grip strength I had an easier time.  But I will allow that it was better technique and not greater strength that contributed to my eventual success with barre chords.  So I edited my answer in case improvement in technique was the real reason my barre chords improved with practice.
Comment: @Tim I left out G shaped Barre chord because I can't stretch far enough to play a barre on one fret and fret the other notes three frets over.  I like the A shape and E shape because it's the barre finger and the adjoining two frets (if you count Am and A7 shape).  I can't skip over two full frets from the barre finger to play a G shape as a barre chord (even if I only fret the high and low e in addition to the barre).  Most folks I know can't either.  I am not sure I have noticed anyone use a G shape as part of a Barre Chord.
Comment: I don't use it much below playing Bb (3rd fret barre), but it's easier towards the dusty end. It's a much more pleasant sound, with a 6th between the top two strings.  I can barely stretch an octave on piano, so my fingers are probably not far short of yours, so to speak...
Comment: @Tim  Just tried the Bb 3rd fret barre G Shape and it does sound nice.  Definitely different than the 1st fret A shaped Bb barre.  It's quite stressful for me to stretch that far though.  I need to go take an ibuprofen now - I think I pulled a muscle in my fretting hand.
Comment: According to someone recently, that's not possible, as there are no muscles in the hand. Not sure about that though. Maybe it's a ligament or tendon? Or a sign more practice is needed?!

Problems with holding Barre Chords


Difference between metric spaces and metrizable spaces (Munkres)

Question: Is there a difference between a metrizable space and a metric space ? For me if $X$ is a metrizable space, then there is a metric $d$ s.t. $(X,d)$ is a metric space. And obviously a metric space is metrizable.
The reason I'm asking this question is in the book Topology second edition of J. Munkres, page 179 they prove (theorem 28.2) that if the $X$ is metrizable, then a set is compact $\iff$ it's sequentially compact. Now page 276 (Theorem 45.1) they prove that a metric space is compact $\iff$ it's complete and totally bounded.
My question : In the proof of the theorem 28.2 in the part $(3)\implies  (1)$ they in fact proved that the space is totally bounded, and then proved that it's moreover compact. But they introduced the totally boundness to prove theorem 45.1 only. I don't really understand why they introduce the concept of totally boundness that late (i.e. page 275) whereas it's exactly the technique used to prove theorem 28.2. So I guess that there is a subtlety that I didn't get between metric space and metrizable space (because for me it almost look to be the same thing, and the method used in theorem 28.2 should also work to prove theorem 45.1).
Comment: Possible duplicate: https://math.stackexchange.com/questions/215986/there-is-no-difference-between-a-metrizable-space-and-a-metric-space-proof-incl?rq=1
Answer: A metric space is a metric space, so a pair $(X, d)$ where $X$ is a set of points and $d$ is a metric on $X$.
A metrizable space is a topological space, so a pair $(X, \tau)$ where the topology $\tau$ is nice enough that it is possible to find a metric $d$ (and thus infinitely many different metrics) on $X$ such that the induced topology on the metric space $(X, d)$ is exactly $\tau$.
Comment: I can see that my question is not that clear. But anyway : why in the Munkres, the concept of totally boundness has been introduced that late ? Because in theorem 28.2 they proved that sequencial compact $\implies $ totally bounded $\implies $ compact. For a metrizable space are there sets that are totally bounded for some metric and not totally bounded to other metrics ? I guess no because different metric in a space induce the same topology...
Comment: @MathBeginner The real number line in the standard metric isn't totally bounded, but it is in the metric given by $d(x, y) = |\arctan(x)-\arctan(y)|$. These two metrics give rise to the same topology.
Comment: @MathBeginner total boundedness and also completeness depends on the metric, so it's only defined for metric spaces, not for metrisable spaces. A metrisable space can have two compatible metrics, one of which is complete and the other not.
Answer: When speaking about a "metric space" it is usually assumed some concrete metric is given, whereas a "metrizable space" is a topological space whose topology can be shown to be equivalent to some topology induced by a metric. In some cases the metrizability of a space follows indirectly, without pointing out a specific metric.
Answer: If $(X,d)$ is a metric space, it is also a topological space. Just consider the topology $\mathcal{T}_d$ generated by the open balls for the distance $d$.
However, when you have a topological space $(X,\mathcal{T})$, it is not obvious that there exists some metric $d$ such that $\mathcal{T}_d = \mathcal{T}$. In fact, sometimes this is not the case. When it is possible to find such a metric, we say that the topological space $(X,\mathcal{T})$ is metrizable. Note that the notion of metrizability is always relative to some topology $\mathcal{T}$.
Answer: There is an unanswered question regarding the order of presentation.  Why is total boundedness introduced so late?  If "in metric spaces, compact iff sequentially compact" was done in p179, why wait till p276 to define total boundedness?
In Kolmogorov & Fomin,

the notion of total boundedness was introdued in p97,
"a metric space is compact iff it is totally bounded and complete" is proved on p100,
"the subset of a complete metric space is relatively compact iff it is totally bounded" is proved on p101.

I believe that this natural order of presentation is what you are looking for.


About complex eigenvectors and proof of Sylvester's criterion

Question: Hello my question is referring to the assertion of the user 1551 in that question:
Characterization of positive definite matrix with principal minors
Assertion itself :"in particular, det(A)>0. It follows that if A is not positive definite, it must possess at least two negative eigenvalues  " - why 2, and not 1? And why that eigenvalues can't be just imagine, so they won't be as positive as negative(and is that possible?  Or they always be strictly positive or negative?)
Answer: The eigenvalues of a Hermitian matrix are real. If there is only one negative eigenvalue then the determinant, which is product of the eigenvalues would be $\leq 0$.


filter variables whose column names contain pattern

Question: I am trying to filter out NA, NaN and Inf values out of a tbl using dyplr's filter function.
The trick is that I only want to apply the filter to columns whose names contain a specific pattern. The pattern is: r1, r2, r3, etc.
I have tried to combine grep and filter to achieve this, but can't get it to work. My current code looks like this:
filter_(!is.na(grep("r[1-9]", colnames(DF), value = TRUE))
& !is.infinite(grep("r[1-9]", colnames(DF), value = TRUE))
& !is.nan(grep("r[1-9]", colnames(DF), value = TRUE)))

However, this code returns a warning message: "Truncating vector to length 1."
And the data returned is unfiltered.
I suspect that it's the is.na functions here that are causing the problem, because I've seen an example online where you can apply grep to filter using a normal condition (i.e. condition == value) and not a condition based on is.na
Comment: Can you provide a [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) of your dataset?
Comment: I believe that the problem is that you are testing `colnames(DF)` for `NA`, `NaN` and `Inf`, when it should be the values in those columns. (The ones that match your pattern `r[1-9]`.)
Answer: dplyr provides matches() that is useful for this
Example 1: How matches() work?
library(dplyr)

# remove columns that start with "mp"
mtcars %>% select(-matches("mp"))

# keep columns that start with "mp"
mtcars %>% select(matches("mp"))

Example 2: Using matches() in the context of your request but using a MWE
# Create a dummy dataset
data = tibble(id = c("John","Paul","George","Ringo"),
              r1 = c(1,2,NA,NA),
              r2 = c(1,2,NA,4),
              s1 = c(1,NA,3,4))

# Filter NAs in columns that start with r followed by a number
data %>% filter_at(vars(matches("r[0-9]")), all_vars(!is.na(.)))

Comment: I've just realised I did not complete the example. Now its complete, @anita-mcgill
Answer: With dplyr, you can use the filter_at function:
dat %>% filter_at(vars(matches("^r[1-9]")), all_vars(is.finite(.)))

Using @lmo's sample data, the result is:

  r1 r2         d
1  1  1 0.2774292

Comment: Hi, this works. But I've now discovered a further issue which can be resolved by only filtering the `Inf` and `NaN` values (and leaving in `NA`). To do this can I replace the second argument with `all_vars(!is.infinite(.)) & all_vars(!is.nan(.))`
Comment: @AnitaMcGill above I posted a solution that points to your current problem
Answer: Here is a base R method to filter rows, comparing specific columns.
# sample data
set.seed(1234)
dat <- data.frame(r1=c(NA, 1,NaN, 5, Inf), r2=c(NA, 1,NaN, NA, Inf), d=rnorm(5))

this data set looks like
dat
   r1  r2          d
1  NA  NA -1.2070657
2   1   1  0.2774292
3 NaN NaN  1.0844412
4   5  NA -2.3456977
5 Inf Inf  0.4291247

We will check the first two columns and ignore the third column. Notice that the only row that should remain is row 2.
dat[Reduce("&", lapply(dat[grep("^r", names(dat))], is.finite)),]
  r1 r2         d
2  1  1 0.2774292

Here, a data.frame that is subset using grep to select the appropriate columns (1 and 2) is fed to lapply. The regex "^r" says only include variables whose names that start with "r". In the lapply loop, each vector is checked using is.finite. This function returns FALSE for NA, NaN, and Inf. The resulting list of logical vectors is fed to Reduce` which returns a logical vector the length of the number of rows of the data.frame where an element is TRUE if and only if every element in a row is finite.


How can I redirect terminal output that "refreshes" using '\r'?

Question: I am attempting to write a (Bash) shell script that wraps around a third-party python script and captures all output (errors and stdout) into a log file, and also restarts the script with a new batch of data each time it completes successfully. I'm doing this on a standard Linux distribution, but hopefully this solution can be platform-independent.
So here's a simplified version of the shell script, omitting everything except the logging:
#!/bin/bash
/home/me/script.py &>> /home/me/logfile

The problem is the third-party python script's output is mostly on a single line, which is being refreshed periodically (~every 90 seconds) by use of a carriage return ("\r"). Here's an example of the type of output I mean:
#!/usr/bin/env python3
import time

tracker = 1

print("This line is captured in the logfile because it ends with a newline")

while tracker < 5:
    print(" This output isn't captured in the log file. Tracker = " + str(tracker),end="\r")
    tracker += 1
    time.sleep(1)

print("This line does get captured. Script is done.         ")

How can I write a simple shell script to capture the output each time it is refreshed, or at least to periodically capture the current output as it would appear on the screen if I were running the script in the terminal?
Obviously I could try to modify the python script to change its output behavior, but the actual script I'm using is very complex and I think beyond my abilities to do that easily.
Answer: The program should have disabled this behavior when output is not a tty.
The output is already captured completely, it's just that you see all the updates at once when you cat the file. Open it in a text editor and see for yourself.
To make the file easier to work with, you can just replace the carriage returns with line feeds:
/home/me/script.py | tr '\r' '\n'

If the process normally produces output right away, but not with this command, you can disable Python's output buffering.
Comment: Ah, so it looks like I was incorrect when I guessed that the python script was using carriage returns to refresh the line.  Your answer works for the example I gave, so I accepted it, and now I will have to troubleshoot how to capture the output from the actual script I'm using, which must be using some more advanced method to refresh the line. Thanks!
Comment: Ansi escapes probably. That's a harder problem. You can `tr -d '\033'` to delete the escapes that start them and see how messy it is. Maybe your program only uses a few types like color and clearing the screen so that you can replace them with sed. Also, double check that your tool doesn't have a batch mode or non-ansi mode like standard Unix tools do.


Need to listen to iTunes track and piano for learning

Question: I am learning piano and thought that if I play the lesson track into my headphones at the same time as the piano output I could practice timing better, and for the pieces I need to learn for the exam I can get the timing perfect. What sort of equipment do I need?
Comment: I would suggest a mixer with at least two stereo channels and a headphone output. You'll also need leads to connect the piano & music player outputs to the mixer inputs. (I can't recommend a specific model.)
Comment: what are the outputs on your piano (1/4" stereo?), your music player (3.5mm stereo?) and what style of input is your headphones ( 3.5mm stereo?). Does your piano have an input?
Answer: For this kind of pratice you don't need a perfect studio mix, you may not need anything more than your music player and in-ear headphones  (preferably not of the tight earplug type). Just put the phones (or even just a single one) in your ear(s) and regulate volume so that you can confortably hear  the "backing track" with just enough volume to let you follow the timing while normally playing your piano.
Comment: I have to say, though, that although this kind of practice may be useful, up to a point, in my opinion it will be detrimental if you take it too far. If you just mechanically follow somebody else's performance you will have a mechanical performance yourself. You need to develop your own sense of timing and of "phrase as unity"  and it's unlikely you achieve that by metronome or "back track" practice alone.


What does it take to make metal music?

Question: I happen to have my father's old guitar laying around in storage. While I was listening to metal music on youtube, I had a thought: I wonder if I could play metal on the thing?
Problem is, I have really bad dexterity, and the only instrument I have any experience with is a snare drum (which I haven't played in probably two decades). I have no idea if I would seriously be able to play metal music at all. Also, I noticed that the kinds of guitars metal musicians use don't exactly look like mine. I think its one of those 'acoustic guitars' country singers like to use (my father used it to play REALLY old country songs). I found some video on youtube of a guy play metal with a guitar that looked A LOT like my father's, and it made everything sound weird and not very 'metal'. This leads me to believe that if I seriously want to try to play metal, I'd have to get myself a different guitar.
Problem is, I don't know anything about guitars (other than the basics of how to actually operate one). I don't know what they cost, or what kind I would have to get. I'm not even 100% sure what kind of guitar mine is to be honest. I found some video online of a guy who managed to somehow play metal on a cheap 'box' guitar he bought for 'forty quid' (w/e that means). Though honestly, I highly doubt he seriously got that sound out of that cheap thing (he clearly had some kind of pre-recorded drum music playing in the background, and maybe a second guitar too). On there, some people were saying they needed to buy an 'amp' (I've heard of those, but I don't actually know what they do, yeah, I'm clueless) that cost $3k!!!!! Okay, that would be a HUGE investment for me, and I may find that I can't even play the thing anyway (like I said, I have abysmal dexterity).
What does it actually take to seriously play metal music? What kind of skills do you need? What kind of equipment do you need? How much should I expect to pay for said equipment? Can an amateur who's just learning even play metal from the start? Do you need decent dexterity to even play a guitar? Like I said, I'm clueless.
Comment: "fourty quid"  is £40 (Great british pounds sterling), approximately $50USD, We use "quid" like Americans use "bucks".
Comment: ...OK, is your guitar an acoustic guitar or an electric guitar? Your comment that you "think its one of those 'acoustic guitars' country singers like to use" makes me suspect the former, and the only time I've heard acoustic guitars in metal is in introductions (or, rarely in  originals but more common in arrangements of non-metal works, interludes or outros).
Comment: You should watch some material from Jack Black and his band, Tenacious D.
Comment: You can play metal on anything. Including harmonica, [harp](https://www.youtube.com/watch?v=KPb20fK0R94), banjo, ukulele, and even a [shovel](https://www.youtube.com/watch?v=IyQOw-_H4yE).
Answer: Hi and welcome to the music stack exchange. I want to preface this answer by saying that metal is not an 'easy' genre to play well. As danmcb mentions, many many metal musicians are highly competent... and even formidable in their technical prowess. It is a bit of a misconception that metal is an 'easy genre to play' considering a good portion of casual listeners might consider it to be something like 'a bunch of distorted nonsense.' If you are a fan, then you likely know that the term "metal" is also a bit of a misnomer considering that the genre has sprouted many heads (musical sub-genres) over the years; the catch-all term 'heavy metal' being somewhat resented by many facets of the metal community... For example, the band KISS claims to be a heavy metal band. You might find yourself in a heated debate with a fan of Metallica or Megadeth over that claim...
If you are considering a certain genre of metal to begin to learn guitar because it is your favorite genre and you simply want to play it, that's one thing... Otherwise if you are looking for an 'easy' genre to start with, I'd suggest you start with any genre of pop-music. You might find this article helpful.
Now for your questions:

"I wonder if I could play metal on the thing?"

Though it isn't the most common guitar type/sound used in metal, there is a lot of material in the various metal genres that exhibit fine work in implementing acoustic guitars. See here for an article detailing the acoustic guitar's role in the evolution of black metal.
And see here for an example of an 'all-acoustic-guitar' black metal song.

"What does it actually take to seriously play metal music? What kind of skills do you need?"

What is takes to play any music seriously is a drive and passion for playing music. Unless you are a prodigy, long hours of effort and practice are required... and perhaps, a certain thickness of skin in the instances in which you will inevitably be frustrated, burn-out, be negatively criticized, experience pain/injury, etc
I hazard answering this with a question but... what do you want to get out of it? If you are simply looking to recite your favorite songs, practicing those songs until you can replay them may be all that's required... If you are looking to write original music, some level of creativity and knowledge of music theory may be required. I say 'may be' because you can possibly write music without these skills/qualities but, you and others may not be so happy with the final results...

What kind of equipment do you need? How much should I expect to pay for said equipment?

You can certainly start with what you have... But, again, depending on what you want out of your experience and hard-work will dictate what you decide you will "need" later on... Most metal guitarists have multiple guitars, multiple amps, and many many pedals/effects processors in order to achieve the sounds they "require" for what they do. You can, however, find very cheap guitar/amp combos to get your feet wet; but, beware that they are usually very low quality. Good instruments are not cheap and, you generally "get what you pay for" in music.

Can an amateur who's just learning even play metal from the start?

Yes

Do you need decent dexterity to even play a guitar?

Yes
Comment: "Unless you are a prodigy" – prodigies still need to practise. They just start earlier...
Comment: "If you are a fan, then you likely know that the term "metal" is also a bit of a misnomer considering that the genre has sprouted many heads (musical sub-genres) over the years" – Some would (jokingly) argue that metal has n+m sub-genres where n is the number of bands and m is the number of fans, most of which contain at most one band and/or fan.
Comment: I don't disagree. I simply don't understand some prodigies and the fact that they are 'good' for seemingly 'no reason' ;)
Comment: Touche - I admit I can't make much sense of the metal 'family-trees' that one can find out there on the web. Many of the distinctions seem to fit but, I'd agree that there are perhaps too many distinctions, as well...
Answer: I'm glad you asked this question. There's definitely a lot to know about the world of guitars, and just by immersing yourself and asking questions just like this one, you will learn it over time.
First of all, an acoustic guitar and electric guitar do the same thing. They make guitar sounds. You can play any genre on either, but most typically you will hear metal music played on an electric guitar, with plenty of distortion and fuzz effects. That being said, if you want to learn how to play metal on an acoustic guitar, you can absolutely do so, despite what any genre elitists might tell you.
The reason they call one of them "electric" is because it uses pickups (basically magnets) to detect the strings vibrating and sends that as an electrical signal, usually to an amplifier (amp for short) which amplifies (makes louder) the sound, usually so an audience can hear it. Conversely, an acoustic guitar needs no amplification, as the strings carry sound by vibrating over a "sound hole" in the body of the guitar, causing reverberations through the wooden body which carries the sound. There are exceptions, like acoustic-electric guitars, which have acoustic bodies and a 1/4" plug so it can be played through an amp. But traditionally, a classic acoustic guitar setup would just have a microphone near the guitar to amplify the sound to a crowd.
So, we can easily deduce what type of guitar yours is. If it's got a big wooden body with a roughly 4" diameter hole in it (usually right under the strings) then it's an acoustic guitar. If it's got a slimmer body with no hole but a 1/4" plug (usually near the volume dial) then it's an electric guitar.
Now that you know what type of guitar you have, you have some options:

If you have an acoustic guitar, you can still learn to play metal, in fact it will be much cheaper money-wise because you won't have any need for an amp or effects. It may not sound very "metal" because you won't have the distortion and effects, but you can play the same melodies and riffs and still make metal music.
If you have an electric guitar, you will, at the very least, need an amp of some kind. It doesn't need to be fancy, you can get by with a mini-amp for $20 that plugs directly into the guitar. Once you have an amp and can hear yourself playing, you can look at adding effects pedals. These pedals will add different effects to your sound, to give it that overdriven fuzzy "crunch" that's traditionally included in metal music. As far as which pedals to get, every guitarist will recommend different pedals, so you'll just have to visit your local music store and try some out to see what works for you. This will be a long and arduous process, but the benefit is that you can customize a sound that's iconic to you. Most guitar players that have assembled a pedalboard have done so through a long period of trial and error, and most will be willing to share some experience and recommendations if asked, so just ask some people and demo some pedals to find the sound you want.

Finally, I'll briefly address your issue with dexterity. It's true that most metal music is played fast, but most of that quickness is developed over time. When practicing a song, it's not uncommon to practice at half-speed or slower, then build speed up as you gain confidence. So I wouldn't worry about being super quick when you start out, that will come with time. The biggest issue to worry about would be whether your physical proportions properly fit your guitar. The best way I've heard to gauge this is to take the neck of the guitar in hand, at the far end (the first few frets) and lay your fingers flat on the fretboard. Your middle and pointer fingers should extend slightly off the opposite (top) edge of the neck. If they don't, you may need a smaller guitar. There are 3/4 and 1/2 size guitars out there, and any good luthier can customize the size of a guitar for the right price. So as long as your guitar fits you physically, the rest will just come with practice and time.
Again, thanks for asking this question, and welcome to the world of guitar! Please keep asking questions, not just here, but to any guitarists you meet! Most people are thrilled to talk about their passions, and it's truly the best way to learn about a topic.
Answer: Lots of metal musicians are highly competent musicians, some are quite virtuosic. Basically, find a decent teacher, and start practicing. You will need to learn guitar and some music fundamentals about time, rhythm, harmony and so on. First learn to basically play the instrument, and then decide on a genre. Don't skip basics.
(A good teacher should advise you about the instrument.)
Comment: How do you tell who is a decent teacher if you don't know anything to begin with? ;)
Comment: do you have any friends that play? (the OP I mean). Good teachers typically have a reasonable number of students.
Answer:
What does it actually take to seriously play metal music?

Practice. It isn't different than any other style in that regard. If you are serious, practice seriously.

What kind of skills do you need?

Fingering knowledge for chords and scales, and a good sense of rhythm/timing. The same as any other style. You can't ignore those fundamentals. Improvising with scales is especially important for the lead guitarist. In terms of specific techniques don't make the mistake of thinking metal guitarists only play metal. Think of it as becoming a guitarist first, then playing metal style.

What kind of equipment do you need?

A basic set up is an electric guitar, a small practice amp, and at a minimum a distortion effect. Nowadays effects can be built into an amp or applied to a guitar signal with a computer. Guitarists spend a lot of time on their gear and there are too many options to list here comprehensively. Just get started with an ax (the guitar) and an amp. FWIW, then I took lessons as a kid my teacher wouldn't let me use effects during lessons. None at all, not even a touch of reverb. Even when he taught me hard rock/metal songs the sound was low volume through a completely 'dry' amp.

How much should I expect to pay for said equipment?

To get started with basic equipment you should only need to spend about $250. Browse guitarcenter.com and use the price range filter. Squier and Epiphone are two entry level brands of guitar. A small practice amp can be had for $100. Don't pay attention to gearheads and snobs who would look down their noses at beginner gear. Start learning to play first then build up a collection of gear.

Can an amateur who's just learning even play metal from the start?

Yes. You can start out learning riffs like Sweet Leaf and Hair of the Dog and that kind of thing.

Do you need decent dexterity to even play a guitar?

Yes. You don't need super human abilities, but you do need to develop your skills through practice. The average person can learn to play if they practice and don't get hung up on self imposed mental blocks.
Answer:
On there, some people were saying they needed to buy an 'amp' (I've heard of those, but I don't actually know what they do, yeah, I'm clueless) that cost $3k!!!!

Others have written a lot of good advice, so I'll keep it brief and just add this:
In 2019, you realistically only need an amp if you are planning on playing with a band and/or live on stage. And even that is getting more and more debatable. For both your wallet and your learning, you would probably be better served with a computer or laptop (doesn't have to be anything fancy, no expensive gaming rig required), a usb audio interface like for example this one or this one and a reasonable pair of headphones. You can plug your electric guitar straight into this device, an acoustic guitar would require a microphone (and would probably be less then ideal for metal, as an aside). The advantages of this are:

There is a lot of software (free or for a modest price) that you can use to "try out" a huge range of amps and effects. So you can experiment to find the sound(s) you like and when you're ready to spent that 3k, you will have a reasonable idea of what to expect.
You can easily mix in metronomes, drum loops or even entire songs to play along with, which makes learning both more fun and more productive.
You can easily record yourself while playing, which will help a ton in getting good (at any instrument btw). Much underestimated imho.

Lastly I would like to add this, it's the best advice I ever got:
Take every opportunity to play with other people! If you've played for a week, you can probably bang out a few (power-) chords. Now if you can find someone who has been playing drums for a week, they can probably lay down a simple 4/4 beat. Congratulations, you now have a punk rock band, go out and have fun! If you keep at it, everything else will follow naturally.
Comment: The focusrite is a bit more expensive and, because of ProTools, has a steeper learning curve.  OP seems like they might benefit more from something a bit more plug-and-play like the [Pod Studio GX](https://line6.com/pod-studio/gx/).  It comes with Cubase LE (instead of ProTools), but you can use the standalone amp simulation to jam along with a song without needing to worry about setting up or even using Cubase (until OP actually wants to record something, of course).
Comment: @J... Point taken, I've added the Pod Studio to my answer. Tbh I never used the software that came along with my interface, I used the focusrite as an example because it's both inexpensive and plug and play for mac/windows/linux but at the same time will serve you for years and years even in a semi pro (in-the-box) setting. That said, they're just examples, OP would be wise to read up on usb interfaces a little before buying one (or find a knowledgeable retailer).
Answer: I'm just going to comment on your last questions.

What does it actually take to seriously play metal music?

Realistically, a guitar and an amp. While a lot of metal musicians are actually incredible and know a lot of music theory, you really don't need to know a whole lot about music itself to write or play it. While I would recommend lessons, but you do start to recognize patterns in riffs, and chords.

What kind of skills do you need?

Time and dedication! Practice is important to getting better. Dedication is important because many people will have a time where they think that they play like shit (even if they aren't) and don't want to play anymore.

What kind of equipment do you need? How much should I expect to pay for said equipment?

This is fairly subjective honestly. The guitar you get doesn't matter a whole lot when you are a beginner. You can buy a 3k guitar that's meant specifically for metal, and still make it sound like shit. You can can even buy a $150 guitar, and make it sound great for metal. My suggestion for that, is to go somewhere and play a few guitars, look at them, see how the feel in your hands.
For an Amp, I'm not too knowledgeable about this because I use everything purely digital on my computer.
If you have a powerful computer, you could get an audio interface (I've had both a scarlet 2i2 and 2i4 and they both are great). After that there are some free VSTs for metal you can use. You then need to use those with a DAW, like Reaper (Reaper is a free DAW).

Can an amateur who's just learning even play metal from the start? Do you need decent dexterity to even play a guitar?

When I picked up guitar, first song I learned was Enter Sandman (not any of the lead stuff though). I played it alright I guess. As long as you aren't playing anything super technical and shreddy, yes you can probably learn it, or learn secitons of it.
Even if you aren't sure, you can try! There are some programs out there that will let you slow songs so you can practice.
You do need dexterity, however, thats something you build up. When I first started, I had poor dexterity. However, I practiced alot and now, its not as much of a problem for me.
Remember, it's never too late to pick something up
Comment: It's 2019.  Any computer with a Pentium 4 and WinXP or newer (ie: anything) can run a simple audio interface and amp sim.  It doesn't need to be a "powerful" computer.
Comment: On a laptop it might help to disable power saving features for audio work.
Comment: I agree, but I also had a new laptop that had a lot of issues with the 2 DAWs I used on it. Running really slow, poor latency, popping sounds, that I never had with my Desktop.
Comment: I would say that a DAW is bridge to far for OP (unless you count Bandcamp or the likes) at this stage. They would be faced with both the learning curve of the DAW and the learning curve of playing the guitar at the same time, both are considerable.
Comment: BTW I run Logic Pro X on a 2012 Macbook and easily record a full metal band on it, you might want to check your power savings features like ojs said and your sample rates (low for recording, high for mixing). Could be you just got a bad laptop though, but like J.. said, any modern PC or laptop should be able to run at least an amp sim, but in most cases also an Ableton, Fl studio or if on mac, Logic pro X
Comment: @Douwe Nevermind a '12 - I used to record on an old 2005 G5.  Had plenty of power for multitrack and plugins.
Comment: @Douwe I probably just got a bad laptop. It has quite a few issues honestly. I changed all the power settings stuff, checked ram, and a bunch of other stuff. So who knows at this point I guess. Never use it anymore instead of my desktop
Answer: Sorry I got to this thread so late, lots of great answers here though and good information.  I thought I'd add a few things as I was --like you-- first inspired to play guitar by a simple (doom) metal riff, and all I had was an acoustic guitar.

Problem is, I have really bad dexterity, and the only instrument I have any experience with is a snare drum (which I haven't played in probably two decades). I have no idea if I would seriously be able to play metal music at all.

If you have "bad dexterity" it's probably just from a lack of practice.  You will get there in time.  I recommend starting out with a drop tuning like drop D (not to be confused with dropped d).  The reason is because then you can play power cords with only one finger and that tuning is fairly forgiving if you make mistakes.  I had a friend though who recommended not doing this due to it making scales harder to learn.. I happen to disagree.  If you start in that tuning then you can always go backwards.  Just keep in mind it changes the scales.

Also, I noticed that the kinds of guitars metal musicians use don't exactly look like mine. I think its one of those 'acoustic guitars'

This may be a problem.. I was in the same boat and tried to play metal on an acoustic (even trying tricks like using acoustic electric pickups); it's almost impossible to get a true metal sound on an acoustic guitar.  Some metal uses intros and you can certainly play the same progressions/scales etc, but the sound will never be quite right for heavy distortion (other than black metal).
In fact, even among electric guitars, not all are cut out for the job.  There are certain models/brands that are well known for being metal guitars such as: ESP, Ibanez, Gibson (Les Paul and flying V)
There are also specific pickup types such as active and humbucker that you can opt for which make huge differences in terms of sound.
EMG active pickups on an ESP guitar (for example) give an extremely unique and inherently apt sound for metal.  You will probably find it a lot easier to do things such as pinch harmonics, tapping, fast melodic scales with active pickups.
http://www.emgpickups.com/81.html
https://shredaddict.com/best-pickups-for-metal/

Problem is, I don't know anything about guitars (other than the basics of how to actually operate one). I don't know what they cost, or what kind I would have to get. I'm not even 100% sure what kind of guitar mine is to be honest. I found some video online of a guy who managed to somehow play metal on a cheap 'box' guitar he bought for 'forty quid' (w/e that means).

I'd recommend starting out with a used Epiphone Les Paul.  They can be had for pretty cheap and will get you started.

What does it actually take to seriously play metal music?

Time, practice, and dedication to the art (along with basic equipment and perhaps a drum machine to help keep beat at first).

What kind of skills do you need?

Music theory and guitar techniques for metal couldn't hurt.  I tend to like the harmonic minor scales for writing metal.  I've read knowing minor and exotic scales helps a lot.  Some people use half tone, dimished, melodic, and chromatic scales.  You kind of need an ear for how it's supposed to sound (which comes with time and a love for metal).  The more you've listened to it, the easier it is to write.
https://www.guitarplayerworld.com/scales/
http://all-guitar-chords.com/
(search online learning scales)
Palmetto, staccato, tremolo, palm muting, pinch harmonics, taping, chromatic and shifting key signatures are all techniques frequently used and the more you can add to your arsenal, the better. \m/

What kind of equipment do you need?
How much should I expect to pay for said equipment?

Guitar 100$
https://www.ebay.com/b/Epiphone-Les-Paul/33034/bn_7023347071
Picks 5$ - Heavy picks like Jazz III Dunlop help a lot for fast accurate picking IMO
https://www.jimdunlop.com/category/products/guitar+picks/jazz.do
transistor AMP (tube amps aren't good for metal unless you pay a lot IMO) - 100$ .. get something like a peavy (with high gain control) or a line 6 (has lots of built in distortion for metal and other genres).  You don't need to spend a lot.

Can an amateur who's just learning even play metal from the start?

Absolutely, I just recommend starting with something like doom metal, which is usually easier to play off the bat than something like technical death metal.
I would also recommend getting a decent drum machine or something like Fruity Loops and make or use other people's beats to play along with.  It will help keep you inspired and prevent boredom during tedious initial learning curve.  Beats will also give you ideas for your own riffs eventually.

Do you need decent dexterity to even play a guitar?

Somewhat, I think you'll be surprised how quickly you pick it up, if you have a passion for it.  It might encourage you to start out in drop D because those bottom cords are really easy and have a pretty crunchy sound when palm muted.
Answer: A lot of metal musicians are quite virtuosic BUT this should never hold you back. But there are also tons of different subgenres with different difficulties. For example, 'classical' metal is really good for beginners, where things like Post-Metal or Metalcore could be more different.
Regarding the price: yes, high-end equipment is quite expensive, but there are a lot of affordable options. If you just want to practice, then combo amps are the way to go. In my case, I started with a 200$ Vox Amp and I still use it. If you want to perform live, then you would have to invest a bit more.
But the most important thing is, to learn the basics. Talk to a teacher or watch one of the countless tutorials, it's really important to get the basics right.
Comment: I don't think "virtuous" is the word you're looking for here -- "having or showing high moral standards".
Comment: Virtuosic*, I think... https://www.dictionary.com/browse/virtuosic
Comment: I had already edited this answer to fix that, but I guess it took a little while to take effect?
Answer: Welcome to the wonderful world of Heavy Metal! A lot to unpack in this question, so let's get started.
To answer your main question, the only thing it takes to play metal music is to play metal music. Now, certain things can definitely help your playing, but the most important thing is to play. Any guitar will work, as long as you're playing it and enjoying it. Finger strength and dexterity will come with the hours you put into the instrument. It can be a painful process at times as you're building up finger strength, but if you can push past that you'll find it hard to put the instrument down.
In terms of gear, your guitar (which you've described as an acoustic) will work for you. There's many metal bands with extensive acoustic work, from almost pure acoustic like Tenacious D to one-or-two per album bands like Def Leppard. If you want the traditional metal guitar tone, however, you'll need to invest in an electric guitar. This doesn't have to be expensive, but you will need some gear to get the "metal" tone you're looking for. An electric guitar generally needs an amplifier to sound decent, but these don't have to cost you thousands! In this day and age, there's vast amounts of extremely affordable, amazing-sounding gear around. Your best bet will be to go to a local music shop and ask someone for good entry-level gear, play around with some of it and see what you like best. You can get a great rig that will serve you well for years for a couple hundred dollars.
If you want to get serious about metal music, you'll need to practice. You don't need to be an amazing guitarist to play metal, but a lot of amazing guitarists do play it. Now you don't need to be Jason Richardson or Yngwie Malmsteen level of player in order to succeed in metal, but you will need to practice. Learn songs you like and play along to them. I'm a career session/touring metal guitarist, and that's how I picked it up. A good guitar teacher can be invaluable to get started, head back to that local music shop and see if they do lessons or have recommendations for great teachers in the area.
I hope you enjoy your journey into the fantastic world of metal music. It's been a massive part of my life for decades, and I'm thrilled that people are still jumping into this massive world. Just remember to have fun playing, and that metal is for everyone!
Comment: An amp isn't necessarily a full plexi stack. A 10W Roland Cube is an amp, a $20 used Spider 2 is an amp, and they're bot a hell of a lot cheaper than a Scarlett interface and an amp sim plugin.
Comment: Yes, but they're extremely limiting, sound like pap, and there are much cheaper and simpler options than a Scarlett interface (like Pod Studio GX, etc).  I'd rather even buy a cheap used TonePort over a Spider 2 - why bother with a giant speaker if you don't need it?
Comment: Fair.  I find a TonePort is totally fine, especially for a beginner, costs like $100, and you get pretty insane flexibility with all the cabinet, virtual mic selection and placement, and head options - not to mention that you get additional pedal effects, reverbs, and even a tuner that you would need to buy as all separate hardware components.  For an ultra-budget learner setup you just can't beat it.
Comment: You don't need an amp unless you're planning to either gig with a band or annoy the s**t out of your neighbours. It's 2019.  I haven't had an amp since the 1990s.  Software amp simulators are fantastic for learners - killer tone you would never get without a mic'd 4x12 marshall cranked to 11, but in the comfort of a set of headphones.  Also - way cheaper!
Comment: The only plugin I've got to sound I've been able to get to sound decent is some Neural DSP stuff, and that's definitely outside the realm of entry-level gear. Everything else I've used required quite a lot of EQ and tweaking, which is also beyond the scope of entry-level. Regardless, let's treat "amp" as shorthand for "amp-like device that adds distortion."
Answer: There is a lot of "metal music" played on acoustic guitar, piano, violin, flute or damn pretty everything. It takes a tallent, learning, practicing and luck, just like any other kind of music. The instrument is secondary and may even change few times in the process. (The music genre can pretty much change, too.)
And, be sure you don't need anything like "amp" in the first, say, 6 or 12 months of learning. You won't probably have anything worth amplifying. YMMV.
Comment: The problem with playing metal music on instruments not typically associated with metal is that listeners may not think it is metal anymore. I was surprised when I read that a chiptune I listened to in my teenage years and still love today, virt's "Sorcerors Battle", was fairly consistently labelled as metal, even though I never considered it as such (probably due to the instrumentation).
Comment: An extreme counter-example: Apocallyptica.
Answer: You are looking to start playing metal music on an acoustic guitar. Whether this is possible depends entirely on your expectations (as others have said). To play like the greats you need a lot of practice and good equipment.
You are though in a very good place to start, timing is VERY important in metal music and having experience with the snare gives you a great base.
One thing that might be useful is to tape the hole in the guitar shut. Metal is usually played very fast and with the acoustic chamber opened to many noises will fly around. Taping it prevents this, palm muting helps as well.
Answer: You are asking a lot of questions in one.
To address one point you can play anything on anything!  That is to say there is really no limit to musical expression so playing Battery on an acoustic is your right (in fact the intro is acoustic so that would sound authentic).  This brings me to the other side of the coin.  If you are interested in playing Metal songs you can certainly play them on an old acoustic or an electric that is not typically associated with Metal but you may not get an authentic replica of the sound you favorite bands are using.
If your goal is obtaining a tone that you think of as Metal then you will want to explore gear options.  But to your point about "cheap" guitars you can definitely get a great sound out of a cheap guitar!  The guitar needs to be set up properly and it's just the starting point.  Eddie Van Hallen's Franken guitars were made from cheap spare parts (probably less than $100 worth total) but they scream like crazy.  You need to get the right kind of amp to start and most Metal guitarists use some pedals, distortion, delay, reverb, etc, to get a very hot sound.  This is as much a personal touch as anything.  To the uninitiated ear all Metal might sound like high pitch hot fuzz but in reality the variety of tone is very broad.  Compare Steve Vai, Joe Satriani, Dave Murry, Randy Rhodes, Marty Friedman, Yngwie, etc, etc.  So the question falls back to you w/r to what type of Metal to you like?  Death Metal, Speed or Thrash Metal, what could now be called classic metal?  This will guide you in the right direction.
As for gear?  You can get a very nice combo practice amp for less than a few hundred buck USD and I think you should start there.  You won't need a Marshal Stack (the cliche standard) for a long time if ever.  If you invest a few hundred in a Line 6 Spider or similar small amp with one or two speakers these come with pretty good effects built in.  In my youth I had a Carvin stack and 20 pedals.  I work professionally and the tiny line 6 is good enough for anything.  Ironically high quality pedals can overrun the gear cost!  Brands like Boss can cost up to $250 a peddle depending on what it does.  So like I said, don't go crazy with peddles until you've had a chance to learn some songs and technique and decide you really want something that isn't in your combo amp.  In other words upgrade over time, not up front.  As you build up the gear you can always buy used.  A lot of musicians do.  There is some great vintage gear out there that isn't currently being manufactured but you can find on ebay and at musician swap meets.  But again, learn what they do and decide if you really need something before paying $$$ for it.  And as you get older the romantic self image of hauling 100s of lbs of speakers and high wattage tube amp heads wear off really quick.  It helps to keep it under 30lbs, easy on the back.
All styles of music and instruments have their unique challenges when it comes to learning.  You asked if one needs dexterity to play.  Of course.  But no one has dexterity to start.  This is why you take lessons and practice.  You don't need to be better than someone to start playing Metal.  You need to start trying to play Metal and you will develop dexterity.  That's the key.  There is no such thing as a less complex style of music.  Learning Blues or country first will not make Metal easier.  You need to jump into the style you like and you will develop the skills you need.  That said I would recommend taking lessons.  A good teacher will be able to give you exercises that accelerate the learning process.  Otherwise it is too easy to get discouraged.

What does it take to make metal music?


MP4 video is playing only the sound with no video in IE9

Question: I try to play an mp4 video in IE9, which has 237 Mo size, i can hear the sound but no video is displayed, my file is conformed to html5 doctype (<!DOCTYPE html>) and the following is how i implement the video tag (all works fine in chrome):
<video id="video" src="videos/myVideo.mp4" type='video/mp4' preload controls>
      Your brwoser doesn't seems to support video
    </video>

I got a white screen with the sound, but no video.
Answer: Internet Explorer 9 has some trouble running the HTML5 video player (as it does with a lot of things). Referencing W3Schools, I think that it would be wise to encode your video file across all filetypes to allow for maximum support cross browser. If you can, try to use a video service like YouTube or Vimeo because they take care of all the encoding for you, but if you need to do it yourself, check out the HTML5 docs on how to use multiple encoded files.
Comment: Hi, would you mean that i need to have the video file in multiple formats? (mp4, ogg, etc.).
Comment: To allow cross browser support you would need multiple formats which is usually why video services are a lot simpler for what you need to do.
Comment: but MP4 container, with H264 video and AAC audio, *is* supposed to work in IE (actually, as of today, it should be the only supported container among HTML5 video sources for IE)
Answer: I read somewhere that the mp4 needs to be encode using the H.264 codec in IE. I haven't confirmed this because I'm having problems playing my video in IE9 too. You can use MPEG Streamclip to convert your video file, at least on a Mac.


undefined local variable or method `last_email'

Question: I'm following the #275 Railscasts of Ryan Bates, but I can't make the tests pass, because of the last_email method.
Here is the test : spec/requests/password_resets_spec.rb

require 'spec_helper'

subject { page }

describe "Password reset for a registered user" do

  let(:user) { FactoryGirl.create(:user) }
  before do
    visit new_password_reset_path
    fill_in t('user.email'), with: user.email
    click_button(t('submit.reset.password'))
  end

  it { should have_selector('div.alert.alert-success', t('flash.success.reset_password.sent')) }
  expect(last_email).to have_content(user.email)
end

And here the error message :
spec/requests/password_resets_spec.rb:19:in `block (2 levels) in <top (required)>': undefined local variable or method `last_email' for #<Class:0x000000063b18d8> (NameError

Here is the new_password_reset view :
<%= form_tag password_resets_path, :method => :post do %>

    <%= label_tag :email, t('user.email') %>
    <%= text_field_tag :email, params[:email] %>

  <%= submit_tag t('submit.reset.password'), class: "btn btn-primary" %>
<% end %>

And the new_password_reset controller create method :
def create
  user = User.find_by_email(params[:email])
  user.send_password_reset if user
  flash[:success] = t('flash.success.reset_password.sent')
  redirect_to root_url
end

And just like in the railscast, I did the following :
spec/spec_helper.rb
require 'capybara/rspec'

RSpec.configure do |config|
  # ...
  config.include(MailerMacros)
  config.before(:each) { reset_email }
end

spec/support/mailer_macros.rb
module MailerMacros
  def last_email
    ActionMailer::Base.deliveries.last
  end

  def reset_email
    ActionMailer::Base.deliveries = []
  end
end

Answer: It's because you're calling the last_email method at the class level rather than within a block.
You need expect(last_email).to have_content(user.email) to be defined with a it block i.e.
it "should show alert and dispatch email to the user" do
   page.should have_selector('div.alert.alert-success', t('flash.success.reset_password.sent'))
   expect(last_email).to have_content(user.email)
end

Comment: Sorry, I had already added it, but I wanted to keep the code simple on Stackoverflow...
Comment: Yes I have the `Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}` line in my `spec_helper.rb` file...
Comment: Change `expect(last_email).to have_content(user.email)` to `last_email.to.should include(user.email)` and tell me if you get any errors.
Comment: Always the same error, I began with it, and changed it afterwards to see if there was any change.


Link opens new window and loads new page on parent window

Question: Okay I don't know if this sounds confusing but:
I want to be able to click a link that opens a new popup window and at the same time also loads a new (and different) page on the parent window.
What I have right now:
Click here

The onclick works but it doesn't open a new popup window. When I take out the onclick, the popup window works. Any ideas or tips?
Comment: Java & Javascript are entirely separate languages.
Answer: Try flipping the href and onclick - onclick happens before the href is evaluated/used.
Click here

When clicked, it should open the popup because of the onclick, then naturally navigate to "page2.html".
Comment: Ah! This worked! Thank you so much! And to think it was something so simple. Now I can call it a day. Thanks again :)
Comment: @jpaugh I'm not sure what you mean. When you click an anchor, the normal events occur (like a mousedown, mouseup, click, etc.), and as long as the event isnt prevented, the href is evaluated.
Comment: Aren't `onclick` and `href` mutually-exclusive?
Comment: @kevinfeng make sure to accept his answer, so he gets credit.
Answer: Instead of adding onClick, you can add
window.location.href = "page3.html"
In your newPopup function, along with opening a popup.


call a js function based on the argument passed

Question: I'm trying to call a js function within another one, but use the argument to specify the function. ie depending on the argument passed, it will call a different function
   function toggle(n){
        if (sessionStorage['toggle'+n]== 0){

            check+n();
        }
        else
    }

So, for example, if the argument 'Balloons' was passed as n, then it will call the function checkBalloons(). "check+n();" is not currently working here. Sorry for my lack of simple js syntax!
Comment: Consider if it would be better to make a generic `check` that takes parameter `n`.
Answer: If the function is defined in the global scope (browser) you can do:
window["check"+n]();

or some tenants like Node.js you would access it from global object.
global["check"+n]();

if it is a part of some other object then you would do the same.
 obj["check"+n]();

Functions and properties defined on an object can be accessed using [] convention as well. i.e obj["propFuncName"] will give you reference to it, so in case of methods you add () to invoke it.
Comment: Thanks, this works. However I will have to read up on the scope of functions as I'm not sure which to use. Thanks for help, though.
Answer: If the function is global, you would do this:
window["check" + n]();

or, you could put your function in an object like so:
myNamespace = {};
myNamespace.checkSomething = function(){ /* ... */ }

// call it like this:
myNamespace["check" + n]();

Answer:
The answers thus far are correct, but lack explanation.
In JavaScript, you cannot call a function by name when that name is a string. What you can do is retrieve a value from an object by name, and if that value happens to be a function, you can then call it. For example:
var myObject = {};
myObject.myFunction = function() { alert('test!'); };

// Get the property on `myObject` called `myFunction`
var theFunctionLookup = myObject['myFunction'];
// Since that property was a function, you can call it!
theFunctionLookup();

In the browser, functions that are defined in the global scope are attached to the window object. For example, this works:
function myFunction() { alert('test'); }
var theFunctionLookup = window['myFunction'];
theFunctionLookup();

You can shorten the last two lines into one:
function myFunction() { alert('test'); }
// Look up and call the function in one line.
window['myFunction']();

For the same reasons, you can use a dynamically-calculated string to look up functions.
function checkBalloon() {
  alert('checking balloon');
}

function toggle(n){
  if (sessionStorage['toggle'+n]== 0){
    window['check' + n]();
    check+n();
  }
}

toggle('Balloon');

Answer: if you do this way:
    if (sessionStorage['toggle'+n]== 0){

        window["check" + n]();
    }

will work
Comment: ok.... out of interest, what's the window part? I thought I was just not writing the syntax correctly?
Comment: you're accessing your method as a object property. Every global variable or function set in your code (not set into another function) will be set in your window object.
Comment: call a function only typing "check()", is the same thing that you call "window.check()", or "window["check"]()"


Representing "No value" in array in Google visualization

Question: For the below array i get a smooth curve.
  data.addColumn('string', 'x');
  data.addColumn('number', 'Cats');
  data.addColumn('number', 'Blanket 1');
  data.addColumn('number', 'Blanket 2');
  data.addRow(["A", 1, 1, 0.5]);
  data.addRow(["B", 2, 0.5, 1]);
  data.addRow(["C", 4, 1, 0.5]);
  data.addRow(["D", 8, 7 , 1]);
  data.addRow(["E", 7, 1, 0.5]);
  data.addRow(["F", 7, 0.5, 1]);
  data.addRow(["G", 8, 1, 0.5]);
  data.addRow(["H", 4, 0.5, 1]);
  data.addRow(["I", 2, 1, 0.5]);
  data.addRow(["J", 3.5, 0.5, 1]);
  data.addRow(["K", 3, 1, 0.5]);
  data.addRow(["L", 3.5, 0.5, 1]);
  data.addRow(["M", 1, 1, 0.5]);
  data.addRow(["N", 1, 0.5, 1]);

Now assume that I don't have the Blanket1 value for row D, how do I represent it so that there is coninuity in the graph?
If I make it like data.addRow(["D", 8, , 1]); the graph becomes discontinuous at the D for Blankets.

I want google to make a guess at that value and keep the curve smooth. Even if the guess is not smart that is fine but the curve should be continuous and smooth.
Comment: I've did some more searching and i've found it. See my answer below.
Comment: Did you happen to find an answer for this?
Answer: What you are looking for is the option:
interpolateNulls = true;

And then you just put a 'null' into the value array at the point where data is missing.
Check the API reference: http://code.google.com/apis/chart/interactive/docs/gallery/linechart.html#Configuration_Options


XPath matches() using regular expressions in C#

Question: Update: I wrote up what I did, with source code and things to watch for, at - Adding XPath 2.0 functionality and variables to .NET XPath (my blog).
Is there a way to pass a regular expression to matches() for an XPath statement in .NET?
Example: do an xpath of
<code>"/windward-studios/Employees/Employee[matches(string(LastName), '.*A?B[12]C.*')]"
</code>
Comment: Can you provide some examples of what you are trying to do?
Comment: Interesting.  Haven't had to use this xpath functionality so I didn't know it wasn't supported in .NET until seeing the answers.  Looks like this other post http://stackoverflow.com/questions/1525299/xpath-and-xslt-2-0-for-net covers libraries that provide support in .NET .
Answer: XPath 2.0 supports the matches function.  However, .Net implements XPath 1.0.
You would need to add a custom function to implement match.  See http://msdn.microsoft.com/en-us/library/ms950806.aspx
It looks like the Mvp.Xml Project: EXSLT.NET module implements most of XPath 2.0; including matches.  See http://mvp-xml.sourceforge.net/exslt/
Comment: what about net core?
Answer: Microsoft's XSLT processor does not support XSLT 2.0, but there are at least two others available on .NET that do: Saxon and XQSharp.
Microsoft basically decided some time ago to go down a proprietary route with Linq, and if you want to stick with a standards-based approach, you will have to go to third parties.


Any way to speed up unit testing in Visual Studio 2015 for UWP

Question: When testing a Windows Universal class library (.NET 4.6) in Visual Studio 2015, the time required in the "red->green->refactor" cycle is quite long. My test project is just the standard MSTest "Unit Test App" project. Even in the simplest scenario of a brand new solution, brand new subject and test projects, and a basic int Add(int n1, int n2) method, it's taking about 8-15 seconds. This is the time (after making a small code change) from clicking "run test" till the pass/fail is shown.
On my machine (Win 10 pro) conducting the same experiment but with a WPF-based solution yields about 1-2 seconds.
In the UWP scenario, the actual test time itself is listed as 79 ms. The rest of the time is compiling and deploying the unit test app container.
Is there any way to significantly speed up TDD with UWP?
Answer: The reason the TDD cycle takes so long is because all UWP code has to run in an AppContainer, and that requires packaging and deployment which is slow.
The way around this problem is to separate as much of your code as you can into a different project that doesn't need to run in an AppContainer and then test that project instead.
The solution for this is to use a portable class library for your application logic instead of a UWP class library. You'll find the portable class library in the new project dialog:

The default settings should work:

You then need to add a reference from the UWP to the portable library so you can consume it. To test the portable library, use a regular non-UWP unit test project:

Note - to make this work you'll need to change the target framework of the unit test project from 4.5.2 (which is the default) to 4.6.
Any unit test you run from the regular unit test project will run as fast as possible because it doesn't need to run in the AppContainer. I tested this out and the inner loop speed was great. Hope that helps!


Bayesian Statistics: Pearl Oyster Problem

Question: Problem:
You're collecting exotic oysters, and there are two different bays from which you could harvest oysters.
    • In both bays, 11% of the oysters contain valuable pearls and 89% are empty.
    • In the first bay, 4% of the pearl-containing oysters are blue, and 8% of the non-pearl-containing oysters are blue.
    • In the second bay, 13% of the pearl-containing oysters are blue, and 26% of the non-pearl-containing oysters are blue.
Would you rather have a blue oyster from the first bay or the second bay?
I have solved the problem, but I am not sure if its correct. Please comment if possible!
My Solution:
Comment: reopened from "off-topic" since this did already show an attempt. Do you have a specific question  about your attempt you'd like addressed?
Comment: I will say I'd have been inclined simply to calculate P(Pearl|Blue, Bay1) and P(Pearl|Blue, Bay2) directly and compare the two conditional probabilities to see which was larger, rather than take the ratio (not that there;s anything wrong with that idea)
Answer: You appear to have prematurely rounded at one point in each calculation and then discovered a tiny difference.

How do you know that apparent difference isn't simply rounding error?
Consider writing your answers as exact fractions.
(Alternatively) Both the calculations are special cases of the following problem:

11% of the oysters contain valuable pearls and 89% are empty. In a bay, a proportion $q$ of the pearl-containing oysters are blue, and $2q$ of the non-pearl-containing oysters are blue.

Find an expression for P(Pearl|Blue) for this problem. Divide numerator and denominator by $q$. What do you notice?


Quando "x >= y" e "x <= y" não se comportam como "x > y || x == y" e "x < y || x == y"?

Question: Como o null pode ser maior ou igual a 0 e menor ou igual a 0 ao mesmo tempo em que não é igual, menor e nem maior que 0?

console.log(null >= 0); //true
console.log(null <= 0); //true
console.log(null == 0); //false
console.log(null > 0);  //false
console.log(null < 0);  //false
Comment: É um ótimo exemplo pra adicionar na lista dos porquês que fazem o JavaScript ser uma linguagem ruim :P
Comment: Veja a "mágica" acontecer executando: `+null `...
Comment: vale a leitura: https://pt.stackoverflow.com/questions/77051/por-que-1-true-%c3%a9-verdadeiro-e-2-true-%c3%a9-falso/77057#77057 - https://pt.stackoverflow.com/questions/154169/se-boolean0-%c3%a9-true-e-boolean0-%c3%a9-false-porque-0-false-%c3%a9-true-em-javas?noredirect=1&lq=1
Comment: Alterei o titulo, porque o problema não é exatamente sobre NULL, mas sobre o comportamento de `x >= y` e `x <= y` ser diferente de `x > y || x == y` e `x < y || x == y` em **alguns casos**. @Matheus se você discordar da edição você ainda pode [edit] ou [reverter a edição](/posts/593268/revisions).
Answer: Este comportamento está bem descrito na documentação de cada operador. Seguem os links abaixo para referência:

Operador
Link documentação

==
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Equality

<
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Less_than

>
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Greater_than

<=
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Less_than_or_equal

>=
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Greater_than_or_equal

E vale ler também a especificação oficial da linguagem para uma explicação mais detalhada (embora bem mais complexa de entender).

Cada link acima tem a explicação do que acontece quando um dos operandos é null. Mas de forma geral, temos dois casos principais: o == e os demais.
No caso do ==, o algoritmo usa a operação IsLooselyEqual, descrita na especificação da linguagem. De forma resumida, uma das condições diz que se um dos operandos é null, o operador só retorna true se o outro operando for null ou undefined. Por isso que comparar null com zero (ou qualquer outro valor) resulta em false.

Já os demais operadores usam a operação IsLessThan para comparar os valores. O algoritmo é bem complexo, mas um dos casos - conforme descrito na documentação - diz que se um dos operandos for null, este é convertido para 0. Por isso que null < 0 é false, pois no fim é feita a comparação 0 < 0. Isso tem como consequência que 0 < null também é false.
O operador > usa este mesmo algoritmo, apenas invertendo a ordem dos operandos - ou seja, x > y equivale a y < x. E como 0 < null é false, então null > 0 também é.

Mas se null não é igual a zero e também não é maior e nem menor que zero, então por que ele é menor ou igual e maior ou igual a zero?
Basicamente "porque sim". A documentação do operador <= diz o seguinte:

When one of x or y is null, and the other is something that's not null and becomes 0 when coerced to numeric (including 0, 0n, false, "", "0", new Date(0), etc.): x <= y is true

Ou seja, se um dos operandos é null e o outro não é, mas quando convertido para número resulta em zero, o resultado é true. E null <= 0 cai neste caso: um dos operandos é null e o outro não, e este quando convertido para número resulta em zero (neste caso nem precisa converter, pois já é o próprio zero). Por isso que o resultado é true. A mesma regra vale para o operador >=.
E por que é assim? A resposta pode ser frustrante, mas basicamente é porque decidiram que seria assim. Cada linguagem define seus operadores da forma que bem entender, faça sentido ou não. Em algumas, por exemplo, o código nem compila se tentar comparar null com um número. Mas em JavaScript decidiram que null >= 0 e null <= 0 retornam true, mesmo que null == 0, null < 0 e null > 0 retornem false. A linguagem é cheia dessas inconsistências e maluquices, então minha sugestão é testar se os operandos possuem valores válidos antes de fazer qualquer operação com eles (a menos que você saiba o que está fazendo, claro).
Vale lembrar que todos esses comportamentos estranhos foram decisões arbitrárias, e não importa muito se eu ou você ou qualquer outra pessoa achar que não faz sentido. É assim porque definiram que é assim e pronto.
Comment: +1, por citar a documentação e por descrever de forma clara, um bom conteudo para o site.
Comment: @Benilson nesse caso pode referenciar as boas respostas que já existem no site, porque o problema da pergunta não é sobre ==, é sobre `x >= y` se comportar diferente de `x == y || x > y` em alguns casos específicos, então citar o === não teria relação, por isso no máximo referenciar como extra, mas não como relacionado. PS: Inclusive estou pensando em mudar o titulo da pergunta atual, para ficar claro que a dúvida é sobre o comportamento dos operadores similares, mas vou esperar a opinião da comunidade.
Comment: +1, boa resposta, talvez seja válido citar como resolver o problema convertendo os dados antes de comparar ou usando `===`, conforme a necessidade.


Where's the rest of the stacktrace?

Question: I have a test that is raising an error. To track down the problem I ended up adding this method to a model called NodeAffiliation:
def initialize a1, a2
  raise "kaboom"
end

and then I get this error:
RuntimeError: kaboom
app/models/node_affiliation.rb:13:in `initialize'
    test/unit/audit_test.rb:10:in `__bind_1318003437_24401'

but audit_test.rb is doing this:
Factory.create :form

Somehow, creating a Form also creates a NodeAffiliation, but those steps seem to be missing in the backtrace. Any ideas why and/or how to get them?
Answer: The test logs might already have the stacktrace, but if not you can call
logger.debug $!.backtrace.join("\n")

where $! is the default name of the exception that was raised. This needs to be in a rescue block. I would check your factory implementation, it is likely associating node_affiliation with form objects, or perhaps there is a chain of relations. Any associations declared in the factory get created when the object is created.

Where's the rest of the stacktrace?