Ambiguity of Roman numeral analysis of Andalusian cadence

Question: When I write that the Andalusian cadence is the chord progression "Am – G – F – E" then there are no doubts in what it means. (All you need to know is what chords are and how they are named)
The definition is exact and everybody knows you can use instead, say, "A♯m - A♭ - F♯ - E♯" with the same harmonic progression effect (I intentionally used mixed sharps and flats that you would probably never use but it is still exact - everybody knows what chords to play).
On the other hand when roman numerals are used to describe the Andalusian cadence then you may find that the Andalusian cadence is:
iv – III – II – I
or
iv - III - ♭II - I
or
iv – ♭III – ♭II – I
or
i – VII – VI – V
or
i – ♭VII – ♭VI – V
So my question is what is the point of using roman numerals when in fact it tells you nothing about what harmony to use unless you are provided with additional information of what is meant by each degree of roman numerals? And this information is what actual chord to play instead of each degree. So you would save time just by providing the chords in the first place instead of any numerals.
The same ambiguity can be found in Roman numeral analysis. We can see:

So they defined that mediant is "♭III" (conventional notation) or "iii" (alternate notation). And then in just the next step they broke their own rules and used "III" as mediant - already a third version of how to label the mediant in a natural minor scale. The same applies for the submediant.
Isn't roman numeral notation just useless?
Since all that is necessary to unambiguously define harmony in the natural minor scale is this:
Cm - Ddim - E♭ - Fm - Gm - A♭ - B♭
which immediately is enough information to figure out that also this can be used (one of many):
Am - Bdim - C - Dm - Em - F - G
Comment: The Wikipedia article is not necessarily correct. If it seems confusing to you, it might be because of the article, not because Roman numerals themselves are off. I've been taught in a minor key the median is just III. Neither iii nor bIII make sense for the minor mediant except for in jazz where bIII sort of makes sense. The mediant in a minor key is a major chord, so it should never be iii.
Comment: @azerbajdzan there are very poorly written articles on wikipediea and the music theory related ones tend to be pretty bad and mix up different schools of thought making them hard to parse for people that don't know the area of study. What's slightly more concerning is based on your comments to this question and the answers you have preconceived notion about Roman Numeral Analysis and you are using this question to push it. It like every system has pros and cons, but it's very very useful. As for ambiguity, have you seen how many different chord symbols there are?
Comment: It just shows that roman numerals notation is ambiguous - it is not universal. It depends on what you studied and where. On the other hand Am is universal. Am=ACE everywhere. Same way chord progression "Am – G – F – E" is universal. But using iv – III – II – I or i – ♭VII – ♭VI – V to describe Andalusian cadence is depended on what your music school was. PS: I do not think that the articles on Wikipedia were written by incompetent people. It just shows they went to different schools.
Comment: There's nothing stopping you nor I from writing articles on Wikipedia, so we can't expect anyone putting stuff on Wikipedia to know more than we do. I very much doubt there is a school anywhere that describes a major chord with lower case Roman numerals.
Comment: 'you can use instead, say, "A♯m - A♭ - F♯ - E♯"': onlee iff yu arre vairy tolirunt uv por spellng.
Comment: @phoog: Except that my "poor spelling" is perfectly clear in what to play in contrast with your poor spelled comment that needs to be deciphered.
Comment: @azerbajdzan you seem to be confusing "unambiguous" with "perfectly clear." Yes, it's unambiguous, but no, it's not perfectly clear, because it's confusing. Many musicians would need to stop and decipher it instead of being able to read it at sight.  Roman numeral analysis is also generally unambiguous, but it requires context.  Without that context, it _is_ ambiguous just as the statement "she ate it" is ambiguous if you don't know what the antecedents of "she" and "it" are. Yet people use pronouns successfully all the time.  Even this question omits context given in the Wikipedia article.
Comment: Examples of this context include "with respect to the Phrygian mode" (where one does not include a flat sign on II because that is an intrinsic feature of the mode) and "with respect to the Aeolian mode (minor)."  You say "it tells you nothing about what harmony to use unless you are provided with additional information," but when it is used correctly, you _are_ provided with additional information.  (Also, as others have pointed out, Roman numeral analysis isn't intended to communicate music to performers; you're judging a hammer on its usefulness for driving screws.)
Comment: I personally have no problems to sight read written chord progressions on guitar even if they are written in mix of several keys. But on the other hand I would never be able to sight read music sheets - I can read them slowly - like counting from position of note C to see what a note it is - so really a better words would be I am deciphering them. But I have no proper music education.
Answer:
Isn't roman numeral notation just useless?

I don't believe so. Whereas strict chord labels (C, Dm, etc.) tell you exactly what chords to play, Roman-numeral notation can tell you:

how these chords relate to an overall tonic,
what tonic even is,
what the functional role of these chords are within the current harmonic environment,
all of this information in a more streamlined manner than "Dm7♭5, acting as a predominant",
etc.

Furthermore, having a key-agnostic labeling system makes transposition far easier, since reading

I V7/iii iii IV V7/V V7 I

and performing it in all twelve keys is a far easier task than parsing out

E G♯7 C♯m A F♯7 B7 E

and transposing by a given interval.
As for whether an E♭-major chord in C minor is III or ♭III, I don't really think that's ambiguity, that's just sloppy work; the accidental preceding a Roman numeral lowers the root from the given key signature, so this chord should be III in C minor (not ♭III), but it would be ♭III in C major. (And, if I may: would someone actually confuse this with E♭♭ major?)
Comment: @azerbajdzan But I would argue that's not ambiguity, that's sloppy labeling. Depending on the given use of the Andalusian cadence, one decides whether the first chord or the last is tonic. With that information, then one decides which set of Roman numerals to use.
Comment: @azerbajdzan I didn't get into this yet, but as far as I know, only one of these examples can actually be considered the Andalusian cadence (the one that goes from i to V). The others might similar in their motion, but are not the "same cadence."
Comment: The second bullet point is a strong enough argument for me!
Comment: @Theodore: So tell me then what tonic is in Andalusian cadence??? If one tells the cadence is iv – III – II – I and another tells you it is i – ♭VII – ♭VI – V. That is the ambiguity of the notation I am talking about.
Comment: @azerbajdzan It depends on the composition and that is exactly the point.
Comment: @Richard: You wrote "how these chords relate to an overall tonic". I do not agree if Andalusian cadence can be described as  iv – III – II – I or as i – ♭VII – ♭VI – V. You would even do not know what chords to play. On the other hand Am - G - F - E describes relationship among chords precisely.
Comment: @azerbajdzan Perhaps another thing I didn't clarify: the information provided by Roman numerals is *not objective*. At least, not always. "Am-G" tells you exactly what notes to play, but the only thing it tells you about their "relationship" is objective data: how far apart they are, their chord qualities, etc. Roman numeral analyses might be something that two different people could argue over (when they get complicated); they help communicate how someone has *analyzed* the music. No, they don't tell you what notes to play unless you also say what the key is.
Comment: @azerbajdzan (The Muscle Shoals guitarist in my example, if he were handed a page full of roman numerals and told "Play," would have to respond "Sure! What key?")
Answer: To add to or expand on Richard's answer: You're running into the distinction between analysis and practice. To get really basic: humans make music. They also put ink on paper to say things about that music. Sometimes the goal of this ink is to help them make music—notated pitches and rhythms help us recreate musical ideas. Sometimes the goal is to talk about the music or explain it.
Confusion can arise because some things, like chord names, roman numerals, and figured bass, are used at various times for both purposes, for making music and for analyzing it. Roman numerals are usually a tool of music theory, but I know a guitarist who worked in the Muscle Shoals studios during their heyday who says they would write out chord charts in Roman numerals rather than letter names so they could transpose at will. Figured bass is usually used today along with Roman numerals in harmonic analysis, but was used in performance in the baroque.
So today, a chord notation like "Am" is often used in performance—"Play some combination of the pitches A, C, and E." It can also be used in theoretical analysis (I often start by sketching these out before asking myself what's going on tonally). But "i" or "III" tell you something different; they tell you about the tonal function of the chord. This is where analysis really becomes analysis. You could have a given chord, a certain combination of pitches, and explain it in various ways, based on a big-picture view of what's going on: Is this a pivot chord because we're modulating? A "V of V" that tonicizes the dominant? A chord that is spelled one way but really works in another because of harmonic motion? (Like, in the key of Am, let's say you have an F chord, then an Am over E, then an E. The "Am over E" isn't really a iv in its own right; it's a suspension as we go from F to E.)
For a metaphor: say I find a large, flat stone. I could identify it objectively: "It's a large flat stone." But I could put it in front of my front door and say "it's a doorstep," or on top of some cabinets in the kitchen and say "it's a kitchen counter," or stand it on end, request to be buried in front of it, and say "it's a tombstone." All the time it's still a large, flat stone, but these roles help explain it, and tell us about the larger story.
Letter-name notation of chords tells you what to play, but without an understanding of the tonal context, it doesn't tell you the chord's "role" in the harmony. On the other hand, Roman numerals tell you about the chord's role, but unless you also know the current tonality, they don't actually tell you what to play!
Comment: @azerbajdzan By "role," I mean that "i" tells you "this is tonic." In language, we can use the same words in two different contexts and have different meanings: "I'm sorry" might mean "I apologize" or "I feel your sorrow" or "what did you say?". The roman numerals provide this context: there's a big difference in the musical "meanings" of iv-III-II-I vs i-VIII-VI-V (I wouldn't use those flats): one "brought us home," and now we have cadenced and are "settled." The other "started from home," and brought us to the dominant, where we are "unsettled."
Comment: @azerbajdzan You seem to be focusing on a "performance use"—given some roman numerals, to know the chords. That can happen (my Muscle Shoals example), though it's not that common. More common is the opposite: given some chords, to decide on their roman numerals, as an act of analysis. Also: I'd be more likely to say "iv-III-II-I; play it for me *in the key of E*" than "start with Am." If I said "start with Am," I'm asking you to figure out "in what key is Am the iv chord?" Which I just did in my head just now to figure out "E", but it took quite a few seconds.
Comment: You wrote: "On the other hand, Roman numerals tell you about the chord's role". So what tells me iv – III – II – I and what tells me i – ♭VII – ♭VI – V? Really nothing because you do not even know what relation is between those degrees unless additional information is provided.
Comment: If I say you. Here you have chord progression "iv – III – II – I", play it for me. Start with Am. What would be the next three chords? Then here you have "i – ♭VII – ♭VI – V", play it for me, star again with Am. What would be next three chords? I am sure each musician would come up with different sequence of chords, because this notation is ambiguous if additional information is not given. But If I give you "Am – G – F – E" and play it for me but start on Cm instead of Am and preserve the same relations between chords everybody would play: "Cm - Bb - Ab - G".
Comment: @azerbajdzan __no__. you get the same chords as the key is what matters. Am is vi in the key of C major and i in the key of A minor and iv in E major since the 3rd is lowered. If you don't understand that, you need go back and look at the basics of Roman Numeral analysis
Comment: So there we have it. You admitted that "iv – III – II – I"  and "iv" is Am is not enough information for you to decide what next three chords would be. That is what I talked about from the very beginning. Additional information is needed  such as what is the KEY. But not only the key but also to what mode it is referenced to and also such things that there are not universal rules whether to use, say, flats or they can be omitted like ♭VII or VII to referring to the same chord.
Comment: @azerbajdzan you are missing the entire point of Roman Numeral Analysis. It's an abstraction/reduction of tonal harmony which if you are working on tonal pieces the key is needed. You literally can't use Roman Numeral Analysis on non tonal systems. You are pretty much saying a screwdriver is useless without having something to screw in and I'd agree, but you're not using the tool correctly. Also you missed the point. If someone says Am is i, you know everything else you need.
Comment: @azerbajdzan (But yes, "'iv – III – II – I' and 'iv' is Am" *is* in fact enough to know what chords to play. I'd probably say "Why can't you just tell me the key," and "Wait, why do we have major II and III but minor iv???" And I just realized I was wrong; this would be Am, G#, F#, E; that was why the flats were there. But this just shows how weird some of these sequences that have been suggested get. "Am" describes a set of pitches objectively; you could also call it "B double flat minor" and sound the same, but you'd have to have a good reason for doing that. Similarly,...
Comment: @azerbajdzan ... in "i-bVII-bVI-V" there's a reason for the flats; it's natural minor. There's no easy-to-imagine explanation for lowered second and third scale degrees, though, aside from dorian mode. Bottom line, you could play the exact same pitches whether you call Am i or iv; the difference is that you would call it one or the other depending on what you'd decided the key and mode are (including what's "happening to them" at that moment).
Comment: Bingo. So you said: "iv-III-II-I" play it for me in the key of E" thinking you would play Am - G - F - E. But then realized that you would probably play Am - G# - F# - E. And this is the ambiguity of roman numerals. Even the key is not sufficient to determine what to play. Because someone uses numerals in relation to major scale, someone in relation to minor scale, someone in relation to particular mode of the scale, someone even though the music may be in minor key but he relate the numeral in relation to major key. This is where all those  sequences went from iv – III – II – ", i-bVII-bVI-V.
Comment: ... and other sequence in my OP. They are all interpreted form different key/scale/mode...
Comment: @azerbajdzan Sorry, I just got so confused by this bizarre series of roman numerals that I forgot the significance of those flats. The fact is, some of these are just so *unlikely* that it's hard for me to wrap my head around. I'm not an expert on the Andalusian cadence, but my impression is *only* the "i ... V" version qualifies. All the others might be similar in that they descend by the same amounts and have the same qualities, but some are hardly even "cadences" let alone the Andalusian progression. And one more time: despite the Muscle Shoals example, no, roman numerals are not a very...
Comment: @azerbajdzan ... efficient way of telling people what to play. They are primarily used in music theory analysis.
Comment: @Andy Bonner: My explanation why someone might use i-bVII-bVI-V to describe Andalusian cadence is this: We have major scale first mode I - ii - iii - IV - V - vi - vii(dim). Which in key C is C - Dm - Em - F - G - Am - Bdim. How to modify this degrees to get Andalusian cadence Cm - Bb - Ab - G? We change first degree from major to minor, i.e. I to i, lower the seventh degree and change its quality to major, ie. vii(dim) to bVII, lower the sixth degree and change it to major, ie. vi to bVI and finally fifth degree unchanged ie. V to V - so we get i-bVII-bVI-V.
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/141307/discussion-between-andy-bonner-and-azerbajdzan).
Comment: I accepted your answer, as you are the only one who focused on the actual question and I guess there will be no better answer. :-)
Comment: @azerbajdzan Thanks, though really as I see it Richard has the "correct" answer, and was certainly more efficient than mine; I was just expanding it.
Answer: All the things you add to your question about "ambiguity" boil down to saying "I don't know what key I am in."
Am – G – F – E
When you identify the progression by name, and list those chords, unless you give some other reason to the contrary, the key is A minor, and then the Roman numeral analysis (RNA) is not at all ambiguous. It is...
Am: i VII VI V
If you don't specify the key signature in the RNA, as you should, or you're presenting RNA derived symbols in a pop/jazz context, then people often neglect to give the key signature, so the mode isn't specified, and so you need to use sharps/flats to ensure the correct chord roots are understood...
i ♭VII ♭VI V
Sharps and flats can be used along with a key label to show borrowed chords, but that is another matter.
The ambiguity of this particular point isn't about the RNA system, it's about failing to give the key label in RNA. Am: VII is not ambiguous, it's a chord rooted on the seventh scale degree of the key signature for A minor, which is G natural.
If you leave out the key label, and write just i VII or i ♭VII, will assume you mean play a root progression of a descending whole step, both chords are root position, the first one minor, the second one major, in any key. The first RNA is careless, the second makes me think you learned it in a pop/jazz setting, both strangely ambivalent about key when using an RNA systems that is based upon the idea of identifying chords withing keys!
The other examples that rotate all the chord identities through the various diatonic positions, like iv – III – II – I, just show an ignorance about what is the tonic. RNA is a system for analyzing tonal music, so disregard or ignorance for the tonic is just a fundamental error in musical understanding. It is not an ambiguity of RNA.
This one - A♯m A♭ F♯ E♯ - displays an ignorance of both keys and the significance of enharmonic spellings. Again, those are fundamental musical misunderstandings of whoever would write that, not an ambiguity of RNA.

There is one aspect of Roman numeral analysis that is most definitely ambiguous, but the OP's example absolutely does not highlight it. The ambiguity is with the seventh scale degree in minor keys. In a minor key signature the seventh scale degree is a whole step below the tonic, and so not a proper leading tone, which is one half step below the tonic. Normally there is no accidental added to the Roman numeral to show the raising of the seventh degree to for a leading tone chord. If you add to that convention, the fact that some RNA systems don't specify chord quality by letter case, and some don't use o for diminished chords, you find RNA for the seventh scale degree is indeed ambiguous. Am: VII in some RNA is implicitly understood to mean a leading tone, diminished triad. Probably more common is Am: viio. But, you don't normally see Am: ♯viio or Am: ♮viio, which strictly speaking would be correct when accidentals are used to show roots altered from the key signature.
Answer: Given that your first 3 examples are probably not correct, as the last chord shown will not be tonic, those are out of the equation.
Of the last 2, both showing that i is tonic, and V is dominant - no argument there - i-VII-VI-V makes most sense, with particular regard to the relative minor of C major, A minor. I would have no problem translating that to the Andalusian sequence, which it portrays nicely. There's no need to travel into harmonic or melodic minor notes, just like there's no need to consider Dorian or Phrygian modes. With the obvious exception that to actually have a leading note, that note needs to be one semitone below the root, so merely using notes diatonic to C/Am won't work well. Although there are Andalusian sequences that I've heard using v rather than V.
By keeping it simple, with chords directly related to diatonic notes, it all works well, no need for any complications, as you seem to have discovered. So, RN would work just fine, thanks!
Comment: Just because it's on the 'net doesn't make it gospel!
Comment: Have you read the links in my OP? All the examples are from Wikipedia article, they are not mine. I guess the article was written by people who knows what Andalusian cadence is.
Comment: @azerbajdzan "I guess the article was written by people who know what Andalusian cadence is": anyone can contribute to a Wikipedia article.  They're mostly pretty good, but sometimes they include woefully incorrect information.
Comment: @phoog: Same as here, dear phoog, same as here. Have I already said same as here? :-)
Comment: @azerbajdzan true enough.  This format, though,  allows users to gauge quality of the information more easily, as well as to engage in a bit of back-and-forth with those answering the questions, so as to form their own opinions about others' assertions.  With Wikipedia the best clue that something is correct is usually a citation of some other authoritative source, but unfortunately in academic matters such as music theory these are often copyright-protected publications that are not freely available online, and most of us are not going to go to a library to verify them.
Answer: The other answers are good and cover everything, but I wanted to add one to try and be crystal clear in highlighting what you are missing.
By default there is a SPECIFIC harmony to use for each degree in roman numeral analysis (RNA), unless specified otherwise. The harmony, ie. which chords each degree of the scale specifies, can be major or minor. In a minor key the RNA will place the tonic minor on the i, a major key will place the tonic major chord on the I. Note that in minor keys the existence of 'natural' and 'harmonic' minor versions could indeed introduce ambiguity on the quality of the vii scale degree, but rest of the RNA system is NOT ambiguous.
Roman numeral analysis expresses chord relations without NEEDING to specify a key (although saying/being clear it is of type major or minor should be a minimum), but a key must be chosen in order to play it. It is useful because it describes the harmony of a piece in a way that can be applied to any given tonic root.
For example, a popular jazz tune has chords;
ii, V, I, IV, vii, iii, vi
Knowing this piece expressed in RNA I can play it in any key very easily and without having to transpose. In fact, knowing songs in this way rather than explicit chords is highly recommended, and in certain genres (jazz, some pop) a required skill. I appreciate that may not be your goal, but for context.
If I learned a song in roman numeral analysis then I know it in every key immediately, I just centre the I chord around the given tonic and play the relationship. This is incredibly useful for both analysis and performance. Analysis cares not what the starting note is, more the complex relationship between notes. It helps us, a little like algebra, to look at the generalised relationship and behaviour of a system rather than one specific example.
Also, each chord in roman numeral analysis tells me the 7th quality of the chord, and the further upper extensions that are available, it is much more than just knowing which chords are major and which minor.
Expressing the Andalusian cadence in RNA using the minor tonic as i gives us i, VII, VI, V. You could also express that chord sequence against a major key as vi - V - IV - III7. The second way may be useful if that particular chord progression appears in a piece that otherwise can be considered to be in a major key.
RNA is incredibly useful, it can demystify complex sequences of harmony in a way that would be tricky to spot from a chord sequence alone, it gives us information about which chord extensions will sound most consonant, it allows us to change key with ease, it allows us to see compositional tools used across various sections, keys and pieces for comparison and greater understanding.
If you are familiar with programming, using chord symbols is like 'hard coding', very explicit and clear, but rigid and not that translatable. Using roman numeral analysis is learning to understand the dynamic system we are operating in. It means that sometimes there are not hard and fast rules, but an expert on the system will quickly recognise the patterns and could express how it would most likely be interpreted.
The examples given in the wikipedia article are not that useful, there is rarely cause to respell RNA against a modal root. It is perhaps of historical curiosity if you are comfortable with modal harmony and have a strong basis in RNA, but I can see how it could be very confusing. Once one is comfortable with using RNA and can see why the Andalusian cadence is usually going to be i VII VI V etc. then the counter examples in the wiki article become very easy to comprehend, not in danger of destabilising the use of RNA, but also of very limited use. Though perhaps they could inspire new compositions, any alternative viewpoint can be the basis for a new exploration. For this reason it's important not to regard theory as an absolute ruleset, to be adhered to at all times. However this doesn't mean that the well established systems we have are not very thorough and complete tools.
Comment: I didn't down vote, but this statement "...based on the diatonic major scale..." is not really true, and I updated my answer to address it. Roman numerals indicate _scale degrees_ used for _chord roots_. I think that is the simplest way to say it. The scale _may_ be major, but it also may be minor. And in minor the RNA for the seventh scale degree is a bit ambiguous, but definitely not simply based on a diatonic major scale. You might say the default RNA scale for minor is a harmonic minor scale.
Comment: You only confirmed that roman numerals are not universal. You are used to how jazz musicians use roman numerals - based on your music education and music genre you are engaged in. But I guarantee you if you and some professional flamenco player got the same progression in roman numeral without any additional information and you were told to play it in key C that you two would play completely different chord progressions. Not different in a sense like to play G instead of G7 were dominant should be played but in a sense that one would play, say, G and the other Gb in the same place.
Comment: Other answers in this thread also disagree that vi - V - IV - III7 is the best/easiest (or even correct) way to express Andalusian cadence. Other answers suggest that the first chord should be "Im" i.e. tonic and the last chord should be V i.e. dominant. You do not even have any tonic in your progression and have V on a place that it even do not act like a dominant.
Comment: Based on that on "iv – III – II – I" you would play  "Am – G# – F# – E" I guess that on "iv – III – bII – I" you would play "Am – G# – F – E" but a flamenco player who takes everything in relation to Phrygian dominant would play on "iv – III – bII – I" the actual Andalusian cadence "Am – G – F – E". So you two would clash in one playing G# and the other playing G on the same roman numeral notation. But if you were given straight at the beginning to play "Am – G – F – E" you would have no problems to agree what to play. Because this is universal notation.
Comment: @azerbajdzan I am keeping my roman numeral analysis entirely against a major key for ease, the tonic in my progression is the vi, which is the one chord in a minor key. To your second comment, I'm suggesting the same as the others. I guarantee you that if a professional flamenco player and I got the same progression we would establish a tonic in two words and then play the same chord sequence. You are correct that when translating between conventions using the explicit chord notation avoids all ambiguity. But chord naming is not universal either, it's also specific to a certain area of music.
Comment: The explicitly written chord sequence would make no sense to a professional Japanese koto player because they don't use a system where Amin would imply a chord of stacked thirds. So yes, moving between areas of music with their own traditions of harmony will require extra explanation. I can still, with my training, analyse a Japanese piece (for example) as both chords and roman numerals and use that to describe to another trained win in theory. In this case still the roman numeral analysis would give me more information about the structure of the piece, and would not be that ambiguous.
Comment: Neither chords nor roman numeral analysis are sufficient for describing.communicating all music across the globe. Using both will make more sense the closer you get to the musics that initially used/developed them. Chords are explicit descriptors for those that know the language, but you need to know more than just the chord symbols to interpret what's going on in relation to the key. Roman numeral analysis tells you what's going on in relation to the key directly and allows cross-key analysis but is arguably less suited the further away you get from the tradition that invented the system.
Comment: Lastly, you don't need a definitive answer to where the Andalusian cadence lie on a scale in roman numeral analysis, in composition it will be quite common to be in one key and treat it one way, then play it again, but this time think of it as in a DIFFERENT key and use it to pivot to a new key centre, for example. There is FAR too much to go into here in a comment, but it doesn't need to be set in stone, it's just four chords, lay it over where it fits on the key for you, call it vi-V-IV-IV7 or whatever you want, next time call it something else and change key, its just a map, not rules.
Comment: One significant information from your comment: "I am keeping my roman numeral analysis entirely against a major key for ease". This is the first time this information was mentioned here. It is not even on the Wikipedia page "Roman numeral analysis". If this is the case that we always relate roman numerals to major key then I could agree that it could be universal. But why nobody already answered it that way? If I got one sentence answer "keeping roman numeral analysis entirely against a major key" I would probably accept it immediately, But nobody mentioned it!
Comment: Ahhhh ok... yeah it's often written that way as people who know the system can generally recognise if the chord sequence is entered around vi or I and decide wether its major or minor from that (if vi, minor, vi is the relative minor key). It's arbitrary (we could use the minor numbers as default) but it's just the one chosen. If you are diving more specifically into analysing a know minor song then you can use i to mean the minor tonic and analyse from there. I that case the Andalusian cadence as I have suggested is i VII VI V.
Comment: This is useful for music that is fully ascribed to the western tradition, but because of the very specific features of that music (the different minor scales, the minor V chord being turned dominant, as in the Andalusian cadence sometimes) I'd argue that the roman numeral analysis using the minor tonic as 'i' is potentially a little more confusing when applied to other musics, hence we often default to describing things just sticking to the major scale version.
Comment: @MichaelCurtis, yes, absolutely right, I'll edit to try and make my answer a bit better. I was focusing more on getting across to the OP that RNA is not useless and on reading back it's not that much of a good answer! I was going to delete as yours explains it much more clearly and in many less characters, but I'll see if I can just tidy it up first..


Render ref dynamically with variable

Question: I've a function that maps an object and return a list of items dynamically. Now i wish to access the inputRef for each of this element's list to apply some effect, but it is only being applied on the last and current element, and not on the rendered items
This is the rendering function:
renderSortByOptions() => {
  return Object.keys(this.sortByOptions).map((sortByOption) => {
    let sortByOptionValue = this.sortByOptions[sortByOption];
    return (
      <li
        className={this.getSortByClass(sortByOptionValue)}
        key={sortByOptionValue}

        ref={this.inputRef} // <====

        onClick={this.handleSortByChange.bind(this, sortByOptionValue)}
      >
        {sortByOption}

    );
  });
};

The ref is happening on the componentDidUpdate like this:
componentDidUpdate();
  {
    if (this.state.showMenu === true) {
      let words = ["viejos", "nuevos", "menor", "mayor"];
      words.forEach((item, i) => {
        this.styleWord(this.inputRef.current, item);
      });
    }
  }

And this is just the styleWord function:
styleWord(target, word) {
  let html = target.innerHTML;
  html = html.replace(new RegExp(word, "g"), "" + word + "");
  target.innerHTML = html;
};

Comment: Can you share a sandbox?
Answer: You need to InputRefArrary for holding reference of multiple inputs that you created dynamically. Here is an example. Please check it.
import React from "react";

export default class MultipleTextField extends React.Component {
    constructor(props) {
        super(props);
        this.inputArr = ["name", "age", "gender"];
        this.inputRefArr = [this.inputArr.length];
        this.inputArr.map((input, index) => {
            this.inputRefArr[index] = React.createRef();
        });
        this.state = {values: [this.inputArr.length]};
        this.inputRef = React.createRef();
    }

    onChange = (event, index) => {
        // 1. Make a shallow copy of the values
        let values = [...this.state.values];
        values[index] = event.target.value;
        this.setState({values}, () => {
            console.log(values, 'values')
        });
    };

    clickHandler = (event) => {
        this.inputArr.map((input, index) => {
            alert(`${input} is ${this.state.values[index]}`)
        })
    };

    render() {
        return (
            <form>
                {
                    this.inputArr.map((input, index) => {
                        return (

                                    {input}
                                    <input key={index} ref={this.inputRefArr[index]} id="message" type="text"
                                           value={this.state.values[index]}
                                           onChange={(event) => this.onChange(event, index)}/>

                        );
                    })
                }
                <input type="button" value="Click Me" onClick={this.clickHandler}/>
            </form>
        );
    }
}

Comment: Hi Khabir, thanks for your reply, i understand the idea but still can't apply it on my example, i'll continue checking :)

Render ref dynamically with variable


openmpi: MPI_recv hangs for specific number of processes

Question: I am running a HPC benchmark (IOR - http://sourceforge.net/projects/ior-sio/) on lustre. I compiled the source of IOR and running it with openmpi 1.5.3.
The problem is that it hangs when the number of processes (-np) is less than 6, which is odd. Removing all other things I do with around, the actual command that I run comes down to this:
/usr/lib64/openmpi/bin/mpirun --machinefile mpi_hosts --bynode -np 16 /path/to/IOR -F -u -t 1m -b 16g -i 1 -o /my/file/system/out_file

Attaching the process to GDB shows that process hangs at MPI_recv:
#0  0x00007f3ac49e95fe in mlx4_poll_cq () from /usr/lib64/libmlx4-m-rdmav2.so
#1  0x00007f3ac6ce0918 in ?? () from /usr/lib64/openmpi/lib/openmpi/mca_btl_openib.so
#2  0x000000385a6f0d5a in opal_progress () from /usr/lib64/openmpi/lib/libmpi.so.1
#3  0x00007f3ac7511e05 in ?? () from /usr/lib64/openmpi/lib/openmpi/mca_pml_ob1.so
#4  0x000000385a666cac in PMPI_Recv () from /usr/lib64/openmpi/lib/libmpi.so.1
#5  0x0000000000404bd7 in CountTasksPerNode (numTasks=16, comm=0x628a80) at IOR.c:526
#6  0x0000000000407d53 in SetupTests (argc=11, argv=0x7fffe61fa2a8) at IOR.c:1402
#7  0x0000000000402e96 in main (argc=11, argv=0x7fffe61fa2a8) at IOR.c:134

This problem happens only when -np is 2/3/4/5. It works for 1/6/7/8/16 etc.
I can't reproduce this problem if I use simple commands such as date or ls. So I am not sure if this is a problem with my environment or IOR binary that I compiled (very unlikely because the same happens with an older/stable IOR binary too).
Also the precise behaviour is observed when using openmpi1.4.3 instead of openmpi1.5.3.
I have also tried by using various number of hosts (--machinefile argument) and same behaviour is observed for the the above mentioned -np values.
The source line it hangs is this:
MPI_Recv(hostname, MAX_STR, MPI_CHAR, MPI_ANY_SOURCE, MPI_ANY_TAG, comm, &status);

Basically I am looking for clues as to why MPI_recv() might hang when -np is 2/3/4/5. Please let me know if other information is needed. Thanks.
Comment: It seems to be an InfiniBand-related problem. `mlx4_poll_cq` is called to poll the completion queue of the IB HCA and since it hangs in a receive operaiton, it means that the other side (e.g. another rank) is not completing the send. Check your InfiniBand equipment and kernel parameters. Or, even better, post your question to the [Open MPI Users](http://www.open-mpi.org/community/lists/ompi.php) mailing list.
Comment: I suspected that too. I checked the HCAs locally and they seemed OK. I am not an IB expert and I'll ask some help from my h/w engineers. I'll give another try on Monday and take it from there. Thanks for your time and helpful suggestions :)
Comment: IOR has been around a long long time.  I doubt the fault lies with IOR, though IOR or any file system benchmark are quite good at uncovering hardware faults.   You might find  IOR from github has a bit better reporting: https://github.com/chaos/ior
Answer: First thing that comes to mind: MPI_Recv is a blocking receive, and will wait until a matching MPI_Send is called. However, if what you are sending is small enough (i.e., it fits in the buffer that MPI sets aside for such tasks), then the function will actually not wait, and instead carry on through the code. For higher core counts, you may be sending less with each MPI_Send/MPI_Recv command, so the data fits in the buffer and everything continues on its way. With lower core counts, there is too much data to fit in the buffer and MPI_Recv hangs as you have not called an appropriate MPI_Send for the information to get there. A quick and easy way to test this hypothesis: decrease the problem size substantially; does it still hang at all of those core counts? If not, then that is further evidence for my hypothesis and you will need to provide more code so we can see what the issue is.
Comment: hmmm...but if doesn't hang with 1 process, which contradicts with this suggestion. It's true that the data I send is quite large (order of GBs). I'll try with various sizes, including very small values. Source can be seen in the link in question. It's too much of code, really and I am not familiar with it completely to produce a SO-fit-problem-reproducible code. Thanks.
Comment: @BlueMoon: It doesn't hang with 1 process because you shouldn't be sending anything at all when you only have 1 processor (nothing to send to), so it should return regardless of problem size.
Comment: Right. I'll try with various sizes with multiple processes then.


Help Solving Textbook Heat Conduction Laplace Transforms PDE Problem

Question: I am trying to solve the following problem:
$$\dfrac{\partial{\phi}}{\partial{t}} = \dfrac{\partial^2{\phi}}{\partial{x}^2} - \cos(x), \ x > 0, t > 0$$
$$\phi(x, 0) = 0, \ x > 0$$
$$\phi(0, t) = e^{-t}, \ t > 0$$
Taking the Laplace transform (in $t$, of course) gives
$$\mathcal{L} \left\{ \dfrac{\partial{\phi}}{\partial{t}} \right\} = \mathcal{L} \left\{ \dfrac{\partial^2{\phi}}{\partial{x}^2} - \cos(x) \right\}$$
$$\therefore s \mathcal{L}\{\phi\} - \phi(0) = \dfrac{d^2 \mathcal{L} \{ \phi \}}{dx^2} - \dfrac{\cos(x)}{s}$$
And since $\phi(x, 0) = 0$, we have
$$s \mathcal{L} \{ \phi \} = \dfrac{d^2 \mathcal{L} \{ \phi \}}{dx^2} - \dfrac{\cos(x)}{s}$$
This is an ODE with constant coefficients.
From now on, let $\mathcal{L}\{ \phi \} = \bar{\phi}$ (for the sake of conciseness).
So our constant-coefficient ODE is
$$\dfrac{d^2 \bar{\phi}}{dx^2} - s \bar{\phi} = \dfrac{\cos(x)}{s}$$
This is a inhomogeneous ODE and, judging by the inhomogeneous term $\dfrac{\cos(x)}{s}$, can be solved using the method of undetermined coefficients.
The homogeneous version of the ODE is
$$\dfrac{d^2 \bar{\phi}}{dx^2} - s \bar{\phi} = 0$$
The characteristic polynomial is
$$m^2 - s = 0$$
$$\therefore m = \pm \sqrt{s}$$
Therefore, the complementary equation for this inhomogeneous ODE is
$$y_c(x) = Ae^{x\sqrt{s}} + Be^{-x\sqrt{s}}$$
And our particular solution for this inhomogeneous ODE is
$$y_p(x) = -\dfrac{1}{s^2 - s} \cos(x)$$
Therefore, our solution to the inhomogeneous ODE is
$$y(x) = Ae^{x\sqrt{s}} + Be^{-x\sqrt{s}} - \dfrac{1}{s^2 - s} \cos(x)$$
Therefore, we have that
$$\bar{\phi}(x, s) = Ae^{x\sqrt{s}} + Be^{-x\sqrt{s}} - \dfrac{1}{s^2 - s} \cos(x)$$
Here is where I am stumped.
HOWEVER, one way that I have seen other problems proceed is by assuming another boundary condition:
$$\lim\limits_{x \to \infty} \phi(x, t) = 0$$
This makes sense from a physical standpoint, but it was not specified in the problem statement, so I am unsure if it is valid for me to use it?
Also, I don't have full solutions for these problems, so I have no way of checking the intermediate steps.
The final solution should be
$$\phi(x, t) = \text{erfc}\left( \dfrac{x}{2\sqrt{t}} \right) - \cos(1 - e^{-t})$$
I would greatly appreciate it if people could please help me solve this problem.

EDIT: If we proceed with assuming the extra boundary condition (that is, if we proceed with assuming that $\lim\limits_{x \to \infty} \phi(x, t) = 0$), then we continue from above as follows:
Now, since $\lim\limits_{x \to \infty} \phi(x, t) = 0$, we have that $Ae^{-x \sqrt{s}} = \dfrac{A}{e^{x \sqrt{s}}} \to 0$ as $x \to \infty$, and $- \dfrac{1}{s^2 - s} \cos(x)$ does not exist as $x \to \infty$, since $\cos(x)$ will not converge.
So what do we do from here? It seems that the $- \dfrac{1}{s^2 - s} \cos(x)$ term is causing us problems?
IGNORE EVERYTHING BELOW THIS POINT

$$\bar{\phi}(x, s) = \int_0^\infty \phi(x, t) e^{-st} \ dt,$$
we have that $\bar{\phi}(x, s) \to 0$ as $x \to \infty$, and hence $B = 0$, since $Ae^{-x \sqrt{s}} = \dfrac{A}{e^{x \sqrt{s}}} \to 0$ as $x \to \infty$.
Therefore, we now have
$$\bar{\phi}(x, s) = Ae^{-x \sqrt{s}}.$$
We now use the given boundary conditions.
Letting $x = 0$ in the Laplace transform of $\phi$ gives
$$\bar{\phi}(0, s) = \int_0^\infty \phi(0, t)e^{-st} \ dt = \int_0^\infty e^{-t}e^{-st} \ dt = \int_0^\infty e^{-t(1 + s)} \ dt$$
$$= \dfrac{-1}{1 + s} \int_0^{-\infty} e^u \ du = \dfrac{1}{1 + s},$$
since $\phi(0, t) = e^{-t}$ for all $t > 0$.
Comment: @Ian Do you mean something is inconsistent with my work or the problem itself? Also, what do you mean by $\phi(x,0)=\lim_{s \to 0} s \overline{\phi}(x,s)$? I do not understand where this comes from.
Comment: Sorry, disregard, I made a silly mistake.
Comment: @Ian Oh, ok. So does my work seem correct? I wonder if it's the problem itself that is inconsistent?
Comment: $y_p$ should be $y_p = \frac{1}{s(s+1)}\cos(x)$ because $\cos'' = -\cos$
Comment: @DisintegratingByParts Are you sure? I had it as that, and then was told that it should be $- \frac{1}{s(1+s)}$. See Dylan's answer in https://math.stackexchange.com/questions/2896937/solving-the-ode-fracd2-bar-phidx2-s-bar-phi-frac-cosxs
Comment: @DisintegratingByParts Also, do you happen to know how to solve this problem by any chance?
Answer: Let $\widetilde{\phi}(x,s) = \int_{0}^{\infty}e^{-ts}\phi(x,t)dt$ be the Laplace transform of $\phi(x,t)$. As you said we have the following ODE for $\widetilde{\phi}(x,s)$:
$$
\dfrac{\partial^2{\widetilde{\phi}(x,s)}}{\partial{x}^2} -s\widetilde{\phi}(x,s)- \frac{\cos(x)}{s} = 0
$$
The general solution is simply:
$$
\widetilde{\phi}(x,s)=Ae^{\sqrt{s}x}+Be^{-\sqrt{s}x} -\frac{\cos(x)}{s(s+1)}
$$
where $A$ and $B$ are two arbitrary constant that need two condition to be determined. The first condition is a physical one: the solution must be limited in time and so $A=0$.
The second condition is that:
$$\phi(0, t) = e^{-t}, \ t > 0 .$$
Now the Laplace transform of $\phi(0,t)$ is simply $\widetilde{\phi}(0,s)=\frac{1}{1+s}$ and so we have $B = \frac{1}{s}$. Finally we have that the Laplace transform of our $\phi(x,s)$ is:
$$
\widetilde{\phi}(x,s)=\frac{e^{-\sqrt{s}x}}{s} - \frac{\cos(x)}{s(s+1)}
$$
Now we have to transforming back. I don't know if you know complex analysis. If not, simply check a Laplace transform's table and see that the inverse Laplace transform of $\frac{e^{-\sqrt{s}x}}{s}$ is $\text{erfc}(\frac{x}{2\sqrt{t}})$ and the inverse L.T. of $\frac{1}{s(s+1)}$ is $1 - e^{-t}$ and so the solution is:
$$ \phi(x,t) = \text{erfc}\left( \dfrac{x}{2\sqrt{t}} \right) - \cos(x)(1 - e^{-t}).
$$
If you know complex analysis you have to perform the following complex integral:
$$
\phi(x,t) = \frac{1}{2{\pi}i}\int_Ldze^{zt}\widetilde{\phi}(x,z)
$$
where L is a vertical line in the complex plane on the right of all singularity of $\widetilde{\phi}(x,z)$. Be careful due to the presence of the square root: is a multivalued function.
Answer: Let $\Phi(s,x)=\int_{0}^{\infty}\phi(t,x)e^{-st}dt$ be the Laplace transform in time of the desired solution. Then the transform of the heat equation is
$$
    \int_{0}^{\infty}e^{-st}\frac{\partial\phi}{\partial t}dt
  =\int_{0}^{\infty}e^{-st}\frac{\partial^2\phi}{\partial x^2}dt-\cos(x)\int_{0}^{\infty}e^{-st}dt.
$$
Expanding this out gives
$$
   e^{-st}\phi(t,x)|_{t=0}^{\infty}-\int_{0}^{\infty}(-se^{-st})\phi(t,x)dt \\
  =\frac{\partial^2}{\partial x^2}\int_{0}^{\infty}e^{-st}\phi(t,x)dt-\left.\cos(x)\frac{e^{-st}}{-s}\right|_{t=0}^{\infty}.
$$
Therefore,
$$
   -\phi(0,x)+s\int_{0}^{\infty}e^{-st}\phi(t,x)dt=\frac{\partial^2}{\partial x^2}\int_{0}^{\infty}e^{-st}\phi(t,x)dt-\frac{\cos(x)}{s}.
$$
Let $\Phi(s,x)=\int_{0}^{\infty}e^{-st}\phi(t,x)dt$. Then $\phi(0,x)=0$ gives
$$
     s\Phi(s,x)=\Phi_{xx}(s,x)-\frac{\cos(x)}{s} \\
      \Phi_{xx}-s\Phi=\frac{\cos(x)}{s}.
$$
A particular solution is $C\cos(x)$. But $\cos''=-\cos$, which gives $C(-1-s)=1/s$ or $C=-1/(s(1+s))$. So the general solution is
$$
      \Phi(s,x)=A\cos(\sqrt{s}x)+B\sin(\sqrt{s}x)-\frac{\cos(x)}{s(1+s)}.
$$
Now you have to transform back in $s$.


Como plotar um gráfico ao encerrar um script de visão computacional no Raspberry pi?

Question: Eu tenho um código usando opencv Raspberry Pi e ele está executando um reconhecimento de padrões de loop. No final encerro o script usando "Ctrl + X" sem nenhum problema! Eu quero ao mandar "parar" o processamento gráfico, executar uma função para traçar um gráfico das informações de reconhecimento, como posição, tamanho e outros parâmetros.
Como posso fazer isso?
No código de exemplo eu coloquei a chamada de função quando parar .. mas não obtive sucesso. Nome da função é "Relatorio_Movimentos
<code>while True:

// outras funções de reconhecimento, loop do processamento

if cv.WaitKey(10) == 27:
 break
Relatorio_Movimentos(Parado, Aj_Sup, Aj_Inf, Dir, Esq, Frt, Trs)
cv.DestroyAllWindows()
gpio.cleanup()
</code>
Comment: Tente adicionar um cv::waitKey(0); depois da função que gera o relatório e antes de destruir as janelas. Talvez o relatório esteja executando certinho mas o programa encerra antes que você possa ver o resultado...
Comment: Crie o Relatorio_Movimentos em uma thread. E mantenha a aplicação original esperando pelo ponto de parada. Se acontecer, mate a thread.
Answer: <code>// Se o usuário pressionar a tecla ESC
if cv.WaitKey(10) == 27:
    // Exibir o relatório
    Relatorio_Movimentos(Parado, Aj_Sup, Aj_Inf, Dir, Esq, Frt, Trs)

    // Observe que se a função acima exibir uma janela do OpenCV, você deve invocar:
    //cv.WaitKey(0)
    break
</code>


What sources of uncertainty are included in a (95%) confidence interval?

Question: Today, I came across an interpretation of 95% confidence interval (thereafter "CI"). The text has been written from education science perpective saying that, beside a (1.) random sampling error of randomly selected cases from a given population, CI also accounts for "(2.) different conditions in which the test was administrated and (3.) which tasks were included into a test (4.) etc."
I am bit confused now, since I have never seen that CI could also account for this other sources of uncertanaity (lets say all of them have stochastic character). Is my knowledge about CI not complete or the author just misinterpreted the meaning of CI?
Thank you in advance.
Comment: A procedure to produce a CI can, on the face of it, correctly account *only* for the uncertainty represented in the underlying statistical model.  Thus, *given a suitable model* along with a *demonstration of its applicability,* a CI procedure *can* account for any or all of (1) through (4).  However, to the extent to which any of these potential sources of variation exist, they can cause the data to vary--and most CI procedures will estimate (correctly or not) the magnitude of that variation and incorporate it within their results.
Comment: On a slightly cynical note (but one amply grounded in experience), one shouldn't look too closely at any account of statistical concepts or procedures written from an education science perspective.  For careful, accurate, and rigorous explanations, look to the statistics literature itself.
Comment: Can you please include (as an edit to the Q, not only as a comment)  a link or reference to that text?
Comment: Continuing: "If the testing took place on a different day, if other pupils were present, or if a test with a different order of assignments were to be taken, the schools could achieve a slightly different result than they recorded in this particular survey. The range in which their result would most likely be found during repeated testing is expressed by the confidence intervals." (Now the second part is obv. wrong interpretation, but it is for another discussion.)
Comment: Those are helpful quotations.  *Provided* the dataset has sampled *all* these different conditions in a way that can be rigorously construed as *random,* then a CI might reflect all those forms of variation. But, for instance, if testing occurred on just one day, how could a CI possibly reflect variation over time, which *a fortiori* has not occurred within the dataset?
Comment: I am sorry, but I am not able to provide a link for this citation. The text hasn't been writter in english, but in rather "minority" language, therefore I do not think it will be particulary helpful. At least I can provide my translation: "The measurement of students' knowledge ... through tests is always associated with a certain statistical error, which is given by what tasks were included in the test, what conditions prevailed on the day of testing, students participated in testing, etc. To capture the uncertainties associated with testing ... [use ] 95% confidence intervals."
Answer:
(1.) random sampling error of randomly selected cases from a given population...

Random sampling error is what most statistical methods are about.

The other sources of error,

...also accounts for "(2.) different conditions in which the test was administrated and (3.) which tasks were included into a test (4.) etc."

, are not part of the random sampling procedure*. These are more like systematic errors. When confidence intervals are computed/estimated then these type of errors are typically not accounted for (e.g an interval based on the t-statistic does not).

* At least they are often are not part of the sampling. You can make it part of the random sampling by performing tests with different settings like changing conditions and tasks. But the text using a phrase like 'beside a (1.) random sampling error...' seems to regard these errors implicitly as different types of error that are not random sampling error. Potentially it could be that they see them as different from 'random error' as they can be seen as 'random effects' and the text might wants to put them aside because of that? There is some duality to random effects and they can be seen as both part of the deterministic part of the model as well as the random part of the model. For these potential subtleties it is better when you provide a link to the original text, even when it is not English.


redirecting url path to port

Question: I have many web apps running on a server.  Some of them I have made, some of them are open source (such as open-cloud).  All my web apps run on a specific port.  My port 80 is still free.
I would like to make a nice welcome web page to all my apps on the port 80 to allow my co-worker to see them on the local network.
My problem is to setup the urls map.  I would like the path http://machine-name:80/open-cloud/ to be the exact same thing then http://machine-name:8080/ (including the static files if possible).
One other possible solution would be to use sub-domaines such as http://open-cloud.machine-name:80/ for each application.  But I do not know how to do that inside a local network.
I am looking for a solution in Python, node.js or Go (that would be awesome!).  It is important to notice that none of my web app runs on ssl and that it will probably handle very low trafic.  It is only a local network after all!
Any help and reference would be welcome.
Cheers!
Comment: Why write such a program when you can do this with something like nginx? BTW, it's trivial (only tens of lines of code) to write in Go.
Answer: If you insist on writing your own solution in Go, you could setup a http.Redirect for each URL
like this:
import "net/http"

var urlMap = map[string]string {
    "open-cloud": "http://machine-name:8080/",
    "some-other": "http://machine-name:9090/",
}

func redirectHandler(target string) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        http.Redirect(w, r, target, 301)
    }
}

func main() {
    for key, target := range urlMap {
        http.HandleFunc("/" + key, redirectHandler(target))
    }

    http.ListenAndServe(":80", nil)
}

However, I would rather rely on a tool like nginx as joewhite86 already proposed.
Comment: I agree on nginx and this is what I will use.  Nevertheless I will look at your solution too because I need to get into Go more seriously!
Answer: You just need some kind of proxy, like http://nginx.org for example.
Then pass the domains you want to your application ports:
server {
  listen 80;
  server_name app1.local;

  location / {
    proxy_pass          http://127.0.0.1:8080/;
    proxy_set_header    HOST $http_host;
    proxy_set_header    X-Real-IP $remote_addr;
  }
}

Comment: Thanks a lot for that one!  It solves my problem.  I know it a basic question but I had a hard time finding an elegant solution for it.

redirecting url path to port


Metrics of Forecast Accuracy that are "fair" with respect to forecasting difficulty

Question: Let's say we want to evaluate and then compare the accuracy of forecasts across different variables for the same country, or the same variable across different countries. Are there ways to account for situations where forecasting of one indicator/country is inherently more difficult than the other, so that we can compare the forecast accuracy in a meaningful way?

Example 1: Growth of business investment is typically more volatile than GDP growth within the same country. Both indicators are flow variables and, as such, they are more "difficult" to forecast than stock variables such as the unemployment rate.

Example 2: GDP growth in emerging economies is typically more volatile than in big mature economies. Compare for example Turkey and the US. Moreover, in most cases, there is typically more and better data available for the big, mature economy, that the forecaster can use, which makes it easier to forecast that economy.

Note that I am not interested in model based forecast evaluation. Knowledge of how the forecasts are derived should not be required.
One strategy might be to divide both forecast and realization by the standard deviation of the realizations. Is that a valid strategy? Are there others?
Answer: The standard way to do this is to compare your forecast performance metric to the same metric evaluated on a benchmark model (as a ratio, say). The result is an evaluation of forecasting "skill", or what you've brought to the table above and beyond what a naïve model can do with the same data (i.e. at the same level of difficulty).
A commonly used benchmark is the random walk, but depending on context it might be fair to use a different model (e.g. white noise, or include seasonality, etc). The point is to evaluate the simplest sensible forecast that could be made without any real model selection effort. You shouldn't use a benchmark that's obviously wrong to make your own forecasts appear as great improvements, obviously. If there is a standard benchmark in the field, use that.
For both the candidate forecast and the benchmark, the forecast performance metric should be computed out-of-sample so you can have a real idea of how the candidate forecast improves performance where it matters.
Considering your proposed strategy, if the standard deviation of the realizations factors out of the performance measure in question (e.g. MAE, RMSE), your idea can be seen to be essentially equivalent to:

Using a white noise process as the benchmark ($Y_t = \mu + \varepsilon_t$, with $\varepsilon_t$ being iid)
Using in-sample performance for the benchmark
Using RMSE for the benchmark performance

The most problematic is point 3 because it doesn't necessarily use the same metric for the candidate forecast and the benchmark forecast, so they might not be directly comparable. Point 2 is not really an issue although the interpretation is slightly different; there are existing measures that use in-sample results for the benchmark, like MASE. Point 1 is fair, especially for growth rates of economic data. If you were looking at GDP in levels, a random walk benchmark would be more appropriate.
So, your strategy may be ok in certain contexts, but the wider point of view of comparing the same out-of-sample performance measure from the candidate forecast and a benchmark forecast will work in a wider variety of contexts.


Speaker impedance connected in parallel for the power mixer

Question: I have Behringer PMP4000 power mixer. I am considering buying two passive fullrange speakers Behringer B1520 Pro Eurolive.
According to the speakers specification they are

8 ohm, with power rating 300 W RMS and 1200 W peak.

Power mixer has to "powered" outputs. According to specification (page 20) it can output:

8 ohm, 300 W RMS and 400 W peak per channel
4 ohm, 600 W RMS and 800 W peak per channel

Thus, to my understand the two speakers fit the setup perfectly.
However, I consider getting a passive subwoofer or a passive monitor as well (the mixer supports a special mode where the powered outputs volume can be controlled separately). Let's for an example consider this subwoofer (again, we are doing it purely for exercise purpose, I am not aiming for this model). According to the specification the subwoofer:

8 ohm, with power rating 300 W RMS and 1200 W peak.

Do I understand correctly, that if connect the two speakers in parallel (that is connect mixer to one speaker, and connect the second speaker to the first one) and add the mentioned subwoofer, the power mixer should handle everything? To my understanding, two connected speakers will drop the impedance to 4 ohm while increasing the power need twice (300 W RMS * 2 = 600 W RMS). And the other mixer power output can still feed the subwoofer with 8 ohm and 300 RMS? Or both mixer outputs should have the same impedance?
Can someone also clarify the demo picture on page 14 in the mixer specification? They connect in parallel two Behringer B1800X (8 ohm, 450 W RMS, 1800 W peak) and Behringer B1220 (300 W RMS, 1200 W peak). To me it does not make much sense since the power mixer is not powerful enough to power that setup.
Comment: Not answering your question, thus a comment. Having a 'monitor' that comes off one side of a stereo powered mixer isn't that productive. Yes, it changes the impedance for one side, but also renders the 'monitor' uncontrollable. Far better to get a powered monitor, and feed it from 'monitor out'. That way, it's independent from the main p.a., and feedback for one won't be such a problem.
Comment: O.k. so there are options to use the mixer as 2 separate channels, but daisychain the monitor to one speaker, *or* plug both foh speakers to one side, and the monitor to the other channel. That'll work, but the 1st idea makes the impedance on 1st channel 4 ohms. For the sake of a bit extra cost, a powered monitor is, in my experience, a far better arrangement. 3 separate power amps and more controllability!
Comment: @Tim thanks a lot for the comment. I am aware about this. The mixer has something called "MON" mode (see page 9 or page 15). Output A (one powered output) goes to monitors and is controlled via separate controls and Output B (another powered output) goes to the main speakers and is controlled via separate slider.
Answer: Two things:
1) yes, within limits, the impedance of two identical objects wired in parallel will be half the impedance of each one.
2) As to "not powerful enough," all that means is you can't go full-blast for the given scenario.  The only important thing to watch is not to let the net impedance go lower than the lowest rated impedance for your source (power amp).  Even then, so long as you don't try to generate huge gain, the worst that'll happen is output distortion.  If you do turn the gain way up, you risk having the amplifier trying to generate more current than it's capable of (because it tries to maintain a voltage level), leading to overheating, component stress, etc.


I get this error when I use npm install npm ERR! Windows_NT 10.0.19041

Question: I'm getting this error when I run npm install, even when I try to upgrade my npm I get this error
npm ERR! Windows_NT 10.0.19041
npm ERR! argv "C:\Program Files\nodejs\node.exe" "C:\Users\jaydev\Desktop\Testreact\node_modules\npm\bin\npm-cli.js" "i"
npm ERR! node v14.15.3
npm ERR! npm  v3.10.10
npm ERR! cb.apply is not a function
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     C:\WINDOWS\system32\npm-debug.log
Answer: I'm using node v12, I have the same error, but I fix it by:

go to C:\Users\username find and delete folder '~

go to your\project\folder find and delete folder '~

re-open command prompt

install again

I'm not sure that it work for you your node version, But it work for me, I hope that can help you.
Comment: wait what folder am I suppose to delete?
Comment: @JahmarLawrence What was your solution? I'm not seeing that folder either.


Error was encountered Severity: ilegal string offset Codeigniter

Question: in view, I try to call kriteria and sub kriteria:
I try to call id_kriteria, nama_kriteria kriteria from tb_kriteria for tittle, and then I call id_kriteria, namasubkriteria for option sub kriteria
Model:
public function GetSubKriteria() {
        $this->db->select('*');
        $this->db->from('tb_kriteria');
        $query = $this->db->get();
        $krit = $query->result();
        $data = array();
        $index = 0;
            foreach($krit as $kr_key => $x){
                $data[$kr_key] = array(
                    'id_kriteria'           => $x->id_kriteria,
                    'nama_kriteria'         => $x->nama_kriteria,
                );
                $id_kriteriax = $this->db->where('id_kriteria', $x->id_kriteria);
                $this->db->select('*');
                $this->db->from('tb_subkriteria');
                $query = $this->db->get();
                $id_kriteriax = $query->result();
                foreach($id_kriteriax as $z){
                    // $idKriteriaKey = $z->id_kriteria;
                    $data[$kr_key]['subkriteria'] = array(
                        // 'id_kriteria'           => $x->id_kriteria,
                        // 'nama_kriteria'         => $x->nama_kriteria,
                        'id_subkriteria'        => $z->id_subkriteria,
                        'namasubkriteria'       => $z->namasubkriteria,
                        'value'                 => $z->value
                    );
                }
                $index++;
            }
            return $data;
            // var_dump($data);
    }

view:
<?php
    $index =0;
    foreach ($subkriteria_ as $kr_key => $kriteria) {
    ?>

        <?php echo $kriteria['nama_kriteria'] ?>

                <input type="hidden" name="status_pemeliharaan" value="1" class="form-control">
                <input type="hidden" name="id_kriteria[]" value="<?php echo $kriteria['id_kriteria'] ?>" class="form-control">
            <select class="form-control"  name="isi_kriteria[]" required>
                <option>Pilih Sub Kriteria</option>
                <?php $no = 1; foreach ($kriteria['subkriteria'] as $data)     { ?>
                <option value="<?php echo $data['value'] ?>"><?php echo $data['namasubkriteria'] ?></option> // this my error
                <?php  $no++; } ?>
            </select>

    <?php } ?>

but i try this code, now show the error at the view, Error was encountered Severity: ilegal string offset 'value' blabla
and if i try to var_dump($data); this array is just show one subkriteria, not all subkriteria data show in the array
thx for your response...
Answer:
You should check whether data occurs or not using empty() in controller and view

public function GetSubKriteria() {
    $this->db->select('*');
    $this->db->from('tb_kriteria');
    $query = $this->db->get();
    $krit = $query->result();
    $data = array();
    $index = 0;
        foreach($krit as $kr_key => $x){
            $data[$kr_key] = array(
                'id_kriteria'           => $x->id_kriteria,
                'nama_kriteria'         => $x->nama_kriteria,
            );
            $id_kriteriax = $this->db->where('id_kriteria', $x->id_kriteria);
            $this->db->select('*');
            $this->db->from('tb_subkriteria');
            $query = $this->db->get();
            $id_kriteriax = $query->result();
            if(!empty($id_kriteriax)){
            foreach($id_kriteriax as $z){
                // $idKriteriaKey = $z->id_kriteria;
                $data[$kr_key]['subkriteria'] = array(
                    // 'id_kriteria'           => $x->id_kriteria,
                    // 'nama_kriteria'         => $x->nama_kriteria,
                    'id_subkriteria'        => $z->id_subkriteria,
                    'namasubkriteria'       => $z->namasubkriteria,
                    'value'                 => $z->value
                );
            }
            }
            $index++;
        }
        return $data;

View

<?php
$index =0;
foreach ($subkriteria_ as $kr_key => $kriteria) {
?>

    <?php echo $kriteria['nama_kriteria'] ?>

            <input type="hidden" name="status_pemeliharaan" value="1" class="form-control">
            <input type="hidden" name="id_kriteria[]" value="<?php echo $kriteria['id_kriteria'] ?>" class="form-control">
        <select class="form-control"  name="isi_kriteria[]" required>
            <option>Pilih Sub Kriteria</option>
            <?php $no = 1;
            foreach ($kriteria['subkriteria'] as $data)     {
            if(!empty($data)){
            echo '<option value="'.$data['value'].'">'.$data['namasubkriteria'].'</option>
            <?php  $no++; } } ?>
        </select>

<?php } ?>

Comment: this problem is solved, i have updated code on this line $data[$kr_key]['subkriteria'][] = array(, thx for your response broo


Different predictions for GAMMs with ordered vs. unordered factors

Question: I've fitted 2 different binomial GAMM models, both with a smooth for a predictor Timesec that varies by Condition (a categorical factor). In the first, Condition was a nominal, unordered factor:
bam(cbind(FixTarget, NFixations - FixTarget)
~ 0 + Condition
+ s(Timesec, by=Condition, bs="tp")
+ s(Participant, by=Condition, bs="re")
+ s(Participant, Timesec, by=Condition, bs="re"),
data = by.subject,
family = "binomial",
discrete = TRUE)

In the second, Condition was an ordered factor, and as such, I have also included a 'simple' smooth for Timesec:
bam(cbind(FixTarget, NFixations - FixTarget)
~ 1 + Condition
+ s(Timesec, bs="tp") + s(Timesec, by=Condition, bs="tp")
+ s(Participant, bs="re") + s(Participant, by=Condition, bs="re")
+ s(Participant, Timesec, bs="re") + s(Participant, Timesec, by=Condition, bs="re"),
data = by.subject,
family = "binomial",
discrete = TRUE)

The first model estimates separate smooths for Timesec for the levels of Condition, whereas in the second we estimate a smooth for the reference level of Condition and additional difference smooths, relative to the reference level (see GAM factor smooth interaction--include main effect smooth?).
What I didn't expect was that the predictions from these two models differ (see image): in the model with an ordered factor (bottom left), the predictions for the non-reference level has a much wider CI than in the model with an unordered factor (top left).

I was expecting these two models to be equivalent but with different parametrizations. But it seems the uncertainty about the difference smooth in the ordered factor model is carrying to the non-reference level. Why so and what am I doing wrong?
Also note: In both of these models, there were by-Participant random effects. I've fitted separate models without random slopes, in which I removed s(Participant, Timesec, by=Condition, bs="re"). These models show consistent CIs for ordered vs. unordered models (see right column in the image). This seems to be a relevant piece of the puzzle.
Finally, for completeness, predictions were obtained excluding the random effects, as such (after creating a data frame with new data called predictions.df):
mterms <- sapply(m$smooth, "[[",  "label")
excluded.terms <- mterms[grep("Participant", mterms)]

predictions <- predict(m, newdata = predictions.df,
                       type = "link",
                       se.fit = TRUE,
                       exclude = excluded.terms)

EDIT
Following Gavin's suggestion below, I have run models with factor smooths, instead of with linear adjustments.
The unordered factor model:
bam(cbind(FixTarget, NFixations - FixTarget)
         ~ 0 + Condition
         + s(Timesec, by=Condition, bs="tp")
         + s(Timesec, Participant, by=Condition, bs="fs", m=1),
         data = by.subject,
         family = "binomial",
         discrete = TRUE,
         nthreads = parallel::detectCores() - 1)

The ordered model:
bam(cbind(FixTarget, NFixations - FixTarget)
         ~ 1 + Condition
         + s(Timesec, bs="tp") + s(Timesec, by=Condition, bs="tp")
         + s(Timesec, Participant, bs="fs", m=1)
         + s(Timesec, Participant, by=Condition, bs="fs", m=1),
         data = by.subject,
         family = "binomial",
         discrete = TRUE,
         nthreads = parallel::detectCores() - 1)

The problem remained. In the ordered model, the reference level still yields smooths with smaller CIs, the non-reference level yields larger CIs:
Answer: Isn't this extra uncertainty due to uncertainty in the estimated random slopes and the potential for lack of identification of the linear basis in each smooth and the linear random slopes you introduced to the model.
I suspect the problem is worse in the ordered factor model because now the linear basis in the ordered-factor by smooth is representing deviations from the global (or average) smooth and the random slopes are modelling deviations in another linear function about the linear basis function in the global smooth.
I'm not convinced you should have random slopes in a model where you also include smooth functions of the covariate that is part of the random slopes. Why not allow for random smooths (say via a bs = "fs" smooth)?
Comment: Many thanks, Gavin. I have now fitted models with factor smooths (see edit), but the problem remains. (I was using these "linear random slopes" because I wanted a more constrained model in which each subject varied little from the average smooth).


How to prove $(1+q)^n \geq1+qn$ for all $\mathbb{N}$ with $q>0$

Question: I need to prove $(1+q)^n \geq1+qn$ for all $n \in \mathbb{N}$ with $q>0, q\in \mathbb{R}$ using mathematical induction.
Firstly, I proved the base case for $n=1$ that indeed $1+q\geq1+q$.
Then in the inductive step, I started with $(1+q)^{k+1}$ and came to the following form $(1+q)^k(1+q)\geq(1+qk)(1+q)$. However I'm uncertain how to explicitly show that $(1+q)^{k+1}\geq1+q(k+1)$ because after the expansion I get the $kq^2$ term that I'm unable to get rid of.
I would appreciate any help! Thank you!
Comment: $kq^2>0$ since $q>0$ :-)
Comment: @RadialArmSaw yes, $n \in \mathbb{N}$
Comment: To clarify,  $n\in {\mathbb{N}}$, correct?
Comment: Normally, in math writing,$n$ and $N$ are different variables. Do you mean them to be the same?
Comment: Or maybe you meant "for all $n\in\mathbb N.$
Comment: Thank you for your replies! Yes, I meant natural numbers. I have $1+q(1+k)+qk^2$  in my notes but is knowing $q^2k\geq0$ enough to just discard the term from the right side?
Comment: @DrMaps so you are restricting $n$ to natural numbers?
Answer: You can actually do the whole same process by admitting that $q>-1$. The calculations could be organized a little more neatly:
$$\begin{align*}
(1+q)^{k+1}&=(1+q)(1+q)^k\\
&\ge(1+q)(1+kq)\\
&=1+(k+1)q+kq^2\\
&\ge 1+(k+1)q\;,
\end{align*}$$
since $kq^2\ge 0$. This completes the induction step. This result is known as the Bernoulli's inequality.


Which ports do I need to open in the firewall to use NFS?

Question: I'm running Ubuntu 11.10 - setting up NFS to share a directory among many other servers. Which ports are required to be opened on the firewall?
Comment: It depends on the version of the protocol you intent to use. NFS 4 only require 2049 while older versions require more.
Answer: $ rpcinfo -p | grep nfs

Port 111 (TCP and UDP) and 2049 (TCP and UDP) for the NFS server.
There are also ports for Cluster and client status (Port 1110 TCP for the former, and 1110 UDP for the latter) as well as a port for the NFS lock manager (Port 4045 TCP and UDP). Only you can determine which ports you need to allow depending on which services are needed cross-gateway.
Comment: I did not know about rpcinfo, that is quite useful. I didn't see the port 111 with the `grep nfs`, but I left off the grep to learn that 111 is for `portmapper`. Also good to know! (and like you mentioned, necessary)
Comment: You need mountd open when you first mount the filesystem. It runs on a dynamic port, so it won't always be the same. I put a link to a guide in my answer.
Comment: Could you give a brief description of what those services ("cluster and client status", "NFS lock manager") do and when they might be needed? If those services are blocked by a firewall, will NFS servers and clients degrade gracefully, or just lock waiting for a response?
Comment: @KennyYounger rpcinfo is a useful too. If this answered your question, don't forget to mark it as the answer for the benefit of future viewers.
Comment: Thanks @bonsaiviking. Critical info for this process. I upvoted your answer!
Comment: Nathan Craike: Not brief, but take a look at `man rpc.nfsd`, `man rpc.mountd`, `man rpc.statd`, `rpc.idmapd` and so on. There is a detailed article on tldp.org: http://tldp.org/HOWTO/NFS-HOWTO/security.html#FIREWALLS
Comment: Which server is initiating the connection? The server where you define the export or the server which does the mount?
Comment: on BSD systems, 111 is for rpcbind but im sure it serves same purpose as portmapper
Answer: In addition to 111 for portmapper and 2049 for nfs, you will need to allow the mountd port and possibly rquotad, lockd, and statd, all of which can be dynamic. This excellent NFS security guide recommends changing your startup scripts and kernel module configs to force them to use static ports.
In addition to the guide above, which has a section on firewalls, see my answer to another question about hardening NFS.
Comment: We really do prefer content, not pointers to content. A précis of the content with a link is ok too and preferred to a link.
Comment: Not sure about 2012 but the recommendation in links seems to be grossly outdated. NFS over TCP is preferred for variety of reason and NFS over UDP can cause silent data corruption on fast link due to protocol limitations. Security guide has no mention of Kerberos/GSS etc.
Comment: With respect to mountd, life is a lot easier if you reconfigure to ensure the mountd port is fixed. On debian systems, edit the RPCMOUNTDOPTS value to something like `RPCMOUNTDOPTS="--port 34567"` and then restart with `sysctl --system` and `/etc/init.d/nfs-kernel-server restart`
Answer: I found useful directions for my problem on this page, but there was no easy to follow recipe. So here's my recipe.
TL;DR - need to allow both nfs ports (111, 2049) and mountd port after fixing it.
Instructions:

Setting up a fixed port for mountd
gksudo gedit /etc/default/nfs-kernel-server

comment out this line: RPCMOUNTDOPTS=--manage-gids
add this instead: RPCMOUNTDOPTS="--port 33333"

Or any other port number.
now try to reset nfs using:
sudo service nfs-kernel-server restart

And test if it helped using:
rpcinfo -p | grep "tcp.*mountd"

For me it wasn't enough, but a full restart fixed the issue.
(credit)

Setting up the firewall (ufw)
(1) delete old rules, do this manually or reset if this is the only use for the firewall:
# WARNING: Don't copy & paste this if you don't understand what it does:
# sudo ufw reset
# sudo ufw enable

(2) add nfs & mountd ports
sudo ufw allow in from 10.0.0.1/20 to any port 111
sudo ufw allow in from 10.0.0.1/20 to any port 2049
sudo ufw allow in from 10.0.0.1/20 to any port 33333

(Change to your local IP's or to "any" instead of 10.0.0.1/20)
That's all there's to it.
Comment: You don't need to restart the entire box. A simple `sudo service nfs-config restart` before restarting the `nfs-kernel-server` will do just fine.
Comment: @jchook That looks factual to me!
Comment: @fbicknel I think you _can_ do it in one "swell foop" if you just remove the `proto tcp` part on the first command :)
Comment: Or if your system uses `systemctl`, it's `systemctl restart nfs-kernel-server.service`.
Comment: You can also do it all in one swell foop: `sudo nfs allow proto tcp from 10.0.0.1/20 to any port 111,2049,33333` followed by (ok, two swell foops) `sudo nfs allow proto udp from 10.0.0.1/20 to any port 111,2049,33333`. Note port 33333 is per the above example. You may use another port as the answer indicates.
Comment: @fbicknel Do you mean `ufw`?
Answer: This will give a list of all ports used by all NFS-related program:
rpcinfo -p | awk '{print $3" "$4}' | sort -k2n | uniq

Answer: With FERM one can use Backticks to get the ports from rpcinfo, for example:
Server:
proto tcp {saddr ($CLIENT) {
  dport (`rpcinfo -p | perl -e 'while(<>){/\s+\d+\s+\d\s+(?:tcp)\s+(\d+)/ and $ports{$1}=1}; $, = " "; print sort(keys(%ports)),"\n"'`) ACCEPT; # NFS
}}
proto udp {saddr ($CLIENT) {
  dport (`rpcinfo -p | perl -e 'while(<>){/\s+\d+\s+\d\s+(?:udp)\s+(\d+)/ and $ports{$1}=1}; $, = " "; print sort(keys(%ports)),"\n"'`) ACCEPT; # NFS
}}

Client:
proto udp {saddr ($SERVER) {ACCEPT;}}  # NFS

(If you're only going to use the TCP then you need only the proto tcp part).
Answer: To mount Synology to Ubuntu 18.04 system, I had to enable ports 111,892,2049
When mounting, here is what I see (NFS 4 is not enabled on my Synology):
root@ub18ovh# mount -a -vv
mount.nfs: trying text-based options 'vers=4.2,addr=220.127.116.11,clientaddr=22.214.171.124'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'vers=4.1,addr=22.214.171.124,clientaddr=188.8.131.52'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'vers=4.0,addr=22.214.171.124,clientaddr=22.214.171.124'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'addr=188.8.131.52'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 184.108.40.206 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 188.8.131.52 prog 100005 vers 3 prot UDP port 892
successfully mounted

Answer: For the records, I had to add permissions for ports 111, 2049 AND 1048 for a configuration where an NFS share is exported by a Windows 2008 R2 server and the clients are Ubuntu 12.04.4.
I hope this helps someone.
Comment: Frankly, I don't have a clue why I needed to add a permission for 1048, but adding that solved my problem. I just wanted to share if that can save time someone else in the future. I am sorry for not being able to answer the question.
Comment: It might help to explain why you needed port 1048 and how you determined that.
Answer: if you use csf firewall and nfs does not mount you most likely miss the open ports used by nlockmgr, find them  by typing
rpcinfo -p

Next edit /etc/sysctl.conf to LOCK the ports on these numbers (example port) and add these 2 lines. Then restart portmap, nfs-server.
fs.nfs.nlm_udpport=38073

fs.nfs.nlm_tcpport=38747


Instrumental variables and noisy measurement

Question: I am interested in the effect of the unemployment rate at the time of labor market entry ($u^{LME}$) on wages later in life (this is an old question, but I have a new data set). I'd like to run something like the following regression:
$w_{i,t} = \alpha + \beta_0 u_i^{LME} + \text{other controls}$
$u_i^{LME}$ is endogenous (because to some extent time of labor market entry is a choice), so I need an instrument for $u_i^{LME}$. Past authors have used the birth year as an instrument, which I seek to do.
Problem: I only observe individuals later in life (a 2-year window at different ages for different individuals), so I must somehow identify the time of labor market entry (which is not explicitly reported) to find the unemployment rate at that time. I construct this variable as:
$$\text{Year of LME} = \text{Current Year} - (\text{Age} - \text{Years of Education})$$
This is an imperfect proxy for Year of LME, primarily because Years of Education is an approximation based on level of education attained, for which there are only 6 reported options.
Question: Assuming Year of Birth is a valid instrument for $u^{LME}$ if we had data on the year of labor market entry, is it still a reasonable instrument for the noisy measure of $u^{LME}$ that I have constructed?
Any other thoughts would be helpful.
Answer: The question is: if you had data on year of labor market entry, why would you still use your constructed measure? Typically instrumental variables are a way around measurement error. In your case you want to regress
$$w_{it} = \alpha +  \beta_0 u^{TE}_{it} + X'_{it} \gamma + \epsilon_{it} $$
but the noisy measure of labor market entry $u^{LME}_{it} = u^{TE}_{it} + \nu$, i.e. the true entry year $u^{TE}$ plus some measurement error $\nu$, generates a spurious negative correlation with the error. When you substitute this back into the regression
$$\begin{align}
w_{it} &= \alpha +  \beta_0 \left(u^{LME}_{it} - \nu \right) + X'_{it} \gamma + \epsilon_{it} \newline
&= \alpha +  \beta_0 u^{LME}_{it} + X'_{it} \gamma + \epsilon_{it} - \beta_0 \nu \newline
&= \alpha +  \beta_0 u^{LME}_{it} + X'_{it} \gamma + u_{it}
\end{align}$$
where $u_{it} = \epsilon_{it} - \beta_0 \nu$ is the error term (since the measurement error is unobserved) such that $Cov(u^{LME}_{it},u_{it})\neq 0)$. This gives the typical attenuation bias towards zero, however, there is another source of bias from selection. Suppose there is a positive wage shock $\epsilon_{it}\uparrow$ and because individuals see this shock they enter the labor market earlier than they would have in the absence of the shock. Then selection will make it look again like entering earlier increases your wage.
Also omitted variable bias is a problem here since $u^{LME}_{it}$ is a function of education which depends on unobserved ability. So you could also imagine that high-ability workers enter the labor market later because they want to signal their ability with a college degree. Later wages are higher but not necessarily because of later entry per se but because these individuals are high ability, i.e. they would have had higher wages also in the case that they had entered the market earlier. The problem is that you don't observe ability.
Long story short: using an instrumental variable for your constructed measure is a good idea. However, even if you were to observe the true time of labor market entry it is still a good idea to use the instrument because of the selection problem. One more thing about your constructed measure is that you should add a constant equal to the school entry age. Otherwise you are assuming that people enter school straight from birth, so a better measure would be
$$u^{LME}_{it} = \text{year} - \text{age} + \text{years of education} + 6$$
If you have U.S. data, for instance, you can get the school entry age by state.


How to load Kafka consumer or producer properties from application.yaml?

Question: Currently I have this code for creating a ConcurrentKafkaListenerContainerFactory.
@Bean
public ConcurrentKafkaListenerContainerFactory<Object, KafkaExampleRecord> kafkaListenerContainerFactory() {
    ConcurrentKafkaListenerContainerFactory<Object, KafkaExampleRecord> factory =
            new ConcurrentKafkaListenerContainerFactory<>();
    factory.setConsumerFactory(new DefaultKafkaConsumerFactory<>(consumerProps()));
    factory.setReplyTemplate(kafkaTemplate());
    return factory;
}

How do I create the DefaultKafkaConsumerFactory object with properties from application.yaml? I don't want to set them inside the consumerProps() method.
EDIT:
My KafkaConfig class now looks like this.
@Configuration
@EnableKafka
public class KafkaConfig {
    @Bean
    public ReplyingKafkaTemplate<Object, KafkaExampleRecord, KafkaExampleRecord> replyingKafkaTemplate(ProducerFactory<Object, KafkaExampleRecord> producerFactory,
                                                                                ConcurrentMessageListenerContainer<Object, KafkaExampleRecord> repliesContainer) {
        return new ReplyingKafkaTemplate<>(producerFactory, repliesContainer);
    }

    @Bean
    public ConcurrentMessageListenerContainer<Object, KafkaExampleRecord> repliesContainer(
            ConcurrentKafkaListenerContainerFactory<Object, KafkaExampleRecord> kafkaListenerContainerFactory) {

        ConcurrentMessageListenerContainer<Object, KafkaExampleRecord> rc =
                kafkaListenerContainerFactory.createContainer("mytopic");
        rc.setAutoStartup(false);
        return rc;
    }

    @Bean
    public KafkaTemplate<Object, KafkaExampleRecord> kafkaTemplate(ProducerFactory<Object, KafkaExampleRecord> pf) {
        return new KafkaTemplate<>(pf);
    }
}

However now I get java.lang.IllegalStateException: a KafkaTemplate is required to support replies.
Answer: Spring Boot automatically configures one; simply add it as a parameter to the factory method:
@Bean
public ConcurrentKafkaListenerContainerFactory<Object, KafkaExampleRecord> kafkaListenerContainerFactory(
    ConsumerFactory<Object, KafkaExampleRecord> cf) {
...

However, you don't need to declare the container factory bean either; Boot will auto configure it for you and automatically set the KT, if it is defined as a bean.
See Boot's KafkaAnnotationDrivenConfiguration.
Comment: Use `` in the bean definition (and its pf parameter).
Comment: See the code in `KafkaAnnotationDrivenConfiguration`; it uses `ObjectProvider> kafkaTemplate` and `this.kafkaTemplate = kafkaTemplate.getIfUnique();` - the generic types must match in order for the `ObjectProvider` to work; the actual types don't matter at runtime due to type erasure. In practice, the types really only matter for `template.receive()` operations since, when sending, the actual type is used by the serializers in the `kafka-clients`.
Comment: Mhm... when I define KT as bean I get a `KafkaTemplate is required to support replies` (see my EDIT above).


Is it possible for regression model to predict patterns separately from data has multiple patterns?

Question: I want to predict sold number of each drink(hot and cold) without clustering.
I have data which contains sold number of hot and cold drinks.
I trained it with linear model in scikit, and I thought I can predict the sold number which is different shape based on it's hot or cold.
But I got one very horizontal(average?) line.
The prediction result are all the same no matter I predict hot or cold drinks.
Experiments:
Expected result: I can predict pattern of each hot or cold drink.
Train X: hot or cold, drink ID, date, week, day(ex:0 for Mon., 6 for Sun.), month.
Train y: sold numbers of 6 drinks in a month has 28 days.
Fig. of train data:
x -> days ex:1,2...28, y -> sold numbers

Fig. when I trained with sklearn.linear_model.LinearRegression:
model = linear_model.LinearRegression().fit(train_X, train_y)

Fig. when I trained with sklearn.linear_model.ElasticNet:
model = linear_model.ElasticNet(max_iter=100).fit(train_X, train_y)

I tried linear_model but it seems only to predict a "average" results.
So linear model only can predict one pattern(trend) of data even if I combine all patterns in a dataset?
So maybe all regression models can only predict one pattern(trend) of data? or other regression model can reach that ?
Comment: Hi, my dear slavic friend! Are you talking about linear regression or any kind of regression?
Comment: @code_worker could you provide more details about your data and your used method  (some code and a table and/or graph for instance). It is not so clear why you do not get the trend that you expect. Possibly you specify the model incorrectly.
Comment: @asdf yes, linear regression and elasticnet.
Comment: @MartijnWeterings I update it some figures to explain
Comment: What were your expectations for these graphs/data? In your graphs, I do not see a strong rise or decline in the number of sold numbers of drinks as a function of time (is it days on the x-axis?). So the fitted curve (when it is a straight line) will be flat and resemble something close to the average.
Comment: What is represented on the x-axis and y-axis of your graphs?
Comment: @MartijnWeterings x is the day, y for sold numbers.
Comment: @MartijnWeterings . Right, the trend is not clear. So I try figure out one way to predict sold numbers of each drinks of each day, firstly I think about regression model.
Comment: @code_worker So what is the question about statistics?
Comment: @MartijnWeterings I don't know if there's a term to describe what I want. I have 6 sold numbers of 6 drinks, and I want model to predict(or seperate?) 6 patterns from the Fig. of train data
Comment: @code_worker what are you doing this analysis for? Is it a hypothetical study question, or is there a practical reason (e.g. is it for work)? Why do you have these data? How did you get the data? What do you think/hope you want to achieve? Etc. If you describe these things in common language, rather then trying to seek for statistical terms that you do not understand or know anyway, then it might be easier for others to help to formulate a good statistical question out of it.
Comment: @MartijnWeterings I fix the title and the context. thx
Answer: Linear regression can predict different patterns of data - you can use splines, you can use polynomial terms and so on.  In addition, there is the whole field of time series analysis, although that usually deals with cases where there are more time points than you seem to have.
But the problem seems to be that your data have no pattern. Thus, when regression (of any sort) or any other method comes up with a flat line, it is doing what it should - saying that there is no pattern in your data.
If you had some a priori reason to expect a pattern, then you have learned something: Your reason is incorrect, at least in this data set without any covariates.
However, since it seems like you are trying to predict the sales of drinks, I'm puzzled as to why you have no covariates in your model.  Certainly the temperature and whether it is a weekend ought to be included.
Comment: I know maybe I lose some important covariates like weather, temperature. What I want to know is model to predict(or seperate?) 6 patterns from the Fig. of train data. But I don't know is that possible or some term to describe the goal. thx


What is the difference between linear and non-linear regression?

Question: In machine learning, I understand that Linear regression assumes that parameters or weights in equation should be linear. For Example:

Y =  W1*X1+W2*X2

is a linear equation where X1, X2 are feature variables and W1, W2 are parameters.
also,

Y = W1*(X1)^2 + W2*(X2)^2

is also linear as parameters (W1, W2) are linear with respect to Y.
Now, I read some articles stating that in the equation like

Y = Log(W1)*x1 + Log(W2)*x2

can also be made linear by considering other variables V1 and V2 as:

V1 = Log(W1)
  V2 = Log(W2)

thus,

Y = V1*X1 + V2*X2

So, in this sense, any non-linear equation can be made linear then what is non-linear regression here. I think I am missing something important here.
 I am a beginner in the field of Machine Learning. Can somebody help me?
PS - This is my first question here at StackOverflow so if there is an error in writing an effective question, please forgive me.
Comment: This is an interesting question, but it's not really a programming question as presented.  You might try asking it on one of the other Stack Exchange Network sites (like mathematics or AI).
Comment: In short, though, as long as the `X`s have an exponent of 1, and the coefficients are constants, it doesn't matter if the constants are expressed as functions (of any kind) of other constants. If `W1` is constant with respect to `X`, so is `log(W1)`.
Comment: The replacement of `log(W1)` with `V1` just makes it "look" more linear; it doesn't change the fact that it was already linear.
Comment: @chepner, if it is already linear then what is non-linear?
Comment: `Y = W1*(X1)^2 + W2*(X2)^2` is non-linear, because neither `X` has an exponent of 1. See https://en.wikipedia.org/wiki/Linear_combination.
Comment: @chepner is mistaken. Please refer to my answer. Linear regression only refers to linearity in the parameters. The comment would be correct if it were referring to raising the $W_1$ and $W_2$ weights (parameters) to exponents other than 1: $Y=W_1^2X_1^2 + W_2^2X_2^2$ Is a nonlinear regression. However, the posted $Y=W_1X_1^2+W_2X_2^2$ is a linear regression.
Answer: In statistics, a linear regression refers to linearity in the parameter. For instance, $y=a + bx +cx^2$ is a linear regression, since we only add and subtract multiples of the weights $a$, $b$, and $c$ (the multiples being the data: $x$). However, $y=a +log(bx)+cx^2$ involves a nonlinear log transformation of the $b$, making that a nonlinear regression.
Comment: @ka1shi clever thinking, but that would still be nonlinear. A property of a linear transformation $T$ is that $T(\vec{0})=\vec{0}$. What is $0+log(0)+log(x)+0x^2$? That is undefined, so it isn't zero. The way to determine if a regression is linear is the usual way for linear transformations: $T(a\vec{x}+b\vec{y})= aT(\vec{x})+bT(\vec{y})$. Trying $\vec{\beta}=\vec{0}$ to check if that results in $0$ also can be a quick way to rule out a regression as nonlinear, though that test alone cannot confirm a regression as linear.
Comment: I was thinking the same but we can separate log(bx) as log(b) + log(x) so it can be considered as linear again: y = a+log(b)+log(x)+cx^2, where a+log(b) can be considered as biased term. So how do we decide if the equation is linear or not?


Change permission on Content Overview page

Question: I have granted an admin 'Administer Content' permission under People->Permissions. This gives him the ability to publish/unpublish nodes. But I don't want him to be able to go to the Content overview page where he can delete and unpublish in bulk. This is because I have setup certain workflow rules with node_validate which only let him delete or unpublish nodes under certain conditions within the node edit screen. If he goes to the Content overview page, he can circumvent the workflow logic.
So how do I programmatically take 'Administer Content' permission away when he goes to the Content overview page? I still want him to be able to see the Content overview, but I don't want him to be able to unpublish, delete, or do any of the other bulk operations.
Any thoughts would be appreciated.
Answer: This is such an old question, but I had the same question and hiding it in css is not the answer.
The correct answer IS to enable the Actions permissions (VBO) (actions_permissions) module.  That will allow you to set which roles can perform bulk operations.
Answer: I am pretty sure there isn't a way to accomplish this.
Maybe you could use css and hide the Edit Options field set like this:
<code>#edit-options {
  display: none;
}
</code>
That would hide it, but if he was malicious he could edit the CSS in his browser and then use it.