Storing currency as BigDecimal in Hibernate/JPA - decimal(19,2)

Question: I changed the fields in my java application that hold currency values from double to BigDecimal to prevent strange double rounding issues like "0.20000000000001".
This now created the SQL date types of decimal(19,2), which means 2 digits right of the ".". Is this really a better way to store currency values or do I even lose precision this way?
Comment: If you're really concerned about losing precision, use an integer datatype instead.
Answer: If you don't want to loose precision value then you should use BigDecimal instead of BigInteger (in your Entity class in hibernate / JPA)
Consider following demo:
    // some calculation
    double calculatedValue = 0.20039930000;

    BigDecimal amount = new BigDecimal("" + calculatedValue).setScale(2,
            BigDecimal.ROUND_HALF_UP);

Comment: Sorry I meant BigDecimal, not BigInteger! Corrected my post
Comment: Thanks for the answer! Can you explain some of its parts? Why doesn't `new BigDecimal(calculatedValue);` work? Why do you need to concatenate the double with a String?
Comment: When I guess Josh built the API he built in such a way that, if we pass the value (not string) then it will do the calculation using primitive data type. In one of the Google IO event he explained and said its a bug in JDK and they have fixed it in their library(google's lib - at the time of IO event he was with google). And Sun/Oracle dont want to change the core API structure as it is used in multiple places and its been there since the introduction of BigDecimal class


How to find scales to improvise with for "How Insensitive" by Jobim

Question: I am pretty new to improvisation. So far I have improvised over a 2 5 1 change using the diatonic 7th chords and the underlying scale tones that fit in with that chord. I am playing on a guitar.
As a precursor I will state what concepts I understand in music theory so far:

Scales (Major, Minor(Melodic/Harmonic/Natural))
Chords
Diatonic Chords

I want to take it to a song, and I like the sound of "How Insensitive", here is the lead sheet I have:

When I started, I thought that the key was C, because there was no key signature but someone told me it was actually in D natural minor. Is this common?
Next I wanted to see how these chords fit into that scale. For the D-9, I see that that's pretty much a diatonic seventh chord starting on the first note of the D natural minor scale, so I would improvise over it with that scale itself.
I checked it out the next chord, which is a C# diminished seventh, I know that C# isn't in D natural minor, but I do see that the melody is playing an A, so if I were to combine the notes of the C# diminished seventh and the A, we get A, C#, E, G, Bb ~ that is a A7 with a flat 9.
At this point I was unsure about how to find a scale that would work under this chord, because C# is not in D natural minor.
I thought about two other minor scales:

D Harmonic Minor: D,E,F,G,A,Bb,C#,D
D Melodic Minor: D,E,F,G,A,B,C#,D

The D Harmonic minor looks quite similar to the chord, but am I allowed to play this scale in the background? Doesn't that contradict with the fact that the song is in D natural minor?
I moved to the C minor 6th which is C, Eb, G, A. I see that the Eb is in this chord, but not in D natural minor, so again, I need to find some other way to see what I can play in the background of it.
Based on the D Harmonic or Melodic Minor, it still wouldn't work as neither contain the Eb. I'm quite confused on how I could figure out what I can improvise with here.
My two main question are:

Is there a process I can follow to determine what scale I can improvise with? Like with the C# diminished 7th It seems like the melody can help at times, but I'm not sure exactly
Can someone explain why lead sheets have non-diatonic chords, I'm confused because they might not have scale tones in them, especially like with the E7 near the end of the song.

Thanks for the help!
Comment: You don't need a scale for every chord.  You need to track the significant changes or modulations, and apply voice leading.
Comment: @ggcg Could you write a whole answer about your approach? If you start with a default scale and then track the significant changes, you end up with at least one possible scale for every chord, whether you need it or not. No?
Comment: What box? What I mean is, if you want to play something in the pitch range between explicated chord tones, how do you decide which notes to choose. Of course nobody forces you to play anything from any particular range. Does the OP have to ask about that specifically - how to select notes within _this range_? You are forcing yourself into the chord tones box. Why do you have to play chord tones? What sense does that make? If you want to play between chord tones. Is that a bad thing to want. What sense does it make not to write an answer if you have one.
Comment: Some thoughts to layer on top of what you get from the answers: (1) It's important that a solo be enjoyable to hear and communicates to your audience in complete sentences. It's not important whether it uses scale X or scale Y over a particular chord symbol on a lead sheet. (2) If you can be aware of the 3 and 7 of each chord and of how they move when the chord changes, it will help your solo to sound like it really is a solo on that song's changes. (3) Pros have "big ears," so they can avoid train wrecks while playing freely. You're a beginner, so just try to be compatible with the chords.
Comment: I think this is a very well thought-out and written question. Our regular visitors are too often hesitant to admit that a question is well written and effort has been put to writing it. +1
Comment: That makes no sense.  You are forcing yourself into that box.
Comment: @piiperiReinstateMonica - your comment *could* be to ggcg, but without '@', it could be to OP.
Comment: this is actually a Bb version for trumpets and tenor sax; the guitar version should be in C minor
Comment: This is the lead sheet from the Real Book in concert key. I've seen it hundreds of times.
Answer: First of all: music is not "in a scale". When we say that it's "in D minor", it does not mean any particular scale, it means that the home note is D and its third is a minor third, i.e. F. And even those two notes can temporarily change during the song. "In D minor" means, when the harmony is at rest at home, then the bass is D and there's an F somewhere above it. That's ALL it means. When the harmony is NOT at home, i.e. at a resting, ending position where it could peacefully lie down and go to sleep, even the F can be F# or something. "In D minor" does NOT say any scale, not natural minor, not harmonic minor, not melodic minor, not Dorian, no single scale. No single scale. This song is full of chromatic alterations, which mean that whatever seven-note scale you initially select, you'll have to make changes to it along the way. Modal music is rigidly restricted to using only a single fixed scale for constructing intervals around the tonic, and the scale is that mode's scale. But How Insensitive is not modal, and the whole genre is like traditional jazz, where you have "changes" and chromatic alterations.
How to track the chromatic alterations
One way to handle non-diatonic changes is to try to see a seven-note scale that's chromatically altered by chords. Chord tones force the scale notes to move from their default positions. (But seven-note thinking is not the only possibility.)
Let's start from the beginning and see what kind of chromatic alterations there are. The song is written without a key signature because someone thought it's a good idea to do so for songs which have many chromatic alterations (I disagree - this is not some weird atonal chromatic nonsense, or something that modulates all over the place, in the end it's fairly comprehensible stuff in D minor), but you should hear trivially by listening to the song that D minor is a home chord. So, since the tune is in D minor, we assume the notes of the D minor natural scale as being a default reference pitch grid, to which we do the chromatic alterations.

First there's a Dm9 chord. It has a C note, meaning that the flat/natural/sharp switch on the C scale slot is in natural position. C is natural.
Then there's a C#dim7 chord. Now C is sharp! The C scale slot was chromatically altered and now C's flat/natural/sharp switch is in sharp position. (However, you can still play notes from the D minor pentatonic scale over it! But I digress)
Then there's a Cm6 chord. It has notes C, Eb, G, A. Two note name slots in the scale were chromatically altered! Now C is natural (again), and E is flat.
Now a G7/B chord. The bass note is chosen as B just to keep the bass line moving step by step (D - C# - C - B), but G7 has notes G, B, D, F. Now B is natural. But what about E, is it still flat? Try it! I think Eb sounds completely fake there so I conclude that the G major chord reset it back to E natural. (And how about C, is it natural or sharp? Both work fine, they just create a slightly different feeling)
Then a Bbmaj7 chord. Now, because it's called Bb, you can immediately tell that B is flat.
And so on.

Chromatic alterations on the fretboard
I'll attempt to visualize the seven-note scale alterations caused by chords on the guitar fretboard. We start from a default vanilla diatonic scale, natural minor as our reference grid. "6" is the minor-side home base, and "1" is the relative major-side's home base. The absolute position doesn't matter, this is all relative.

Now comes the C#dim7 chord, which is a #5 dim7.

We can think that the #5 note forces the note 5 to move up as if pulled by a magnet. You might recognize the result as having the same notes as the harmonic minor scale, and that would be true. But in my opinion it's not necessary to have a proper Scale Name for everything - this stuff comes and goes, scale positions move back and forth as commanded by the chord tones. And it's up to you, if you want to see a seven-note scale, or a diminished scale, or whatever. You throw in chords and notes in the harmonic soup, twisting the taste to your liking. Using a seven-note grid is just a commonly used tool for reasoning about what all these notes do.
Anyway. The magnets stay in place keeping the C pulled up while the C#dim7 chord is playing, but then another set of magnets show up, namely the Cm6 chord:

The C note is released or pulled to the regular place by the chord's C note, but now the E note is pulled down i.e. made flat by the Eb note in the Cm6 chord. You might see that it looks like a Locrian scale on the "6" note, or Dorian on the "5", and why not. Mentioning mode names has a certain weirdness to it, because this is not a modal tune, and the chord is only played for two bars.
Tracking changes to a seven-note scale is not the only possible way to reason about harmony, but for me it's the most basic way of thinking. As an additional feature, you can think of stuffing extra notes somewhere, like for example playing both C and C#, or adding "blue notes" between scale notes. How you emphasize the extra notes, changes the harmonic feeling. Do the "colliding" notes sound simultaneously or do you use a magnet to move the scale position back and forth? Or you can see half-whole or whole-half diminished scales there. Any perspective is equally valid, as long as it enables you to operate. In the end, harmony is the sum of its parts.

Is there a process I can follow to determine what scale I can improvise with? Like with the C# diminished 7th

Above I tried to outline such a process. About diminished sevenths, there are several ways to see the role of diminished seventh chords: (1) dominants, and (2) something else like modified minor chords or just some chromatic in-between form on the way to another chord. Using them as dominants is more common IMO.
This means that when you see a chord like A7, you can see it as C#dim7 (or Gdim7 or Bbdim7/A#dim7 or Edim7). And the other way around - when you see a dim7 chord, you can see it as a dominant chord for something. In the beginning of How Insensitive there's a C#dim7 chord, and if we interpret it as a dominant, the most obvious choice in this song is A7, which is Dm's dominant chord. Try to replace C#dim7 with A7/C# - it's almost exactly the same.
Another way to see this is to imagine a half-whole diminished scale over a dim7 or a dominant chord (which can be thought of as being equivalent, see above).
So, over the C#dim7 chord, you can overlay for example

D natural minor scale with C# (which creates a D harmonic minor scale)
C# whole-half diminished scale (or A half-whole diminished scale). Which by the way do not have our home note D!

It seems like the melody can help at times

Consider the melody as a part of the harmony. For example if there's a written Dm chord, but the melody plays a strong B note, it forms a Dm6 chord.

Can someone explain why lead sheets have non-diatonic chords

Non-diatonic chords mean that there are chromatic alterations. (An individual non-diatonic note can be either a strong chord tone or a weak passing tone, and passing tones don't necessarily have to make changes to your harmony scale) Chromatic alterations are such an elementary basic 101 thing in tonal music that every music student should be introduced to the concept on day 1. But for some inexplicable reason, they aren't, and then we have people who cannot understand even the simplest for-dummies polka song harmony. Music is not in a scale. You have an expected note-probability scale in your head, and the music you hear makes changes to the probabilities. There is a switch on the scale slots in your mind: flat / natural / sharp. And when you hear notes, they move the switches to different positions. Now C is natural. Now C is sharp. Now C is natural again. Now I'm not sure but either could work. Now I hear both natural and sharp, and it feels bluesy. Music is not in a scale.

I'm confused because they might not have scale tones in them,

Music is not in a scale. An expectation of a scale is in your mind, and music makes changes to it. Surprise! Not the note you expected? This chromatic alteration of notes creates a specific sensation which makes the music interesting in a particular way. In modal music, the chromatic alterations are permanently locked to fixed positions. For example in C Lydian, you have C as your home base and the white notes of the piano around it, except the F slot is permanently superglued to F# position.

especially like with the E7 near the end of the song.

E7 - A7 - Dm is one of the most basic things to do harmonically in songs in D minor. During the E7 chord, G is sharpened, and during the A7 chord, C is sharpened. The E7 chord is called "secondary dominant".
One more thing. You can play D minor pentatonic or D minor blues over the whole song! This is because the song is composed in such a way that even though there are chromatic alterations, it always revolves around D minor. This is an important thing to know, and it works both ways. The song and the accompaniment (backing track) could play a fixed Dm chord or something Dm pentatonic-ish, and you as a soloist could overlay a complete harmonic progression with chromatic alterations over it. Basically, if the backing track is playing Dm, you can solo How Insensitive over it! Melody and chords.
The different kinds of scales you could see over How Insensitive:

(1) Five-note scale (pentatonic)
(2) Seven-note scale, starting with diatonic and following the song's changes, making the proper chromatic alterations along the way
(3) Seven-note scales according to what you WANT to do and what the backing chords allow you to do, for example the jazz minor i.e. D-E-F-G-A-B-C# fits over some chords, but not all.
(4) Eight-note scales, i.e. diminished scales, over the dominant chords

Or, my own favourite: concentrate on chords and let the scale question solve itself automatically. If you know that the notes of A7 and Dm must fit, find those. If you know there's an Eb maj9 chord, find those notes, and who cares about scales. Chords rule, scales obey! Why care so much about scales, because they totally obey chords. If you play an Eb chord there, the E slot in the scale is left no options, it must drop flat on its belly. If you can command chords, you automatically control scales as a side-effect.
The different harmonic styles of jazz soloing
If I sum up the different strategies to use in different backing/comping + soloist situations:

Style 1: the tune has changes, and the soloist follows the changes and their chromatic alterations and makes sure that her actions are compatible. This is the traditional jazz way. Everybody has lots of things to keep track of. Comping has to follow harmonic changes, and the soloist has to follow harmonic changes and take into account the restrictions the changes impose on how she can twist the harmony. Air Traffic Control State of Mind! Lots of moving objects in the air that must not collide.
Style 2: the tune has changes, but the soloist plays safe pentatonic notes around the tonic only and DOES NOT (HAVE TO) TRACK the tune's changes and their chromatic alterations. This is the blues/rock way. The soloist only has to react if the tune modulates so radically that the pentatonic around the tonic doesn't fit anymore.
Style 3: the tune does not have changes or chromatic alterations. The soloist is free to do whatever she likes. Play safe pentatonic, or stick to the same scale as the tune has, or invent her own changes and chromatic alterations, maybe "go outside" and play pentatonic or harmony-outlining lines from other keys. Or anything. Complete freedom for the soloist. This is the modal jazz way. It's like style 2 but with the roles reversed and the whole situation turned upside-down. The tune is stripped of "changes", and comping plays a harmonically static groove, in order to set the soloist totally free.
Style 4 could be a variant of style 3: the tune is modal, but the comping plays such thick and dense harmony that it in practice restricts the soloist's freedom. This can be a desired situation, or not. Had the comping played something harmonically simpler, it would have left more space for the soloist.
(For completeness, I have to add that it's possible to go outside even on traditional non-modal tunes that have changes, two-five-ones and all that. IMO, playing outside means that the soloist builds a competing harmonic context which diverges from that of the backing instruments and the listener's expectations. The listener starts to recognize a different "drawing" being superimposed on top of the backing music. An extreme example of such soloing could be George Garzone's Triadic Chromatic Approach, which constantly moves the perspective around without letting the context settle down at all.)

Styles 1 and 2 are applicable to How Insensitive, because the tune has changes. You either follow the changes or play pentatonic. Styles 3 and 4 are not applicable, because How Insensitive is not a modal song.
There. I think I did a damn good job at explaining how to find sets of notes over jazz changes. That means that I probably explained it only to myself and nobody else has any clue what I'm rambling about.
Comment: @jdjazz Thanks for the comments on dim7 chords, I knew there had to be more because a dim7 is completely symmetric. I dodge the "chromatic passing chord" idea even though I play those sometimes, because I don't have any alternative harmonic perspective or interpretation to it. It's like harmonic passing chords like Am7 - G#m7 - Gm7 - very clear and easy to reproduce, but what does the G#m7 do. Calling it "just chromatic" means I haven't yet figured out anything better than that. For the Cdim7 - C ending, to me that sounds like Jailhouse Rock in C: B - C... in disguise.
Comment: @Tim Thanks! It's very hard to know what if anything I managed to communicate. Even the up/down votes, maybe someone just had a good feeling for an inexplicable reason. It would be best to be able to play together and have musical interaction.
Comment: Believe me - votes, especially dvs are fickle, and as I say many times, carry no weight without a comment to support them. I don't understand the last sentence, though.
Comment: Oh, about musical interactions. Yes, that's how you can really check how others understood things. You can talk about modes and someone says they understand, but when they start playing you might feel that the tonic is in the wrong place, so ... did they understand or is it a matter of too little practice. Or  can we really feel about the tonic so differently.
Comment: Not rambling at all! I learned a ton. I'm definitely going to keep the "every note has a chromatic switch (flat/natural/sharp) idea, it seems to be cool. And thank you for shattering my idea that the song had use the scale that the key is in. I'll be using this answer as a reference in the future as well.
Comment: Nice answer. Re: fully diminished chords in jazz, the ways I think about them are: (a) rootless dom7#9 chords, (b) passing chromatic chords (e.g. | C Maj | Dmin D#dim | E7b9 | ), (c) delayed resolution to major chords (e.g. | Dmin | G7 | Cdim CMaj | ), and (d) none of the above (which sometimes happens too)!
Answer: Is there a process I can follow to determine what scale I can improvise with? Like with the C# diminished 7th It seems like the melody can help at times, but I'm not sure exactly.
As a general rule, as long as you incorporate the notes in a particular chord, you can use any other notes you wish as passing tones. Taking the initial D-9 chord as an example, given the chord alone, you could play a scale from A to C using either Bb and or B to "pass" from one note to the next. However, since this specific melody has a Bb, you would likely choose Bb.
Note, for example, in the sixth measure, the melody itself uses both B and Bb to pass from C to A within a C-6 chord. Improvising, you could do the same.
Can someone explain why lead sheets have non-diatonic chords, I'm confused because they might not have scale tones in them, especially like with the E7 near the end of the song.
The use of non-diatonic chords is common across a huge range of music -- not just jazz and not just those written as lead sheets. Non-diatonic chords can add color and interest and/or can indicate a temporary change of key/scale. We say a piece of music is in the "key of X" as an overall description, but the piece need not stay in that key exclusively.
Comment: Hi Aaron, thank you for that information, very helpful. I want to take your ideas right into a concrete example, can you let me know if I applied it correctly? (Next comment I make)
Comment: The E7b9: E Ab D F, I chose the passing tones G & C from the D natural minor scale, so I will improvise with E G A Bb C D E F (D natural minor starting on the note E).

The Bdim7: B C# E G, I chose the passing notes D & F as they are used in the melody after this bar and are in D natural minor, together they form B C# D E F G A (D melodic minor starting on the note B).

The Ebmaj7 chord is not diatonic, it has the notes Eb G Bb C# which complements the melody which is playing a G. I can't quite explain the C#, but could it be there for color/interest?
Comment: @cuppajoeman The best answer I can give is that you're over-thinking things. Or maybe better to say that you're being too rigid. Go ahead and try improvising with those pitches and find out how they sound. With that said, the pitches you suggest will give you good starting points.
Comment: @cuppajoeman Regarding the Ebmaj7: there is no C# in that chord. It's Eb G Bb D.
Answer: Truly, the best approach is to analyze the solo(s) on a famous recordings of the song. This is a practical approach that will save you time and keep you moving forward in your understanding of scales, chords, and songs. It is also the fastest way to arrive at the information you seek.
Fight the urge--which is common among beginner jazz musicians--to analyze theory in a vacuum. Theory is like grammar: it tells you how to understand and organize the sentences you hear. It doesn't teach you how to speak.
By transcribing/analyzing a famous solo, you'll simultaneously (a) learn some strong scale choices and (b) see how they are applied in-context. That is, you aren't just getting names of scales--you're also getting:

examples of how to use the scales to resolve from one chord to the next
shared scale tones that you can emphasize
how to phrase your ideas
how to expand a motif from the melody
voice leading and chromatic movement
how to allude to the tonic even when playing a non-diatonic chord
how to emphasize chord tones that aren't in the key center

But what about guiding logical principles like choose the scale that minimizes the differences from the previous scale or choose the scale with the most matching tones with the key center? With time, you'll eschew those sorts of guiding principles, and instead, you'll pick scales based on the possibilities they create. In other words, the sorts of techniques/ideas I've listed above (and the doors they open) will become the criteria you use to determine which scale you want to improvise with. That's a freer melodic approach which is guided by context and ideas rather than by strict logical principles.
For a guitarist on How Insensitive, I recommend checking out Wes Montgomery for a simpler but beautiful solo and Pat Metheny (if you want more complex).
Comment: Here are common scale choices for How Insensitive (measure by measure): **|** D Aeolian | % | C# whole-half diminished | % **|** C Dorian | % | G Mixolydian | % **|** Bb Ionian | % | Eb Lydian | % **|** *E Locrian ♮2* | *A Altered* | D Aeolian | *Db Lydian Dominant* **|** C Dorian | % | *B whole-half diminished* | % **|** Bb Ionian | *E Locrian ♮2  A Altered* | D Aeolian | *Db Lydian Dominant* **|** C Dorian | F Mixolydian | *B Locrian ♮2* | *E Altered* **|** Bb Lydian | A Altered | D Aeolian | % **|**. I've bolded every 4 bars & italicized scales where a range of different choices are common.
Answer: As already mentioned - but to underline - no piece is 'in a scale' - ever! A scale is simply a man-made article which lists all of a certain set of notes in ascending/descending order.
Insensitive is fairly and squarely in key Dm - which opens up the set of notes to include D E F G A B♭ B C and C♯. Actually all can be and are used in pieces in key Dm. Leaving only E♭, F♯ and G♯ as notes not included. Having said that, he manages at least all of them somewhere in the melody! If he can, why can't you?
It, like so many jazz (and other) pieces, moves about, not adhering rigidly to the 'diatonic' notes (as listed), and uses what Jobim needed to write the song. So having your Dm hat on while extemporising won't help - much.
If you're struggling, I suggest you go through the changes very slowly. A lot of the chords stay for 2 bars anyway, which helps. A start might be to concentrate on chord tones, which straightawaay gives you options of 3, 4 and sometimes 5 notes for the main ones. Even a straight arpeggio will suffice in parts, and the passing notes, since you're working really slowly, will be in between, and your ear should decide whether a B or a B♭ sounds better in a particular bar.
The usual tricks - chromatic approaches, enclosures, etc., will all come out to play while each change occurs. But stop thinking it as a scales exercise. It could work in parts, but that approach will trip you up at some point.
Answer: If you have your scales and arpeggios mostly under your fingers in most positions just noodle along to a backing track and listen intently without thinking or evaluating.  You'll hit some clams but the more you do it the better your ears will get and the freer you're playing will become.  There are a few "avoid" notes for every chord but even those are chromatic neighbors of chord tones and extensions.  No wrong notes, just notes you don't want to land on, but when you do, you can just slide up or down a fret and you're hitting a "good" note.


How to log request and response body in WebFlux using MDC + Logbook

Question: I have a Spring Boot application, a reactive service which calls external systems via WebClient. I also have some non-reactive services. There is already implemented logging using MDC + Logbook for Feign clients. It's highly desirable to reuse existing classes or at least make log records look the same as non-reactive services logs.
RESPONSE_CODE=200, RESPONSE_HEADERS=[transfer-encoding:"chunked"], METHOD=POST, REQUEST_QUERY={}, REQUEST_BODY=FluxMap, RESPONSE_BODY=, REQUEST_PATH=/ping, REQUEST_HEADERS=[Session:"11111abcd", Content-Type:"application/json", ..., Content-Length:"37"], TYPE=INREQ

Current logs look like that. Problematic parts are REQUEST_BODY=FluxMap and RESPONSE_BODY=
I managed to log body using wiretap() on WebClient, but it's not customisable, so it doesn't suit.
For reference, I used sample code from this article
which I found in this Github discussion
I've tried different approaches: using ExchangeFilterFunction and WebFilter + ServerHttpRequestDecorator (and other decorators) where I try to get body from DataBuffer. Still no luck.
Answer: Solution that worked for me:

Add this dependency

<dependency>
   <groupId>org.zalando</groupId>
   <artifactId>logbook-spring-boot-webflux-autoconfigure</artifactId>
</dependency>

Pass Logbook object when you create a WebClient

private WebClient getWebClient(Logbook logbook) {
    HttpClient httpClient = HttpClient.create()
            .doOnConnected(connection -> {
                connection.addHandlerLast(new LogbookClientHandler(logbook));
            });

    return WebClient.builder()
            .baseUrl(myUrl)
            .clientConnector(new ReactorClientHttpConnector(httpClient))
            .build();
}


How to remove Cursor Position ANSI escape code from `HijackedResponse` in Go?

Question: I am trying to exec (interact with) a docker container, with Go.
This is the code I am using:
func (docker *Docker) redirectResponseToOutputStream(outputStream, errorStream io.Writer, resp io.Reader) error {
    _, err := stdcopy.StdCopy(outputStream, errorStream, resp)
    return err
}

func (docker *Docker) holdHijackedConnection(inputStream io.Reader, outputStream, errorStream io.Writer, resp types.HijackedResponse) error {
    receiveStdout := make(chan error)
    if outputStream != nil || errorStream != nil {
        go func() {
            receiveStdout <- docker.redirectResponseToOutputStream(outputStream, errorStream, resp.Reader)
        }()
    }

    stdinDone := make(chan struct{})
    go func() {
        if inputStream != nil {
            io.Copy(resp.Conn, inputStream)
        }
        resp.CloseWrite()
        close(stdinDone)
    }()

    select {
    case err := <-receiveStdout:
        return err
    case <-stdinDone:
        if outputStream != nil || errorStream != nil {
            return <-receiveStdout
        }
    }

    return nil
}

...and call the the holdHijackedConnection here:
func (docker *Docker) ContainerExec(ctx context.Context, container *injection.Container) error {
    createResponse, err := docker.client.ContainerExecCreate(ctx, container.ID, types.ExecConfig{
        AttachStdout: true,
        AttachStderr: true,
        AttachStdin:  true,
        Detach:       true,
        Tty:          true,
        Cmd:          []string{"sh"},
    })
    if err != nil {
        return err
    }

    stream, err := docker.client.ContainerExecAttach(ctx, createResponse.ID, types.ExecStartCheck{})
    if err != nil {
        return err
    }

    defer stream.Close()
    docker.holdHijackedConnection(os.Stdin, os.Stdout, os.Stderr, stream)
    return nil
}

Some notes:

sh is necessary, it's an alpine image
injection.Container just holds information about the container, it's a custom struct
Docker is a struct, that holds the docker client (an instance from Client at github.com/docker/docker/client)

What I get as a result to the cli, if I execute my application, is something like this:
/usr/app $ ^[[43;12R
I far as I know, the ^[[43;12R is the ANSI escape code for the position of the cursor.
I can execute commands, like ls or npm i whatever, but I alwasy get back these ANSI escape codes.
My question is, is there some way to remove these from the stdout?
Answer: I eventually found out.
The problem was, that I should use the github.com/docker/cli/cli/command package and its DockerCli instead of os.Std.... This can manage this for me, by setting the output, error and input stream like this:
func (docker *Docker) holdHijackedConnection(resp types.HijackedResponse) error {
    cli, _ := command.NewDockerCli()
    outputStream := cli.Out()
    errorStream := cli.Err()
    inputStream := cli.In()

    inputStream.SetRawTerminal()
    defer inputStream.RestoreTerminal()

    receiveStdout := make(chan error)
    if outputStream != nil || errorStream != nil {
        go func() {
            receiveStdout <- docker.redirectResponseToOutputStream(outputStream, errorStream, resp.Reader)
        }()
    }

    stdinDone := make(chan struct{})
    go func() {
        if inputStream != nil {
            io.Copy(resp.Conn, inputStream)
        }
        resp.CloseWrite()
        close(stdinDone)
    }()

    select {
    case err := <-receiveStdout:
        return err
    case <-stdinDone:
        if outputStream != nil || errorStream != nil {
            return <-receiveStdout
        }
    }

    return nil
}

If you want to add CTRL+C to escape, you should set the DetachKeys in ExecConfig at ContainerExecCreate. Otherwise executing exit will detach it.


Using Newtonsoft JToken.ToObject to deserializeNodaTime Instant

Question: I am using NodaTime throughout my project as a replacement for the standard BCL DateTime classes, with amazing results.
I would like to banish DateTime from my project completely. I am only using it in one place - where I deserialize a large JSON file that contains dates. These dates are always UTC and always specified in ISO 8601 format.
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Schema;

namespace Foo.Parsing
{
public class Parser
{
    JSchema schema;
    JObject jsonObject;

    public Parser(string jsonContent, string schemaString)
    {
        schema = JSchema.Parse(schemaString);
        jsonObject = JObject.Parse(jsonContent);
    }

    public MyObject Parse()
    {
        return jsonObject.ToObject<MyObject>();
    }
}

public class MyObject
{
     [JsonProperty(Required = Required.Always)]
     public DateTime date { get; set; }

     // A lot of other properties here.
}
}

This all works extremely well, but at the moment I have to use Instant.FromDateTimeUtc(myObject.date); to convert the DateTimes to Instant. Inside MyObject I have DateTimes and if I try to change this to Instant, I get the following error:

Newtonsoft.Json.JsonSerializationException' in Newtonsoft.Json.dll
  Additional information: Error converting value 2016/07/07 12:00:00 AM to type 'NodaTime.Instant'. Path 'date', line 4, position 37

Is there any way for me to deserialize my JSON file to my strongly typed MyObject, with my dates deserializing directly to Instant instead of having to convert a DateTime?
Comment: Are you using the [`NodaTime.Serialization.JsonNet` package](https://www.nuget.org/packages/NodaTime.Serialization.JsonNet)?  It has converters that should help with this.  See [Deserializing Noda Time's LocalDateTime with JSON.NET](http://stackoverflow.com/q/21050002/10263) for details on setting this up.
Comment: Thanks, that has gotten me a step further, though now I am getting a error saying "Unexpected token parsing Instant. Expected String, got Date." How does the JObject know that it's a date? I notice when I set MyObject.date to be a string rather than an Instant/DateTime it comes back in a different (non ISO) format, so something in JObject.Parse seems to be recognising the format as a date...
Answer: Per the Noda Time User Guide:

Install the NodaTime.Serialization.JsonNet nuget package
Call ConfigureForNodaTime on your JsonSerializerSettings

If you're still getting "Unexpected token..." messages, then likely you have changed the DateParseHandling setting.  The ConfigureForNodaTime call will set this to DateParseHandling.None, and it should be left that way.
Note that this works well with the JsonConvert.SerializeObject and JsonConvert.DeserializeObject classes, as they accept a JsonSettings parameter, which can also be set via JsonConvert.DefaultSettings.  Here is a live demo using this approach.
However, if you're using the JObject APIs, there is one slight problem.  JObject.Parse has no ability to take a settings parameter, and by default you won't have the DateParseHandling.None that is required at this stage.  The solution to this involves using a JsonTextReader, and is described well in this StackOverflow answer.
Comment: No, I call Configure right before I deserialize. I tried adding "DateParseHandling.None" as follows. Still no dice:

`JsonSerializer serializer = new JsonSerializer();`
`serializer.ConfigureForNodaTime(DateTimeZoneProviders.Tzdb);`
`serializer.DateParseHandling = DateParseHandling.None;`
`return jsonObject.ToObject(serializer);`
Comment: See my updated answer.  I believe the last part address your current issue.


Finding $f(x)$ when given a composite function?

Question: CONTEXT: Uni homework question made up by lecturer
If, for $n=0$, $1$, $2$, $\ldots$ you're given $f_0(x)=\frac{1}{2-x}$ and $f_{n+1}=f_0(f_n(x))$, how do you prove that your formula for $f_n(x)$ is correct by mathematical induction?
UPDATE
I have computed the first few terms:
$f_1(x)=\frac{2-x}{3-2x}$
$f_2(x)=\frac{3-2x}{4-3x}$
$f_3(x)=\frac{4-3x}{5-4x}$
$f_4(x)=\frac{5-4x}{6-5x}$
Then, from observation of these I developed:
$f_n(x)=\frac{n+1-nx}{n+2-(n+1)x}$
But I don't see how you're supposed to prove this with mathematical induction as it has variable $x$ in it as well as the $n$.
Comment: You know $f_1$, since $f_0$ is known. Then you know $f_2$, since $f_1$ is known, etc .... . Try to find a pattern.
Comment: @Rócherz I appreciate your effort to improve posts. But if a question is put on hold, it's better to avoid minor edits. In this instance, your edit pushed the question into the [reopen review queue](https://math.stackexchange.com/review/reopen/1167582) - and since there were no substantial changes to the post, it's not too surprising that the reviewers voted to leave closed. More details can be found in a [post on meta about this problem](https://math.meta.stackexchange.com/questions/16820/should-i-avoid-minor-edit-of-a-question-which-was-put-on-hold).
Comment: @user21820 I understand your answer, thanks for the help! :)
Comment: Observe that $f_{3n} = \frac {x-1} {2-x}$ for all $n \geq 1,$ $f_{3n+1} (x) = \frac {x-2} {2x-3} , f_{3n+2} (x) = \frac {2x-3} {x-1}$ for all $n \geq 0.$
Comment: Just calculate $f_{n+1}$ from $f_{n}$ and check if the pattern holds
Comment: Do you understand my answer? The induction taught in many schools is just **not** right, and if you want to learn proper mathematics you must ensure that you understand everything in my answer.
Comment: Ok good, you're welcome! =)
Answer: Said in very broad terms, induction is the process of solving some problem for all values of $n$ when you can solve it for some $n_0$ and can derive the solution of the problem $n+1$ knowing the solution for $n$.
In your case, the problem is to "find the symbolic expression of $f_n(x)$", which you managed perfectly.

$$f_0(x)=\frac{n+1-nx}{(n+2)-(n+1)x}=\frac{1}{2-x}$$ is true.
And
$$f_n(x)=\frac{n+1-nx}{(n+2)-(n+1)x}\implies f_{n+1}(x)=f_0(f_n(x))=\frac{n+2-(n+1)x}{(n+3)-(n+2)x}$$
is true as well, as is shown by reworking the expressions.
Answer: I think you have not been taught what induction (along the naturals) really is. Induction should always be understood as a rule that can be applied to any property $Q$ on $\mathbb{N}$. When we say that $Q$ is a property on $\mathbb{N}$ we mean that $Q(k)$ is a statement about $k$ for any $k ∈ \mathbb{N}$. Note that if $Q$ is defined in any context where we have other objects, $Q$ can also depend on those other objects.
In your case, you should be given (the quantifiers are erroneously missing from the question):
  $f_0(x)=1/(2−x)$ and $f_{n+1} = f_0(f_n(x))$ for every $n ∈ \mathbb{N}$ and $x ∈ \mathbb{R} ∖ {2}$.
And in this context you want to prove:
  $f_n(x) = (n+1−nx)/(n+2−(n+1)x)$ for every $n ∈ \mathbb{N}$ and $x ∈ \mathbb{R} ∖ {2}$.
It suffices to do the following:
  Given any $x ∈ \mathbb{R} ∖ {2}$:
    [In this context you have $x$.]
    Define $Q(n)$ to assert that $f_n(x) = (n+1−nx)/(n+2−(n+1)x)$, for each $n ∈ \mathbb{N}$.
    Then $Q(0)$ is true because ...
    Given any $n ∈ \mathbb{N}$ such that $Q(n)$ is true:
      ...
      Thus $Q(n+1)$ is true.
    Therefore by induction $f_n(x) = (n+1−nx)/(n+2−(n+1)x)$ for every $n ∈ \mathbb{N}$.
Answer: You have already shown the base case for your induction.  Suppose that your proposition
$$f_n(x) = \frac{n+1-nx}{n+2-(n+1)x}, $$
holds for $n=N$.  We must show that the result also holds for $n=N+1$.  Following the formula you were given for computing $f_{N+1}(x)$ we have,
$$ f_{N+1}(x) = f_0(f_N(x)) = f_0 \left( \frac{N+1-Nx}{N+2-(N+1)x} \right),$$
where we have used the inductive assumption to substitute $f_N(x)$.  Now we apply $f_0$,
$$ f_{N+1}(x) = \frac{1}{2-\frac{N+1-Nx}{N+2-(N+1)x}}. $$
We now simplify this expression,
$$f_{N+1}(x) = \frac{N+2-(N+1)x}{2(N+2-(N+1)x) - (N+1 - Nx)} $$
and collect together all of the like terms
$$f_{N+1}(x) = \frac{(N+1) + 1 - (N+1)x}{((N+1) + 2) - ((N+1) + 1)x}.$$
This completes the inductive argument.
Answer: In fact having $f_n(x)$ in hand, we need to prove that $$f_{n+1}(x)=\frac{n+2-(n+1)x}{n+3-(n+2)x}$$while this is easy since $$f_{n+1}(x){=f_0(f_n(x))\\=f_0\left(\frac{n+1-nx}{n+2-(n+1)x}\right)\\={1\over 2-\frac{n+1-nx}{n+2-(n+1)x}}\\={1\over {2n+4-2nx-2x-n-1+nx\over n+2-(n+1)x}}\\={1\over {n+3-nx-2x\over n+2-(n+1)x}}\\=\frac{n+2-(n+1)x}{n+3-(n+2)x}}$$and the proof is complete $\blacksquare$
Answer: Just calculate $f_{n+1}$ from $f_{n}$, with both depending on x and check if the pattern holds. The variable x is irrelevant in this case, as you perform your induction on n.
Or, if you prefer it in these words for better understanding, the "pattern" has to still be true for every value of x


How do I make a twitter scheduler on drupal 7?

Question: I basically want to make a twitter scheduler. What it does is ,it fetches the old tweets from my twitter account and queues them so that I can schedule them to be sent at a later time.
So the three main jobs of this would be --

Fetch tweets from my account.
Queue them
Schedule them to be tweeted as per the user.

Could someone please tell me what all modules Id require for the same?
Thanks
Appreciate it.
Answer:
Create a node type 'Tweet'
Create a Feed Importer
Add JSON mappings
Add the Import URL
Use some custom code or perhaps Rules in combination with Tweet module api to schedule and send tweets

Modules:

Feeds
Tweet
JSONPath Parser
Rules

Credits: http://www.bariswanschers.com/blog/easily-import-tweets-drupal-using-json-and-feeds
Comment: Thanks but I already found a module called Twitter DB to fetch the tweets. It stores all of them in a database. But I cant figure out how to access the tweets from the database and schedule them . Are there any pre existing modules for it? Or do I need to create a custom one?
Modules:
https://www.drupal.org/project/twitter_db
Comment: Are you trying to "re-tweet" the tweets that the twitter_db module has pulled down from some account? If so, some combination of the suggestion made by undersound would still probably be the solution, although I'm not sure of which api calls are exposed by the module to do the tweeting.
Comment: Ok that information was not in your initial question :)   Looking at twitter_db module you could retrieve the tweets with the _twitter_db_get_last_tweets() function. You wil have to write a custom module for that (https://www.drupal.org/developing/modules/7)


Will Playing a Guitar Through a Bass Set-Up Damage the Amplifier and/or Speaker?

Question: I am moving and have sold my guitar amp, but still have my electric guitar. I've been playing it unplugged and I'd like to plug it into my bass combo amp.
Am I right in being afraid of damaging either the speaker or amp by plugging it in?
I realize that a similar question has already been asked, but it asks about plugging an electric bass into an electric guitar rig.
Comment: Note that [one of the most-acclaimed and influential guitar amps](http://en.wikipedia.org/wiki/Fender_Bassman) is actually a bass amplifier. (Modern bass amps are typically designed quite differently though, much cleaner than what you normally want for guitar.)
Answer: No, it will not damage the speaker. The high tones of a guitar are simply not capable of damaging your bass amp. Even if the amp isn't really suitable for these high tones (and in many cases, they can actually play them just fine), the amp will perfectly survive them. No harm there. The other way around is a much worse idea, because deep bass tones tend to be quite destructive to amps that can't play them!
If you're looking for a decent sound though, you'll notice that a bass amp probably sounds too plain for a guitar. A guitar amp has several effects that make the guitar sound better, which are absent in a bass amp. Unless you're really looking for that specific sound (or going to alter it with floorboards), there isn't really a reason to specifically use a bass amp.
Comment: There are some songs that use bass guitars plugged into amps for the distortion (overdrive off; clean signal distorted because that's what the amp does)
Answer: Speaker damage is caused by either the cone moving too much, or the voice coil overheating from distortion.
Most sounds from modern amps will not cause either, regardless of the instrument you connect. There is nothing that says you cannot plug a synth or a guitar into a bass amp, or a bass or synth into a guitar amp. If it sounds good, you can use it. Generally the sound of a good speaker is what you want in front of a microphone, or as a part of a band.
Comment: My coffee is all over my keyboard and screen now. Playing a bass guitar through a guitar amp can **definitely** damage the amp in case you turn it up a bit too much. Speakers can suffer from trying to play sounds that are too low for them, so it's not a good idea to try this.
Comment: @LeeWhite: It should certainly be possible to design an amplified speaker in such fashion that the electronics would never drive the voice coil beyond its safe mechanical limits, and I would expect that many are in fact designed in that fashion.  An amp driven hard enough that the electronics had to restrain the output would probably sound lousy until the volume was reduced, but not as bad as one with a blown voice coil.  I don't know if manufacturers clearly indicating whether amps will be robust in the face of any remotely-reasonable input signal, but it should be possible.
Comment: I guess it would be possible to design a guitar amp that couldn't be damaged by playing a bass through it, and given enough clever circuitry or DSP could even still produce the kind of sound when overdriven that guitarists actually want, but it's a lot simpler for the manufacturer to say 'this is a guitar amp, for guitars - if you want to play bass get a bass amp'.
Comment: Can't see why this answer is being questioned.
Answer: Buddy Holly used a Fender Bassman Amp feed by his Stratocaster. Great sound and and that is why I also use a Fender Bassman, but also, I play bass and need only the one amp undistorded and that suits my use right down to the ground.
Comment: nice! https://equipboard.com/pros/buddy-holly


Correctness of a statistical evaluation of a parameter

Question: I have a question about a non-Gaussian distributed parameter that can only take certain values in a defined interval.
Knowing that I have to define this parameter starting from a set of its values and in the end I must use only average value and tolerance, I am asking myself if the mean value should be calculated in the whole set, or only inside the tolerance.
I'll try to explain my situation more in detail: I know that  I have to consider only 84% (this is incorrect! +/-1.5*sigma is 86.6%) of the original set of values (cutting the same percent from the head and from the tail) and those considered should be those who give me the esteem I am looking for. While in the case of a Gaussian I would use avg value and +/- 1.5 * standard deviation to have in the end my parameter and its tolerance (yes, in that case I would be a little higher than 84%, but I'm really looking for 84% of the values - also 86.6, not 84 ),

This picture is incorrect. The percentage should be 86.6%
in the current case I must decide whether to calculate an avg value (weighted by probability of occurrence of the value) on the whole set or on the "cut" set and eventually to decide if it is better to calculate the tolerance as the maximum deviation of the 8th-percentile/92nd-percentile (really the 6.7th and the 93.3rd) from the avg value or as the average of the deviations of both, or whatever... I am not sure here too.
Below a chart Values vs. Probabilityof my parameter (in this case avg valuehas been calculated on the original set):

This picture is incorrect. The percentiles should be 6.7 and 93.3
Blue line is a trendline made with Excel, the columns include all the values between those shown in the x-axis and the next one. This representation is maybe not the best one ever, but helps to understand how the distribution goes.
Which are the most correct options?
Comment: The last figure is the histogram of your parameter calculated over all possible values it takes right? 3 sigma rule is just a rule of thumb to avoid outlying observations. There are much better ways than that.
Comment: Yes it is. Can you tell me any way better than this? (must however discard first and last values too)
Comment: i don't understand - are you asking whether to use a continuous rv or a discrete one?
Comment: or whether to take the mean before or after the chop - in theory there is no difference except for in extreme circumstances - this makes for an interesting proof!
Comment: or the classical 'mauve or purple'?
Comment: all of which needs a context. what area do your statistics come from? Average number of cars per household? Shoe size over age? Daily amount spent on  food? This is important in order to answer your Q.
Comment: @JonMarkPerry I don't quite understand what you mean by "continuous rv" or "mauve or purple". Anyway yes, I am asking you if the mean should be taken before of after the "chop" and in that case what is the best "deviation" that describes the final set of values.
The context is a bit complicated, those would be values of parameters of an electromagnetic device measured vs. frequency... And more than one parameter could have such charts
Answer: The issue that you rised with this question is in the area of robust statistics. In the case of estimating a parameter, it is called robust parameter estimation. There is a good book by Huber. I think this one will help alot.
The idea is as follows. When you are estimating a parameter, the regular process first finds the log likelihood ratio of the density function. Then tries to find the parameter such that the log likelihood function is maximized. Therefore, it is called maximum likelihood estimator (MLE). In many practical applications, the data under the test contains some outliers, which are the data samples that are inherently wrong and which do not follow the given density function. This can happen for example when a patient's EEG data is recorded and the patient moves his/her head involuntarily.
Let $f$ be the density function and there are $n$ data samples, each denoted by $x_i$. The maximum likelihood estimator is found by solving
$$\hat\mu=\arg\max_{\mu}\sum_{i=1}^n \log f(x_i,\mu)$$
The idea is to replace $\log f$ with some nice function $\rho$. Then the problem is
$$\hat\mu=\arg\max_{\mu}\sum_{i=1}^n \rho(x_i,\mu)$$
Assume that the interested parameter is the mean value of the distribution function. In robust estimation context, it is called the location parameter. For this case one can write
$$\hat\mu=\arg\max_{\mu}\sum_{i=1}^n \rho(x_i-\mu)$$
Now as an example, if $\rho(x)=x^2$. Then this corrsponds to the maximum likelihood estimator of the location parameter of Gaussian distribution. If you just take the derivative and make it equal to $0$, you will find
$$\hat\mu=\frac{1}{n}\sum_{i=1}^n x_i$$
If you choose $\rho$ differently? For example if you choose $\rho(x)=|x|$, this corresponds to a very robust estimator. This is actually the maximum likelihood estimator for meadian. But For Gaussian distribution, mean and median are the same and it depends on how much problem you have at the tails of the distribution. By Huber, there is a very nice transition from mean to median by the function
$$\rho(x)=\begin{cases}x^2\quad \mathrm{if}\quad |x|<c\\c(2|x|-c)\quad \mathrm{if}\quad \mathrm{otherwise} \end{cases}$$
With this nice function, one can trade of the strength of the estimator against outliers. In other words, if $c\to 0$, this estimator is simply the median estimator and if $c\to \infty$, it is the MLE of location estimator.
Coming back to your question, if you are completely sure that higher absolute values of your observations are clean and following the Gaussian distribution, then you must use all data points.
If you know that your data may be contaminated, then one needs to consider robust estimators. There is a trade-off between robustness and the efficiency, this can be adjusted by choosing a suitable value of $c$ as given above.
Comment: This is a great answer (even though I didn't understand: is "c" the number of measurements/samples?) and first of all let me thank you for it.
I **am not** sure that my measurements are all contained between what presented in the chart above, since (as stated in some previous comments), the values are measurements of an electromagnetic device vs. frequency. I can measure **only some of those frequencies**, therefore I cannot be sure. I have, though, a specification (whitepaper) that says that my measurements are **enough** to describe the current parameter in that frequency-set.
Comment: Correct me if I'm wrong: I must in this case assume that my samples are **all inside** the max and min values in the above chart. In this case I'd have to use the mean on all values, this is clear. What your answer DID NOT specify is, actually, how to introduce a **tolerance**. My parameter must be described by 2 values: mean and tolerance. _(tolerance would be the 1.5*St.Dev. if I had a gaussian distribution)_
Comment: $c$ is the tolerance parameter in the above given setting. For example if $c=1.345$, then this corresponds to $95\%$ efficiency of the estimator for the normal distribution. If there is a predefined tolerance as $1.5\%$, then yes, it is the $3$ sigma rule and it means one needs to consider only the values lying in $3$-sigma for the estimation. On the other hand, if some data samples are not observable, then there may be a bias upto some degree. In this case the approach would be bias correction.
Comment: Yes, but in this case I'd have a specification that explicitly says not to consider any more values... So basically I have to assume that all I have is all that can be observable.
So let's say I would use the mean on all the values (BEFORE the crop), how should I choose my tolerance? If I had a gaussian, I could have my parameter written as **m +/- c** where **m** is my mean and **c** the tolerance... Now I only have a clue on **m**. Is there no other solution rather than "split" the tolerance in + AND - ?
Comment: Up to a certain degree, the answer is correct. For the sake of simplicity, at the end I am using the average of the tolerances (distances of 6.7th/93.3rd percentile from the average value calculated on all the samples). I am aware, though, that this is not the best option


Mostrar comentários tabela

Question: Bom dia galera,
Gostaria de saber se é possível eu listar no SQL-SERVER todos os comentários da minha "tabela", não comentários da coluna, mas sim apenas das "tabelas"
Comment: utilize o comando `sp_help [tablename]`
Comment: Eu tentei, mas não trouxe a Extended Properties da tabela
Answer: Há algumas formas de obter as propriedades estendidas de uma tabela. Uma delas é através da função fn_listextendedproperty():
-- código #1
SELECT sys.objects.name as [Nome da tabela],
       EP.name as [Nome do comentário],
       EP.value as [Descrição do comentário]
  from sys.objects
  cross apply fn_listextendedproperty(default,
                                    'SCHEMA', schema_name(schema_id),
                                    'TABLE', name, null, null) as EP
  where sys.objects.name not in ('sysdiagrams');

O código acima lista todas as propriedades estendidas. Se quiser pode limitar a uma propriedade em específico, acrescentado o filtro na cláusula WHERE.
No tópico Consulta para selecionar a descrição de uma coluna ela também é utilizada, mas para obter a descrição das colunas.
Answer: Você tem que inspecionar a tabela do sistema extended_properties
Veja o exemplo:
CREATE table Tabela (id int , campo char (20))

EXEC   sp_addextendedproperty 'Descrição', 'ID', 'user', dbo, 'table', 'Tabela', 'column', id

EXEC   sp_addextendedproperty 'Descrição', 'Campo', 'user', dbo, 'table', 'Tabela', 'column', campo

Para listar os comentários basta fazer:
select *
from sys.extended_properties
where NAME = 'Descrição'

Veja funcionando no SQLFiddle
Verifique a documentação da tabela Sys.extended_properties para maoires detalhes.
Answer: Segue abaixo script que fiz e retornou:
select ob.name,ep.value from sys.extended_properties ep
 inner join sys.objects ob ON ep.major_id=ob.OBJECT_ID AND class=1
 where minor_id = 0 and ob.name not like '%sp%'

Mostrar comentários tabela


SocketTimeoutException connecting to database on Linux Virtual Machine

Question: Oracle NoSQL is a database that should be hosted on Linux, so I have a virtual Ubuntu running in Oracle VM VirtualBox. My host computer is running Windows 7.
I've got a bridged connection going, and I definitely am able to communicate between the two. With the database running on both machines, I can connect to the other machine's admin console, which runs on port 5001, through a web browser just fine. I can connect in that way using either the IP address of the machine on our network, or its network name.
From both machines, I can connect to that machine's database and use that database from a java program using either the IP address or the network name.
From the virtual machine, I can connect to and use my PC's kvstore using the network name, but I get a socketTimeoutException using the IP address. I believe that is due to a limit in the kvstore.
From my pc I cannot connect to and use the virtual machine's kvstore using the network name or the ip address. When I try to connect using the virtual machine's network name I get the following stacktrace:
Exception in thread "main" oracle.kv.FaultException: Could not contact any RepNode at: [pc30129vm:5000] (126.96.36.199.26)
Fault class name: oracle.kv.KVStoreException
    at oracle.kv.KVStoreFactory.getStore(KVStoreFactory.java:123)
    at nosql.Test.main(Test.java:18)
Caused by: oracle.kv.KVStoreException: Could not contact any RepNode at: [pc30129vm:5000]
    at oracle.kv.impl.util.TopologyLocator.getInitialTopology(TopologyLocator.java:226)
    at oracle.kv.impl.util.TopologyLocator.get(TopologyLocator.java:85)
    at oracle.kv.impl.api.RequestDispatcherImpl.<init>(RequestDispatcherImpl.java:285)
    at oracle.kv.KVStoreFactory.getStore(KVStoreFactory.java:118)
    ... 1 more
Caused by: java.rmi.ConnectIOException: Exception creating connection to: pc30129vm; nested exception is:
    java.net.SocketTimeoutException: connect timed out
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:632)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
    at sun.rmi.registry.RegistryImpl_Stub.list(Unknown Source)
    at oracle.kv.impl.util.TopologyLocator.getInitialTopology(TopologyLocator.java:175)
    ... 4 more
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:75)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:157)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
    at java.net.Socket.connect(Socket.java:579)
    at oracle.kv.impl.util.registry.ClientSocketFactory.createSocket(ClientSocketFactory.java:300)
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
    ... 9 more

I've tried running Nmap on that port on the virtual machine, and that seems to come back fine. With the nmap command:
nmap -p 5000 pc30129vm

I get the result:
Starting Nmap 6.25 ( http://nmap.org ) at 2013-03-18 14:51 Pacific Daylight Time

Nmap scan report for pc30129vm (192.168.0.25)

Host is up (0.0030s latency).

PORT     STATE    SERVICE

5000/tcp filtered upnp

MAC Address: 08:00:27:44:A1:10 (Cadmus Computer Systems)

Nmap done: 1 IP address (1 host up) scanned in 11.74 seconds

I just can't figure out why I can't connect to and use the virtual machine's database. With a coworker's laptop which runs linux I was even able to connect, but for some reason the virtual machine seems to stymie my efforts.
Comment: Yes, you most likely have a firewall in place that prevents you from accessing that port. The firewall may protect the server and reject incoming connections to it or perhaps it prevents outgoing connections from your machine -- or both, worst case scenario.
Comment: What does "filtered" mean in the `nmap` STATE column ? :)
Comment: I don't actually know, but nmap.org tells me it means `Nmap cannot determine whether the port is open because packet filtering prevents its probes from reaching the port. The filtering could be from a dedicated firewall device, router rules, or host-based firewall software.`
Comment: When you can actually access a port, `nmap` reports "open" in the STATE column.
Comment: Well ... you were right. I thought I'd permanently turned that dumb firewall off, but it turned itself back on when I restarted the virtual machine! Thanks. You can put your response in an answer and I'll accept it if you want.
Comment: OK, I'm glad you solved it.
Answer: Comments pasted as an answer:
You most likely have a firewall in place that prevents you from accessing that port. The firewall may protect the server and reject incoming connections to it or perhaps it prevents outgoing connections from your machine -- or both, worst case scenario.
When you can actually access a port, nmap reports "open" in the STATE column.


how to include external php file in drupal 8

Question: I want to merge two xlsx file and download merged file in drupal 8. So i used a PHPExcel library for doing so.
But i am not able to include a PHPExcel.php file in custom controller for that. I used require_once, module_load_include() but does not work for me.
How can i include it . please help.
require_once DRUPAL_ROOT.'/'.drupal_get_path('module','custom').'/src/Controller/PHPExcel.php';

or

module_load_include('php', 'custom', 'src/Controller/PHPExcel');

$file1 = 'ford.xlsx';
$file2 = 'toyota.xlsx';
$objPHPExcel1 = PHPExcel_IOFactory::load($file1);
$objPHPExcel2 = PHPExcel_IOFactory::load($file2);

foreach($objPHPExcel2->getSheetNames() as $sheetName) {
$sheet = $objPHPExcel2->getSheetByName($sheetName);
$sheet->setTitle($sheet->getTitle());
$objPHPExcel1->addExternalSheet($sheet);
}

header('Content-Type: application/vnd.ms-excel');
header('Content-Disposition: attachment;filename="ford-toyota.xls"');
header('Cache-Control: max-age=0');
header('Cache-Control: max-age=1');
header ('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
header ('Last-Modified: '.gmdate('D, d M Y H:i:s').' GMT');
header ('Cache-Control: cache, must-revalidate'); // HTTP/1.1
header ('Pragma: public'); // HTTP/1.0

$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel1,
'Excel2007');
$objWriter->save('php://output');
exit;

and error is :
Fatal error: Class 'Drupal\custom\Controller\PHPExcel_IOFactory' not found

Answer: I am not sure from your question which phpexcel library you are using, so there are potentially different answers, here is one:
1. On Packagist
If like in the case of mvlabs/mvlabs-phpexcel you would do the following:
on the command line:
composer require mvlabs/mvlabs-phpexcel

Then in the file where you are wanting to use it you can simply add a "Use" statement (as per this answer):
use ExternalPackage\namespace\Foo;

Drupal 8 does an auto-include and is "aware" of all the PHP libraries inside the vendor folder.
Comment: i used composer require phpoffice/phpexcel


Smarty PHP, AbanteCart E-Commerce User Agent Theme

Question: I've been using the open source AbanteCart E-Commerce for selling productus and was wondering if someone knew there was a way to redirect the template for use of a mobile template instead.
Answer: i guess the easiest way is hook of all controllers and adding new variable into request (sf) which contains template_text_id... plus using cookie about template mode(desktop or mobile).
<code> /**
 * redirect to mobile template if we load page in mobile device
 **/
public function onAHook_InitEnd() {
    if ( $this->_processHooks() ) return;
    $device = $this->baseObject->request->getDeviceType();
    if ( !empty($device) && empty( $this->baseObject->request->cookie['abantecart_mobile_redirect'] ) ) {
        //set flag that redirect was done
        setcookie("abantecart_mobile_redirect", "1", time() + 3600*24);
        //set template to mobile
        $url = $this->baseObject->html->removeQueryVar($_SERVER['REQUEST_URI'], 'sf');
        $url .= '&sf=your_mobile_template_text_id';
        header ( 'Location: '. $url );
    }
}
private function _processHooks(){
        return $this->registry->get('config')->get('config_storefront_template') == 'your_mobile_template_text_id';
    }
</code>
That variable (sf) processed by engine in core/init.php file and switch desktop template to mobile.


computing the distribution over the latent function values with the form of a GP predictive

Question: If we have a latent state space $\mathbf{X}$ and the observations $\mathbf{Y}$ and the transition function between two states $\mathbf{x}_{t-1}$ and $\mathbf{x}_{t}$ is given by $\mathbf{f}$ which is a gaussian process with mean function $m_f$ and covariance $k_f$, where its graphical model is as follows:

The generative model is given by
$$f(x)\sim\mathcal{GP}(m_f(\mathbf{x}),k_f(\mathbf{x},\mathbf{x}'))\\
\mathbf{x}_0\sim p(\mathbf{x}_0)\\
\mathbf{f}_t=f(\mathbf{x}_{t-1})\\
\mathbf{x}_{t}|\mathbf{f}_{t}\sim\mathcal{N}(\mathbf{f}_{t},\mathbf{Q})\\
\mathbf{y}_t|\mathbf{x}_t\sim p(\mathbf{y}_t|\mathbf{x}_t,\boldsymbol{\theta}_y)$$
I would like to know how the following probability can be rederived
$$\mathbf{f}_2|\mathbf{f}_1,\mathbf{x}_{0:1}\sim\mathcal{N}(\mathbf{f}_2|m_f(\mathbf{x}_1)+k_f(\mathbf{x}_1,\mathbf{x}_0)k_f(\mathbf{x}_0,\mathbf{x}_0)^{-1}(\mathbf{f}_1-m_f(\mathbf{x}_0)),k_f(\mathbf{x}_1,\mathbf{x}_1)-k_f(\mathbf{x}_1,\mathbf{x}_0)k_f(\mathbf{x}_0,\mathbf{x}_0)^{-1}k_f(\mathbf{x}_0,\mathbf{x}_1))$$
Thanks!
Comment: @Ben note that $f$ itself is a random function, so $\mathbf{f}_2$ is random even when $\mathbf{x}_1$ is fixed.
Comment: According to the specification of your generative model, $\mathbf{f}_2$ is a deterministic function of $\mathbf{x}_1$, so the "distribution" of $\mathbf{f}_2 | \mathbf{f}_1, \mathbf{x}_{0:1}$ would just be a point-mass on $\mathbf{f}_2 = f(\mathbf{x}_1)$.  I take it this is not what you intended for the question?
Comment: Ah yes, sorry, I missed that bit.
Answer: If we assume a Gaussian Process prior on the function values,
$$P(f_1|\mathbf{x}_1)\sim\mathcal{GP}(m_f(\mathbf{x}_1),k_f(\mathbf{x}_1,\mathbf{x}_1))$$
then for the new state, we have
$$\begin{bmatrix}
    f_1\\
    f_2\\
     \end{bmatrix}
=\mathcal{N}\Bigg(
\begin{bmatrix}
    m_f(\mathbf{x}_1) \\
    m_f(\mathbf{x}_2)\\
\end{bmatrix},
\begin{bmatrix}
    k_f(\mathbf{x}_0,\mathbf{x}_0)       & k_f(\mathbf{x}_0,\mathbf{x}_1) \\
    k_f(\mathbf{x}_1,\mathbf{x}_0)       & k_f(\mathbf{x}_1,\mathbf{x}_1) \\
\end{bmatrix}\Bigg)
$$
the conditional distribution of $f_2$ given $f_1$ is itself Gaussian-distributed, we can use the Schur complement and write this as
$$f_2|f_1\sim\mathcal{N}(m_f(\mathbf{x}_1)+k_f(\mathbf{x}_1,\mathbf{x}_0)k_f(\mathbf{x}_0,\mathbf{x}_0)^{-1}(f_1-m_f(\mathbf{x}_0)),k_f(\mathbf{x}_1,\mathbf{x}_1)-k_f(\mathbf{x}_1,\mathbf{x}_0)k_f(\mathbf{x}_0,\mathbf{x}_0)^{-1}k_f(\mathbf{x}_0,\mathbf{x}_1))$$


Como evitar HTML injection e XSS em páginas .JSP?

Question: No PHP já vi usos de htmlspecialchars e mysqli. Mas no Java existe alguma forma de evitar XSS e HTML injection?
Também gostaria de saber qual seria a melhor maneira (a mais segura): Escapando os elementos (HTML, Javascript) ou inserindo uma tag
 para visualizar os dados para que isto não afete a página?
Comment: De cara, sem pensar muito já digo que a tag `` não resolve sem mais algum outro tipo de tratamento mais complexo junto, pois nesse caso o hacker começaria o HTML injetado com `` e o terminaria com ``.
Comment: [Para ler enquanto não aparece uma resposta](http://blog.caelum.com.br/prevenindo-ataques-de-html-injection/).
Comment: Yeap. Eu vi uma implementação no WhatsApp Web em que usa-se a tag `` em conjunto com ``
Answer: JSP's
A maneira mais simples, é utilizar a tag out da biblioteca JSTL Core. Exemplo:
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<c:out value="${suaVariavel}" />

seria o mesmo que:
<c:out value="${suaVariavel}" escapeXml="true" />

Exemplo mais detalhado neste link de referência
Um erro comum
Achar que a JSTL já vem dentro dos containers WEB e esquecer de colocar a JSTL no projeto, que pode ser encontrada aqui, ou, ser importada como dependência em um projeto maven:
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.2</version>
</dependency>

Vale a pena dar uma olhada ...
ESAPI - Segurança extra para aplicações WEB
Uma lib utilitária que implementa diversos mecanismos de segurança. Permite ler de forma segura cookies, parâmetros da request, sessão e outros.
Answer: Como o Victor disse, o
 não evita isto, talvez você esteja pensando que está tag converta o conteúdo para entidades, mas talvez seja apenas uma confusão.
Eu pessoalmente, permito a gravação exata do que foi "escrito", mas no momento da leitura que eu utilizo métodos semelhantes ao htmlspecialchars, assim evita conflitos, pois os textos não sofreram alterações na gravação.
Se a sua preocupação é ler os dados de um banco e imprimir na tela, você pode usar no momento de ler os dados da linha:
import org.apache.commons.lang.StringEscapeUtils;
...
String data = StringEscapeUtils.escapeHtml("");

Não sei como está o seu código, e qual framework você usa, mas acredito que o sentido seja sempre este, "escapar" o conteudo na hora de exibir (corrija-me se eu estiver enganado).
Documentação: StringEscapeUtils (Commons Lang 2.6 API)
Comment: Só a ter cuidado com gente que crafta coisas como `` (eu nao sei se o editor deixa, mas existe um line-break entre ``) e o browser consegue perceber que isso === `
Comment: @MoshMage se queres mostrar como HTML mesmo o DOMPurify parece bom, mas se não quer mostrar então converter o HTML em entidades acho que já basta.
Comment: Como XSS normalmente é para correr na maquina dos clientes, injecta-se em sítios que se pode usar html; Imagina um WYSIWYG que fica desprotegido? Ele quererá mostrar como HTML *e* não lhe apetece nada ter ali uns scripts escondidos ;P
Comment: Acho que você não entendeu o que eu disse @MoshMage, vou tentar fazer mais claro para você o texto, **SE não** houver necessidade de mostrar o HTML gerado pelo usuário o `StringEscapeUtils.escapeHtml` é suficiente, **SE há** a necessidade de mostrar o HTML criado pelo usuário então ferramentas semelhantes ao DOMPurify são boas soluções. Entendeu? Eu não te critiquei, só coloquei o momento da necessidade.
Comment: Na boa :) Mas eu referi mesmo o dompurify porque realmente li mal a pergunta, percebi que ele queria mostrar o html. my bad ^^


Switching to Java 1.8 on openSUSE 13.2

Question: I removed the installed version of Java 1.7 using:
sudo zypper remove -y java-1_7_0-openjdk java-1_7_0-openjdk-devel java-1_7_0-openjdk-plugin

Then installed Java 1.8:
sudo zypper install -y java-1_8_0-openjdk java-1_8_0-openjdk-devel java-1_8_0-openjdk-plugin

But when I run java -version I get:
java version "1.7.0_71"
OpenJDK Runtime Environment (IcedTea 2.5.3) (suse-1.1-x86_64)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)

Am I missing a symbolic link somewhere?
Answer: This would be better as a comment, but I don't have that privilege.
Do you have a proprietary Java package?  Did you re-logon?  Check what is installed using sudo zypper search -i java.  You should be able to see whatever is providing java version 1.7 and remove that package.  It is possible to manually install proprietary java packages, and indeed there are manually-created symlinks in the instructions (which I have never successfully implemented) but I would think finding and removing the package providing 1.7 would be the place to start in this case.
Comment: I had to uninstall `java-1_7_0-openjdk-headless`. Your answer this led me to this.  Thanks.

Switching to Java 1.8 on openSUSE 13.2


How to build a song arrangement from a melody

Question: You have the melody and know what and when the chord changes are. Now you want to build a rhythm section arrangement and need the following instrumentation added
1) Rhythm guitar
2) Bass
3) Drums
Where do you start and what should you keep in mind when choosing a strum pattern, bassline & drum pattern?
Comment: Honestly from my experience, imagine what you want the final product of the song to sound like. Then play that. Work towards that. Does it sound good? Does it showcase your musical skills or compositional skills?
Comment: Build around the root notes of the melody
Comment: @Jason, forgive me but what is the "root" note of the melody? Do you mean the tonic note of the key? The melody doesn't need to even have the tonic note in it at all so not sure I see how to apply that idea
Comment: I did mean the tonic.  Many melodies I've written revolve around a tonic.
Answer: A central creative action in most or all forms of art is transformation, and transformation is highly powerful in music. Just one example of the power of transformation is a fugue, where a relatively short musical statement is transformed many times to create an entire work lasting many minutes.
Usually, once you have the germ of a musical idea, you can generate many related ideas using transformation.
In this case, you have two germs. You have the rhythm that is the rhythm of the melody, and you have the harmonic rhythm of the chord changes. Changing the harmonic rhythm will change how the melody sounds, so that is something you can play with.
To write the rhythms for any instruments that will have a rhythmic contribution, you can base their rhythms on the vocal rhythm. You want to pay attention to how the rhythm instruments accent or diminish the impact of the notes of the vocal rhythm. When choosing what notes to highlight or diminish in impact, consider both the melodic function of the note and the verbal function of the lyrics (if any) surrounding that note.
Pick any one rhythm instrument, write a rhythm for it that has the desired effect with the vocal line, and then have other rhythm instruments double that rhythm or "part out" the rhythm between the instruments. The simplest way to start with the first rhythm is to simply mirror the rhythm of the melody. A simple early transformation is to take one statement of the rhythm of the melody and keep repeating it even when the melody rhythm changes.
Other types of transformation are available, such as rhythmic sequence, retrograde, etc.
Comment: Thanks. When I said i had the chord changes, I meant just the chords not the actual rhythm part or strum pattern. What you said about the rhythm of the vocal I found interesting especially since the vocal in question does something different every few bars so is hardly steady enough to be used as a rhythmic idea. Do producers really just play around until they find something that fits or do they look for certain criteria? in 4/4 there are hundreds of drum grooves so what makes one sound better than the other?


How determine if application is web application

Question: In a core assembly, which is run both in a windows service, and in a web application, I need to store information per user session. The service will have a single user session, and the web application uses HttpContext.Current.
I want to configure which method to use within the core assembly - convention over configuration. I want to do this only once, and I believe HttpContext.Current will be null when run from Application_Start.
How can I reliably determine if the application is a web application?
Answer: if(HttpRuntime.AppDomainAppId != null)
{
  //is web app
}
else
{
  //is windows app
}

Comment: HttpRuntime is a good solution for framework apps, but System.Web doesn't exist in core / standard st this point in time.
Answer: I'd go for
HostingEnvironment.IsHosted

Note that there is a slight overhead incurred when you're using a method from an assembly like this, even when you don't intend to use it. (System.Web will be loaded and several classes could be initialized and JITed.) Also, there's a hard dependency on System.Web now, so you can't use it in a limited framework setting (currently IIRC only with the Client Profile).
Another way (although not as neat and documented), is to check
Path.GetFileName(AppDomain.CurrentDomain.SetupInformation.ConfigurationFile)

If this returns web.config (or a casing variant thereof), it's probably a web application. (Although you can setup any appdomain with a config file named web.config, this is not a likely scenario.) This avoids taking a dependency on System.Web.
However, HostingEnvironment.IsHosted is intended to indicate whether an appdomain is configured to run under ASP.NET.
Comment: Thank you for the details here. This worked for me when other things did not (trying to detect if it was a gui or a console, especially while working in debug in VS2022). `HostingEnvironment.IsHosted` worked the best for me.
Answer: Just so no one else are making the same mistake as me.
Assembly.GetEntryAssembly() do not work to define if its an web application or a not. When it's running as a service then Assembly.GetEntryAssembly() are null, but when i debug from VS, it's not null.
Answer: In web application Assembly.GetEntryAssembly() is null. I use it in two libraries and so far it works great.
Comment: I second this solution. Just tested it with code running in a web context via ASP.NET and also in a Windows Service.
Comment: This means exactly what the name implies: that there is no .NET entry assembly, which is not the same as running in a web app or service.  For example, if someone uses your library via COM--say because they have a legacy desktop application that can't interoperate with .NET in any other way--then entry assembly can also be null and OP's code may incorrectly behave as if there were a licensing violation.
Answer: If possible I'd suggest having it as an input parameter to some initialize method in the class library that would need to be called before the class library can be used.
If that's not an option I'd look at HttpRuntime.Cache which I think would be non null even if HttpRuntime.Current is null. I'm not a webforms guy but I remember someone mentioning that for a similar question sometime somewhere (can't find that question now).


Verifying steps for ARIMA with exogenous variables

Question: I have a time series data with two exogenous variables. I am using auto.arima from the forecast package to determine best fit. I wanted to know if I am implementing the auto.arima function correctly since I believe I am getting good forecast results. The figure below shows the sample time series of having 200 data points and 200 instances of exogenous variables (Var1, Var2)

I used the first 170 data points to fit ARIMA model and the next 30 data points for forecasting. I use auto.arima to find best fit ARIMA model.
model <- auto.arima(raw_data$timeseries[1:170], xreg=as.matrix(raw_data[1:170,2:3]));
pred <- forecast.Arima(model,h=30, xreg=as.matrix(raw_data[171:200,2:3]))

I get RMSE between actual and forecasted value as 0.00169 and the image is shown below (I have not shown 95% CI). The result seems acceptable however I have the following questions

Is this the correct implementation  (assuming I have already done all checks for stationarity, seasonality, correlations etc)
You can see form the example that ARIMA (0,1,0) is best fit model. The output from forecast.Arima is however not the differenced value but non differenced value. Does forecast.Arima inverse the differences?
Comment: why don't you post your entire data set and I will try and help you . The solution you are using apparently doesn't identify any needed lags or leads in the predictors or deal professionally/correctly with the possibility of unusual values such as pulses,level shifts,seasonal; pulses or local time trends effecting model form/parameters.
Answer: The approach you are using assumes that the two input (supporting series the X's) have a purely contemporaneous effect. The general approach to forming a ARMAX MODEL (Transfer Function)

is as follows.

develop arima models for the predictors
pre-whiten to identify the form of the TF Which test for lagged effect of one time series on another?
3) identify pulses,level shifts, seasonal pulses (the I's) and add to the model
4) identify any additional arima structure
5) identify any additional lag structures in X that may be needed.

examine residuals to identify any additional structure.
You might also peruse https://stats.stackexchange.com/search?q=user%3A3382+transfer+function for  more hints of forming a useful model
Comment: Doesn't auto.arima apply regression with ARMA errors rather than ARMAX?
Comment: i am not an expert on auto arima but I think I have seen output whicj include include fiixed/determinstic X's
Answer: 1. Is this the correct implementation
Why do you include the drift term if it is non-significant?
2. Does forecast.Arima inverse the differences?
Yes it does


What methods are used to solving regression problems in Machine Learning (like GLMs)?

Question: I'm going through Andrew Ng's lecture notes on Machine Learning.
The notes start with introducing linear regression and intuitively explaining what the cost function for the problem should be and how to minimize it etc. without any framework.
Then the notes proceed to explain Generalize Linear Models, and we see that how using GLMs we can derive ways to solve linear regression/ logistic regression problems. So here, GLMs seem like they're a common framework to solve a variety of machine learning problems.
So I was wondering, if there are other such similar frameworks for solving machine learning problems? And also, is there a name for solving such problems intuitively/logically without the use of a framework? (like the notes explain solving linear regression problems initially)
I realize that I'm just starting this course and I may learn of other more general methods (like SVM or Neural Nets?) ahead, but I just wanted to know what other methods exist? And when to choose which route to solve a given problem (if that's answerable)
Comment: What do you mean by "framework"?
Answer: It's a bit unclear what you mean by "framework"; for example, would Least Squares be the framework you're looking for when it comes to ordinary linear regression? Following Least Squares, GLMs is a broader "framework" that encompasses least squares regressions and more. Eventually, the later techniques you mentioned like SVMs or ANNs can also be considered frameworks.
If you look ahead into the course syllabus you should see what other methods the course covers. In addition, there's a great flow chart from SciKit Learn that describes many machine learning methods and when to use what.


How to delete document from Firestore

Question: Can't figure out how to delete document from Firestore.
Inside PopMenuButton I am unsuccessfully trying access postID by doing the following:
onSelected: (value) async {
            await FirebaseFirestore.instance.collection('thread').doc(FBCloudStore.  ).delete();
            //await FBCloudStore.deletePostInFirebase(postID);
          },

Here is my FBCloudStore, where I think I need to change something.
Move postID parameter to local variables and then what?
class FBCloudStore {
  static Future<void> sendPostInFirebase(String postID, String postContent,
      MyProfileData userProfile, String postImageURL) async {
    String postFCMToken;
    if (userProfile.myFCMToken == null) {
      SharedPreferences prefs = await SharedPreferences.getInstance();
      postFCMToken = prefs.get('FCMToken');
    } else {
      postFCMToken = userProfile.myFCMToken;
    }
    FirebaseFirestore.instance.collection('thread').doc(postID).set({
      'postID': postID,
      'userName': userProfile.myName,
      'userThumbnail': userProfile.myThumbnail,
      'postTimeStamp': DateTime
          .now()
          .millisecondsSinceEpoch,
      'postContent': postContent,
      'postImage': postImageURL,
      'postLikeCount': 0,
      'postCommentCount': 0,
      'FCMToken': postFCMToken
    });
  }

//Should it be like this?
  deletePostInFirebase(String postID) async {
    await FirebaseFirestore.instance.collection('thread').doc(postID).delete();
  }
}

And this is the thread item with PopupMenuButton.
  class ThreadItem extends StatefulWidget {
  final BuildContext parentContext;
  final DocumentSnapshot data;
  final MyProfileData myData;
  final ValueChanged<MyProfileData> updateMyDataToMain;
  final bool isFromThread;
  final Function threadItemAction;
  final int commentCount;

  ThreadItem(
      {this.data,
      this.myData,
      this.updateMyDataToMain,
      this.threadItemAction,
      this.isFromThread,
      this.commentCount,
      this.parentContext});

  @override
  State<StatefulWidget> createState() => _ThreadItem();
}

class _ThreadItem extends State<ThreadItem> {
  MyProfileData _currentMyData;
  int _likeCount;

  @override
  void initState() {
    _currentMyData = widget.myData;
    _likeCount = widget.data['postLikeCount'];
    super.initState();
  }
  @override
  Widget build(BuildContext context) {
    return Row(
      children: <Widget>[
        PopupMenuButton<int>(
          itemBuilder: (context) => [
            PopupMenuItem(
              value: 1,
              child: Row(
                children: [
                  Text("Delete post"),
                ],
              ),
            ),
          ],
          initialValue: 1,
          onCanceled: () {
            print("You have canceled the menu.");
          },
          onSelected: (value) async {
            print('delete');
            //TODo how to delete post?
            //await FBCloudStore.deletePostInFirebase(postID);
            //await FirebaseFirestore.instance.collection('thread').doc(FBCloudStore.  ).delete();
          },
        ),
      ],
    );
  }
}

Answer: In your onSelected function do the following:
onSelected: (value) async {
            print('delete');
            //TODo how to delete post?
            //await FBCloudStore.deletePostInFirebase(postID);
            FirebaseFirestore.instance.collection("thread").doc(widget.data['postID']).delete();
      },

Answer: Try below code hope its helpful to you.
 onTap: ()async{
   await FirebaseFirestore.instance
         .collection('collection name')
         .doc(your id here)
         .delete();
  }

How to delete document from Firestore


Finding the limit of (3^n)/(n^3) and how to tell if it is convergent or divergent

Question: In Calculus 2 we just started on doing sequences and I understand that to find the limit you can use l'hopital's rule and the sandwich theorem and a few other tricks but I'm generally confused on when to use what.
So we have this problem:   3^n/n^3
and we need to state if it diverges or converges, and if convergent, find the limit.
I really have no idea how to go about solving this, if someone can point me in the right direction it would be much appreciated. Thanks.
Comment: See what happens when you for instance double $n$. The numerator is squared, the denominator is multiplied by $8$. Which of those operations do you think is strongest in the long run?
Comment: Have you compared exponential functions to polynomials before?
Comment: I don't believe we've covered that yet
Comment: The strongest would be the one that becomes largest? So I believe 3^n would in the long run be larger than n^3
Comment: yes so every time you double n you multiply the number by 9/8 , so it should not converge
Comment: If you covered Taylor expansions you could just see what happens if you take the logarithm (since $\log$ is continuous and strictly increasing).
Comment: http://math.stackexchange.com/questions/1208683/the-convergence-of-an-alternating-series-test
Comment: Oh ok so the way to look at this is that because the numerator will be so much larger than the denominator for large numbers, the whole sequence tends to go towards infinity? Therefore, it's divergent.
Comment: @Jay That is the idea behind it, but that's all just intuition. A proof is required.
Answer: $$3^n=(2+1)^n=2^n+2^{n-1}\cdot \binom{n}{1}+\cdots+2\binom{n}{n-1}+\binom{n}{n}$$
When $n\ge 4$, there's a term $\binom{n}{4}=\frac{n(n-1)(n-2)(n-3)}{4!}$, which is a polynomial of degree $4$ with a positive $n^4$ coefficient. All the other terms of this expansion are positive. Ratio of higher degree to lower degree polynomial with positive highest degree coefficients as $x\to\infty$ is $\infty$.
(you can see this by dividing both the numerator and denominator by $x^k$ ($k$ -- numerator's highest degree))
Answer: You may observe that
$$
f(x)=x \ln (3/2)-3 \ln x >0, \quad x>24, \tag1
$$ since
$$
f'(x)=\ln (3/2)-\frac3x >0, \quad x>24.
$$ Then you deduce from $(1)$ that
$$
\frac{3^n}{n^3} > 2^n,  \quad n>24,
$$ and, as $n \to \infty$, $\dfrac{3^n}{n^3} \to+\infty$.
Answer: $$\frac{3^n}{n^3}= \left( \frac{\sqrt[6]{3}^n}{\sqrt{n}} \right)^6$$
Now, by Bernoulli inequality
$$\sqrt{3}^n \geq 1+(\sqrt[6]{3}-1)n>(\sqrt[6]{3}-1)n $$
Therefore
$$\frac{3^n}{n^3}\geq \left((\sqrt[6]{3}-1)\sqrt{n} \right)^6=\left(\sqrt[6]{3}-1\right)^6 n^3$$


Which were the most popular key signatures in the Medieval era?

Question: I am writing a piece of composition software and need information regarding different musical eras in order to make the algorithms work for different time periods. Therefore it would help to know if there were a few specific key signatures that were favoured back then.
The same information would also be helpful regarding the Renaissance, Baroque. Classical, Romantic, and Impressionist eras however they're not as much of a priority. Thanks ^_^.
Comment: Perhaps you could edit the question to remove the reference to "Renaissance, Baroque," etc.
Comment: This question is far too broad, as it covers the entire recorded history of Western music. If you asked about individual style periods in separate questions, that might work better.
Comment: Omg, I did say "the same information WOULD BE HELPFUL", I was hoping for the medieval era more than the others.
Comment: @Rariolu  Them don't add them to the question.  Your question should be about one specific thing w/o things on the side.
Comment: Why does it matter? I made it clear the main thing so why care? I just figured it would make more sense to add them in here than to create individual questions for each of them. I apologise for your lack of how conversation and vocabulary works in reality.
Answer: Remember that Key signatures and the general Major / minor tonalities where an invention of the Baroque era. In medieval times it was the church modes that where the big daddy.
I know this is not going to be the answer that you want but none of the key signatures where popular in Medieval times simply because they had not been invented yet.
Comment: I thought it a bit long for a comment.
Comment: Thanks, I was mislead by some medieval music that was written in the conventional format with a key signature included.
Comment: This is an excellent point. I'm torn between upvoting for its excellence versus suggesting it should be a comment in the sense that it's not so much an answer as suggesting there is no answer. The question also has 4/5 closure votes barring editing to make it less broad.
Comment: If you were to copy and paste it into a comment field, you would have 257 characters left, so in terms of the system length limit for a comment you would be fine. In terms of whether it is appropriate content for a comment, again I feel like it could go either way.
Comment: The only pitches that were used in medieval notation were the "white notes" of the piano, plus B flat. http://www.dolmetsch.com/medieval.pdf is a short introduction. Don't forget that a modern edition may have transposed the original notation for some reason.
Comment: But if you're talking about collections instead of keys, there were basically ones with no accidentals and ones with a B-flat. Other alterations were either conveyed with accidentals or were implied, i.e. not notated but expected to be performed anyway.


Python Selenium not finding element

Question: I am trying to find the other sellers of an product by the class name but no matter what it will not locate it. I have checked that is not in an iframe. Any help?
Python Code
 def similarproducts(self):
        print("Finding other sellers")
        try:
            self.othersellers = WebDriverWait(driver, 10).until(
                EC.presence_of_all_elements_located((By.CLASS_NAME, 'a-box mbc-offer-row pa_mbc_on_amazon_offer')))

        except TimeoutException:
            print("Found no sellers")

Link to page:
https://www.amazon.com/Bentgo-Kids-Childrens-Lunch-Box/dp/B00PKNO7LK/ref=sr_1_4?dchild=1&keywords=Bentgo%2BKids%2BPrints%2B(Camouflage)%2B-%2BLeak-Proof%2C%2B5-Compartment%2BBento-Style%2BKids%2BLunch%2BBox%2B%E2%80%93%2BIdeal%2BPortion%2BSizes%2Bfor%2BAges%2B3%2Bto%2B7%2B%E2%80%93%2BBPA-Free%2Band%2BFood-Safe%2BMaterials&qid=1624574710&s=home-garden&sr=1-4&th=1
Comment: What are you trying to do here? To get the other sellers name? To add to the  shopping cart from alternative seller?
Comment: @Prophet I am trying to find the price of the product, then if the product is cheaper or the same price as the original product, then it buys it.
Answer: The alternative seller price can be found by:
alternative_price = driver.find.element_by_css_selector("#mbc-price-1").text

Adding the product to the shopping cart can be done with:
driver.find.element_by_css_selector("#mbc-buybutton-addtocart-1 .a-button-input")

UPD
In case several alternative sellers available you can open the alternative sellers view by clicking this:
driver.find.element_by_css_selector("div.olp-text-box").click()

Now you can get all the prices:
price_els = driver.find_elements_by_xpath("//div[contains(@id,'aod-price-')]")
prices = []
for price in price_els:
    prices.append(price.text)

To add a product to cart you can use:
driver.find_elements_by_xpath('(//input[@name="submit.addToCart"])[x]').click()

where x is the index according to the price elements above.
To open the alternative sellers menu you can click this:
driver.find_elements_by_css_selector('div.olp-text-box').click()

To close alternative sellers view click this:
driver.find_elements_by_css_selector('#aod-close i.a-icon-close').click()

Comment: How about finding other sellers? This would work if there's always only one.
Comment: I saw no such pages, tried several. Can you give me a link or two with multiple "other sellers"?
Comment: the link I gave has multiple sellers. Under adding to list and what not
Answer: See if this works:-
otherSellers = driver.find_elements_by_xpath("//span[text()='Other Sellers on Amazon']//following::div[@class='a-box mbc-offer-row pa_mbc_on_amazon_offer']//following::span[contains(@id,'mbc-price')]")
for e in otherSellers:
    print(e.text)

Python Selenium not finding element