Describing the distribution of half-lives from highly uncertain data

Question: I am a medical doctor working on a project within the field of pharmacokinetics. A lot of detailing is required, so please bear with me.
The aim is to describe the distribution of half-lives of a drug in a population, using data that was not intended for this specific purpose. I would like to know whether my chosen strategy is sound, and I also have some specific questions. I am not a mathematician, so please forgive me if my take on this is non-standard.
Blood samples were collected from hundereds of individuals. Two consecutive blood samples were taken 30 minutes apart from each individual. For simplicity, we name the drug concentrations in the first and second blood sample $C_1$ and $C_2$, respectively, and the time between blood samples $\Delta t$.
We are assuming that the drug is eliminated from blood according to first-order kinetics. As such, the relationship between $C_1$, $C_2$ and $\Delta t$ can be described by a simple exponential decay function:
$$C_2=C_1\cdot e^{-k\cdot \Delta t}$$
In each case, the rate constant $k$ is calculated as follows:
$$k=\frac{\log C_1-\log C_2}{\Delta t}$$
Half-life ($t_{1/2}$) can be calculated from $k$:
$$t_{1/2}=\frac{\log 2}{k}$$
The main challenge is that $C_1$ and $C_2$ are subject to a random error, most importantly the imprecision of the chemical analysis used to determine drug concentrations. The coefficient of variation (CV) of the analysis is 9 %, such that each measured value of $C_1$ and $C_2$ can be assumed to be the true value $\pm$18 % (approximately).
In order to check whether there is a significant difference between $C_1$ and $C_2$, the critical difference $D$ was calculated using the method described by C. G. Fraser:
$$D = \sqrt{2}\cdot Z\cdot CV \approx 0.25$$
Z is the Z-score for a bidirectional difference to be significant with p = .05, approximately 1.96. Expressed in words, we assume that there is a true difference between $C_1$ and $C_2$ if their absolute difference is greater than 25 %. In more than 90 % of all cases, there was no significant difference. The mean difference was only 4.4 %.
The most obvious implication of this is that the values of $k$ and $t_{1/2}$ calculated from $C_1$ and $C_2$ in individual cases cannot be trusted to be anywhere near the true values. The distributions of calculated $k$ and $t_{1/2}$ are severely distorted or 'smeared out' compared to the 'true distributions' that we would have found if there were no random error.
The distribution of calculated ("measured") values look like this (with the x-axes cropped to show the most relevant areas; there were many extreme outliers well beyond what is shown):

$\bar{X}_k \approx 0.120$ and $s_k \approx 0.360$.
$\bar{X}_{t_{1/2}}$ is in fact infinity and $s_{t_{1/2}}$ impossible to calculate due to the presence of infinite values.
We can make some, although very limited, assumptions about the a priori distributions of $k$ and $t_{1/2}$. They must have non-zero, positive values. They are very likely to have marked positive skewness, and especially $t_{1/2}$ is expected to have a substantial right tail. They may be lognormally distributed, but we cannot be sure of this. Based on previous litterature about the drug, $t_{1/2}$ can have values anywhere between 0 and 70, maybe even higher.
Given the analytical CV of 9 % and $\Delta t$ of 0.5 hours, we can simulate e.g. a million calculated values of $k$ if there were no true difference between $C_1$ and $C_2$ and all the observed difference was due to random error. Using R:
<code>(log(rnorm(1000000, 1, .09)) - log(rnorm(1000000, 1, .09))) / .5</code>
The SD of the simulated values, $s_{analytical}$, is 0.257.
Question 1: Assuming that biological variability and analytical imprecision are only two sources of variability of calculated $k$, is it justifiable to calculate the 'biological SD' from the 'total SD' (0.36) and the simulated 'analytical SD' (0.257)?
$$s_{biological} = \sqrt{s_{total}^2-s_{analytical}^2} \approx 0.252$$
Question 2: Is there an easy way to calculate $s_{analytical}$ directly, without using simulation?
Question 3: Making the crude assumption that $\mu_k = \bar{X}_k = 0.120$ and $\sigma_k = s_{biological} = 0.252$, and that $k$ is lognormally distributed, we get the following distribution of 'true' values of $k$:

Converting this into $t_{1/2}$ ($\log 2 / k$), we get:

Now, if I simulate a large number of $k$ and $t_{1/2}$, I find that $\mu_{t_{1/2}}\approx 31$ hours. On the other hand, the $t_{1/2}$ corresponding to $\mu_k$ is $\log 2 / 0.12 \approx 5.8$ hours.
What is the mathematical explanation why $\mu_{t_{1/2}}$ is apparently unrelated to $\mu_k$ even though $t_{1/2}$ and $k$ are two sides of the same coin? Despite the fact that $\bar{X}_k$ seems to be a useful approximation of the central tendency of $k$, it is unclear to me what the $t_{1/2}$ calculated from it actually tell us.
Question 4: If all we have is $\bar{X}_k$ and $s_k$ and the only assumption we make about the true values of $k$ or $t_{1/2}$ is that they are positive numbers, can we know anything about the central tendency and spread of $t_{1/2}$ at all?
Comment: What you propose is close to a Bayesian approach: start by stating broad, physically realistic assumptions about what the parameter values might be, then use the data to refine your estimates. [Statistical Rethinking](https://www.routledge.com/Statistical-Rethinking-A-Bayesian-Course-with-Examples-in-R-and-STAN/McElreath/p/book/9780367139919) is a superb introduction. In your case, the many negative "measured" $k$ values suggest that your sampling times aren't well chosen, as @AdamO suggests--perhaps they're not during an exponential-decay phase, or sampling times are too close together.
Comment: Is the model assumption that $k$ is fixed, with observed variability due to sampling errors on $C_{1,2}$, or that $k$ itself varies in the population? I wasn't sure if the "Hypothetical distribution of k in the population" was exactly that latter thing, or more an a priori expectation. If $k$ varies from person to person, and there's only one noisy measurement for each person that doesn't peg $k$ with much precision, I worry the problem may be underdetermined?
Comment: $k$ varies in the population, and that variation is precisely what I am trying to extract from the noise caused by measurment imprecision. The reason for the large number of negative $k$s is that, based on a priori knowledge of the drug, the true difference between $C_1$ and $C_2$, can be less than 1 % in some cases, which will of course "drown" in a measurment CV of 9 %. So the individual measurments of $k$ are very misleading. The "hypothetical" distributions are plausible but not necessarily correct distributions of true $k$ values that will give rise to results similar to what we have.
Answer: I can't follow this approach 100%, but it's a little surprising because the approach seems like nothing I've done in pharma. When we do pharmacokinetic analysis, we sample blood concentrations of drug at several timepoints, at least 5 times. These data are input into a statistical software package called winnonlin which is basically a fancy non-linear least squares that fits a standard dose concentration curve to the data. When data has to be pooled across many subjects or many administration time-points, we test for something called dose proportionality by fitting the predicted curves and adding random subject/visit level effects. If the test for the random effect is non-significant, we say that the doses are proportional.
The point is that the $k$ parameter seems to be too random. A dose concentration curve has to start at 0 mechanically, quickly ramp up, then taper off logistically. If the time point 1 is planned for the Cmax (the highest concentration) time point, it will be slightly under and slightly over if the dosing is non-proportional. I wonder if this is the issue. It might explain why such a large number of $k$ are close to 0, and why the half lives are non-sensiscal.
Comment: I agree that this approach seems very unusual. Some clarification is needed. The pharmacokinetics of the drug in question has already been well characterised in previous litterature. The blood samples used in this study were acquired by the police from car drivers suspected of impaired driving. The samples were never intended for pharmacokinetic studies. Nevertheless, we wanted to see if anything sensible could be said about the elimination rate at the time of blood sampling, in this specific population, by looking at only two samples taken a very short time apart.
Comment: I'm curious about the result of a regression (or fit incorporating $C_1$'s sampling uncertainty) $\log C_2 \sim \log C_1$; it it would be good to see that the data is consistent with the assumption that $C_2$ is a fixed proportion of $C_1$ (i.e. that the slope in this regression is consistent with 1; the intercept would correspond to the decayed proportion, assumed to be constant since $\Delta t$ is fixed). Since each concentration is non-negative, assuming log-normal sampling uncertainty seems reasonable.
Answer: For question 2, there can be no exact answer with a log of normally distributed variables, since some of the variates will be negative.
However the ends of the distribution are managed, we can quickly approximate the result using the delta method with $f(X)=\ln(X)/0.5$, $\mu=1$, $\sigma=.09$:
\begin{align}
Var\Big[f(N(\mu,\sigma))\Big] &\simeq f'(\mu)^2\, \sigma^2\\
Var\Big[\ln(C_i)/0.5\Big]&\simeq 4(.09)^2\\
Var\!\left[\frac{\ln(C_1)-\ln(C_2)}{0.5}\right]&\simeq 8(.09)^2\\
SD\!\left[\frac{\ln(C_1)-\ln(C_2)}{0.5}\right]&\simeq 2\sqrt{2}(.09)=.255
\end{align}
Comment: This is very helpful, thank you so much!


Which type of trumpet is typically used to perform Bach Brandenburg 2

Question: Could anyone explain how to choose what design of trumpet to perform Brandenburg 2 on?  I'd expect it to be performed on a Bb piccolo trumpet - but there are three-valve and four-valve piccolos, and there are piston valve and rotary valve piccolos. And presumably Bach wrote for natural trumpet - as if the music wasn't hard enough already...
So what are the advantages/disadvantages of one physical design over another?
Note - I'm specifically not looking for brand recommendations - rather I'd like advice on which design to choose and why.
And is a piccolo trumpet mandatory - could you perform it on some other high-pitched trumpet instead?
Answer: The part as written is for a natural trumpet (clarino) in F. You could theoretically play it on any high trumpet.
There is some evidence that the part might have been intended to sound an octave lower then it is normally played. (See this discussion):

Thurston Dart postulated the part was intended for "Tromba, o vero corno da caccia" but this would put the part an octave lower than written. I believe there was a player, possibly even Reiche, who could have played it or had the required skill. Remember the portrait of Reiche holding a "Jaegertrompete?" It's impossible to say if such an instrument was common or not despite the famous painting as the only surviving such instrument was destroyed during WWII by allied bombing.
A feasible answer to the question "trumpet or horn?" might be that on occasion Bach allowed for a performance of the work incorporating a horn instead of a trumpet. The German name "Jaegertrompete" (hunter's trumpet) for a trumpet which was coiled, and looked like a horn, may have been mis-translated into Italian as "Corno da caccia" (hunter's horn), and thus the debate began.

If you do decide to play it on the B♭ piccolo, you don't need the fourth valve. And as for the choice between piston and rotary valves: that is entirely your personal choice. The part is hard enough, you should play it on the instrument you feel most comfortable on.
Comment: Thank you for the answer. I was hoping for some solid reasons for preferring either piston or rotary valves.
Comment: @BrianTHOMAS You won't hear solid reasons for either choice. It's all a matter of personal preference. There are plenty of excellent recordings on both instruments.
Answer: Yes, it's usually played today on a B♭ piccolo trumpet.  As there's obviously no issue with authenticity, choice between the various designs is completely at the player's discretion.  The arrangement of the valves makes minimal if any difference to the sound.
Comment: Why do you say "no issue with authenticity" ?  piccolo trumpets do not sound the same as a standard trumpet in high octaves, for example.
Comment: @CarlWitthoft B♭ piccolo trumpet is not authentic, so it doesn't matter which type of B♭ piccolo you use.


Tracking elements of another array in foreach loop

Question: I have two arrays, $files with file names and $imagesFormats with formats of this images. I want to move every file from $files to directory given in $imageFormats. Indexes of files and it's formats are same in both arrays. What's the best way to do this - creating another associative array ($image => $format) or there is another solution?
Comment: what kind of output do you want? try `array_combine($files, $imagesFormats)` and `array_map(NULL, $files, $imagesFormats)` and look if they are ok.
Comment: I don't want the output, I want to move very image to it's corresponding folder (all formats have their folders).
Answer: If the two array have the same key  you can do in the same foreach  .. eg:
  foreach( $files as $key => $value ) {

        yourfunction($value); ///
        yourfunction($imagesFormats [$key] ) ;
  }


Place Image from DLL onto a web page

Question: I'm so stuck on something i thought would be easy.
I have a DLL that returns an Image object.
I just cant figure out how to display that image on a webpage.
I've tried a few ways, and google a million different variations.
Is it not possible to just bind an Image object to an element on the page like an HtmlImage or a simple img?
Or do i need to convert the Image to a Stream? or a Bitmap? I'm really stuck!
Any help appreciated.....
V
Comment: What are you using (WebForms, MVC...)?
Comment: yes, you have to produce a stream containing the image bytes. Is the image in a resources file in the DLL ?
Comment: @Steve B: no, the DLL literally returns an image. i can a method, pass it a few vars, and i get a chart back.
Comment: @Boris B: i'm using WebForms. :)
Answer: With Asp.Net WebForm, the easiest way is to create a custom ashx file.
In Visual Studio, create a new Custom Handler (I'm not sure of the name of the template in Visual Studio). This will create a .ashx file.
In the code of this handler, write something like (does not have VS under the hand to test the syntax) :
public void ProcessRequest(System.Web.HttpContext context)
{

   byte[] raw;
   using(var ms = new MemoryStream()){
       Image myImage = GetFromDll();
       myImage.Save(ms, ImageFormat.Png);
       raw=ms.ToArray();
   }
   context.Response.ContentType = "image/png";
   context.Response.BinaryWrite(raw);
}

Then, in your browser, navigate to http://yourserver/app/yourhandler.ashx.
You can if you want add url parameter, and get it from the Request.QueryString collection
Answer: It's not as simple as binding. On the client side images are retrieved from the web server as a separate GET request, which means you have to have a URL that resolves to an image. The other option, as Asif suggested, is embedding your image in the HTML as a Base64 string, which is bad practice for shared images (see Steve B's comment).
You either have to provide an URL (route that returns the image file in MVC, or a custom page with proper content type and Response.Write in WebForms), or embed in html.
EDIT:
There is also a third option involving custom HTTP handlers. These have the advantage of bypassing the app framework and serving the content almost directly off the web server, see MSDN.
Answer: Convert your image to base64 string and then set it in the  tag.
 can show the image in base64 string.
Alternatively you can save the image and use the path in the .
Comment: Very bad idea regarding performances. You'll add a great amount of data in the page, downloaded each time the page is download. If page weight is important for you, you should better create a custom handler that will serve the image binaries with the correct http headers for letting the browser cache the file.
Comment: +1 for pointing out the performance issue, @SteveB can you please explain the technique of creating custom handler in some detail, its interesting ...
Comment: @Asif http://stackoverflow.com/questions/1096122/how-do-i-use-an-ashx-handler-with-an-aspimage-object

Place Image from DLL onto a web page


Can anyone help me solve these sequence limits?

Question: I would be really thankful if someone helped me with these two sequence limits.
$$\lim_{n\to\infty} \frac{1}{n^2} \left(2+ \frac{3^2}{2}+\cdots+\frac{(n+1)^n}{n^{n-1}}\right)$$ I've tried bounding the second term, but I don't know how to solve it. I think it's zero but I'm not sure about separating the limits..
 $$\lim_{n\to\infty} \frac{\sqrt{(n-1)!}}{(1+\sqrt{1}) (1+\sqrt{2})\ldots(1+\sqrt{n})} $$
I've tried using the Stirling formula with this limit, but I'm not sure about how to solve what I get….
Thank you!
Comment: You should post each of these limits in a separate question. (With regard to the second: you should state explicitly "what [you] get" so that people can help you interpret it and won't waste time duplicating your effort.)
Comment: See also" [Evaluate the limit of the sequence: $\lim_{n_\to\infty}\frac{\sqrt{(n-1)!}}{(1+\sqrt{1})\cdot(1+\sqrt{2})\cdot (1+\sqrt{3})\cdots (1+\sqrt{n})}$](https://math.stackexchange.com/q/3036510)
Answer: By the Stolz-Cesro Theorem , one has
\begin{eqnarray}
&&\lim_{n\to\infty} \frac{1}{n^2} \left(2+ \frac{3^2}{2}+\cdots+\frac{(n+1)^n}{n^{n-1}}\right)\\
&=&\lim_{n\to\infty} \frac{\sum_{k=1}^n\frac{(k+1)^k}{k^{k-1}}}{n^2}\\
&=&\lim_{n\to\infty} \frac{\sum_{k=1}^{n+1}\frac{(k+1)^k}{k^{k-1}}-\sum_{k=1}^n\frac{(k+1)^k}{k^{k-1}}}{(n+1)^{2}-n^2}\\
&=&\lim_{n\to\infty} \frac{\frac{(n+2)^{n+1}}{(n+1)^{n}}}{2n+1}\\
&=&\lim_{n\to\infty} \frac{n+2}{2n+1}\frac{(n+2)^{n}}{(n+1)^{n}}\\
&=&\lim_{n\to\infty} \frac{n+2}{2n+1}\left(1+\frac{1}{n+1}\right)^n\\
&=&\frac e2.
\end{eqnarray}
Answer: For the first $$\frac {(n+1)^n}{n^{n-1}}=(n+1)\left(1+\frac 1n\right)^{n-1}=\frac{n+1}{1+\frac 1n}\left(1+\frac 1n\right)^{n}\to (n+1)e$$
For the second, if you erase all the $+1$s you decrease the denominator and increase the fraction.  But it makes the fraction
$$\frac{\sqrt{(n-1)!}}{\sqrt{n!}}=\frac 1{\sqrt n} \to 0$$
Comment: Sorry but I don't know how to use that equivalence... Should I replace each term of the sum with it and then calculate the limit? And with the second one... How can I match them if they are different factorials? I'm not sure about it… Thank you
Comment: For the first, all the big terms will work like this and the small ones won't matter.  It takes some justification but you can replace each term this way.  Now you have to sum the series, which will wind up with a factor $n^2$ to cancel the division out front.  For the second, when you separate the numerator there aren't any factorials any more.  The first term is $\sqrt 1,$ the second $\sqrt 2$, and so on.  Match them in order with the terms in the denominator.
Comment: For the first, it depends on how quickly the limit is approached.


Letters above staff?

Question: I see letters above the staff like DbMa7 (#11). Could you explain? (note: I basically just know that music has notes and you can play em at the same time aka i'm dumb about music)
Comment: I wouldn't be surprised if this were a duplicate, but if it isn't it deserves a good answer.
Answer: It is the chord that's being played at that point, by the person playing the song or in the backing track/other band members (in the occasion of playing in a band)
Comment: So like if it says CMaj above, you play C Major? Do you need to write the notes on the staff?
Comment: @DaveRainbowin Depends on the context, however unless it's a jazz lead sheet in which only chord symbols are given, in most cases you only have to follow the score and you'll automatically be playing the chord notate above, likely with the optimal voicing within the context of the song provided the score is well-made, or if you're playing in a band or just jamming with a bunch of friends, the chord could be split into parts and other people might play part of the chord, so if your score says CMaj at the top, you don't have to play C E G, instead you may, for instance just play E.
Comment: I don't really understand, but good enough for me.
Comment: It's hard to explain concretely without actually knowing what your score looks like.

Letters above staff?


Unnecessary and confusing bass clef in Mozart's Fantasia in D minor, Why?

Question: I just played Mozart's Fantasia in D minor and I noticed at the Allegretto section in D major, an unnecessary and confusing bass clef. So I looked on IMSLP to see if any of the editions of Mozart's Fantasia in D minor have this unnecessary bass clef. Turns out that at least 3 of the editions on IMSLP have it, and they are all editions from the 1930s and later. Editions from the 1800s, I see no unnecessary bass clef, but I do see at the Alberti bass section, a large bass clef followed immediately by a small, courtesy, treble clef. Now that isn't as confusing as the unnecessary bass clef I see in editions from the 1930s and later. The editions that I looked at and that have the unnecessary bass clef are:

Unknown editor, no date, but looks similar in style to scores by the Mutopia Project
Mozart Album: Zongorára, 1951
Neue Mozart-Ausgabe, 1930-1995

All of which you can see here:
https://imslp.org/wiki/Fantasia_in_D_minor%2C_K.397%2F385g_(Mozart%2C_Wolfgang_Amadeus)
Here is an example of what I mean by an unnecessary and confusing bass clef taken from the Mozart Album edition:

The bass clef at the very end of the first volta is completely unnecessary, any decent pianist is going to know that the repeat starts in the bass clef and changes to the treble clef later on. But that isn't what makes this bass clef confusing. No, what makes it confusing is that it makes it look as though the Alberti bass starts in the bass clef on an F# minor harmony when in fact it starts in the treble clef on a D major harmony. If the editor just didn't put in this unnecessary bass clef, it would be very clear that the Alberti bass starts in the treble clef on a D major harmony.
So, why am I seeing this unnecessary and confusing bass clef almost universally in editions of Mozart's Fantasia in D minor from the 1930s and later? I have seen a lot of courtesy clefs and clef changes, but until I looked at this piece, I have never seen a case where a clef makes things more confusing.
Comment: Yeah they should have put those 2 measures into tenor clef (snark).   Srsly: notice the inconsistency that the treble indicator in the upper line is after the measure bar but subsequent indicators are before the measure bar.  Bad Typesetter!
Answer: That bass clef is necessary -- to indicate that the clef changes from treble to bass at the start of the second volta. Your argument that "any decent pianist is going to know" isn't valid, I'm afraid -- the change back to bass clef must be indicated.
And that bass clef doesn't affect the Alberti bass because the bass clef was in the first-time section, and the Alberti bass is in the second-time section.
Comment: If that bass clef is truly necessary, the editions from the 1800s, such as   Wolfgang Amadeus Mozarts Werke would have it. They don't, so that bass clef at the first volta is unnecessary.
Comment: @Jomiddnz I said that in the second para of my answer.
Comment: It is usual when changing clefs to add the new clef sign at the end of the bar before the change where they the two bars are not continuous in the score. This would be the case even where the new clef is on the next line of the score. The example seems typical and standard practice.  .
Comment: But the second volta is right where the Alberti bass starts and if you listen to any recording of this piece, that is pretty clearly in the treble clef on a D major harmony, not in the bass clef on an F# minor harmony. And if you are talking about at the repeat before the second volta is played, the bass clef is unnecessary, because the large bass clef at the start of that repeat and the change to bass clef at the previous repeat both make it clear that at the repeat, it changes back to bass clef. Why else would I not be seeing this bass clef at the first volta in editions from the 1800s?
Comment: @ Rosie F - That small bass clef in the 1st time bar has nothing to do with the 2nd time bar (which is clearly still in the treble clef).
Answer: The logical way to get rid of the bass clef is to get rid of the treble clef two bars before, but some editors think pianists can't read leger lines.
Then, a treble clef can go after the repeat barline, where it belongs.
That is was done in the first NMA edition (1878). However the editorial policy of the second NMA edition is more strictly "urtext", which means the original notation is reproduced even when by modern conventions it is illogical or even "wrong."
Your statement that the bass clef is not in the second NMA edition (on IMSLP) is incorrect. It is included as an editorial edition in brackets.
Editions with titles like "A Mozart Album" are usually worthless from a musicological point of view. So are anonymous contributions to crowd-sourced editions like Mutopia (which had grandiose ideas of taking over the entire world of music publishing, but never delivered them - it has only published about 2000 scores in its 20 year history).
Note also that computer notation software is often not very good at things like placing cautionary clefs after bar lines instead of before, and many users either don't know there is anything wrong, or don't know how to make the notation right. The logic that "it must be right because the computer did it that way" is nonsense, but depressingly common.
Comment: According to the preface in my Henle edition, the autograph is lost. It was first published posthumously in 1804 as a fragment (the last 10 bars are missing) and in 1806 in a completed form. Henle also has the "spurious" bass clef, by the way.
Comment: I never stated that it isn't in the NMA edition. What I did state is that it isn't in the earlier editions from the 1800s.
Comment: @Caters is there a manuscript? Has anyone checked it? Guest: 'The logic that "it must be right because the computer did it that way" is...depressingly common': this is even more true in other areas of endeavor, more's the pity.  W.r.t. crowdsourcing, I'm singing a piece by Christopher Tye in an edition from the Choral Public Domain Library.  It's truly awful: it seems like the editor ran out of steam, or time, in the last few pages. There are some good ones there too, of course, but it definitely requires some searching to find them.  The ease of editing computer files has made us careless.
Answer:
...unnecessary and confusing bass clef

Maybe it is unnecessary because the bass clef is given at the beginning of the line, but even if it is unnecessary it is simply a courtesy.
It's only confusing if upon repeating and playing the second volta the first volta is read when it shouldn't be.
If you imagine what the score would look like for just the second repeat you get...

...the first volta is not there, there is clearly a clef change and volta 2's bass is treble clef.
Mistakenly reading the clef in volta 1 when playing volta 2 should not be consider a fault in the edition.
Comment: I replied to your comment on phoog's answer asking where the G clef would go if bars 15 and 16 are in the bass clef, but by the time I finished my reply, I saw that you deleted that comment that I was replying to.
Comment: Right, I deleted because I answered by own question: it can go _after_ the barline. Which I agree is ideal. But that doesn't really matter about reading the particular edition you selected.
Answer: It's correct usage of a cautionary clef.   And yes, this is a case where breaking the rule and omitting it might well be clearer.  (Though it might not be so clear if the repeat back had NOT been to the beginning of a line!)
Or perhaps we'd prefer this?
Answer:
Why am I seeing this unnecessary and confusing bass clef?

It's obviously because the editor thought it helpful or necessary to remind the reader that the first measure of that repeated section is in bass clef.  As a general rule it makes sense; consider the case where the first measure of the repeated section is not at the beginning of the staff.  In that case it would be easier to miss the clef change.
It is indeed confusing that the clef change does not apply to the second ending.  A solution would be to keep measures 15 and 16 in bass clef, as are 11 and 12, but 11 and 12 are already somewhat crowded, and 15 and 16 would be even more so.
Another solution would be to write the entire passage in tenor clef, or even alto clef, but that would be far more confusing to most modern pianists.
Comment: @NeilMeyer how is "undoing many years of sight-reading training" not confusing?  A lot of pianists approaching this piece for the first time are unlikely even to know what a C clef _is,_ and they will also be confused.
Comment: @MichaelCurtis It would go on bar 17, where the Alberti bass and thus the second volta starts. I see exactly that in some editions from the 1800s including the oldest edition I can find on IMSLP, the Wolfgang Amadeus Mozarts Werke edition.
Comment: C clefs are not confusing. For a pianist it just undo many years of sight-reading training.


Properly counting notes from sheet music

Question: I'm beginner piano player. I'm trying to play easy song from notes like below:

In whole song I'm counting notes: 1 2 3 4. Because I can see that the shortest note is 1/4. But there is an exception in first measure, last line. There is 1/8 note. When we have 1/8 notes then we are counting: 1 and 2 and 3 and 4 and. The question is: Should I count in such way for all song because I can see at least one 1/8 note?
Should I count 1 and 2 and 3 and only in measure with 1/8 notes? How to do it properly?
I try to play this song with metronome.
Answer: Because this song has simple rhythms and there are only two eighth notes in the entire song there is no need to count the "and" for the entire song but there is nothing wrong with counting all the "ands" if it makes you more comfortable and helps you play the song correctly. Your idea of just counting the "and" for bar 13 is a good one. The rhythm in that bar will then be:
ONE and TWO and three and FOUR AND
The numbers and "ands" in capital letters are where the notes fall.
Answer: In this song, counting 1 2 3 4 in each bar will work just fine. You will reach a stage when you won't even need to count that, you'll just feel it.
On the bar in question, most would count 1 2 3 4&, obviously the 1 2 3 4 will remain at the same pulse as they did in all the other bars. Don't see the need for all the & counts, but if it helps, there's no damage done.
EDIT: when you use the metronome, you would benefit from making it click on all eight counts - 1&2&3&4&, or at very least 1 2 3 4. That way, you play exactly with the clicks.
Answer: There is no right or wrong way. If counting 1 and etc helps then do it. You're going to collect a big tool box of stuff like this and you just take out the tools as you need them. Some of those tools you grow out of. Some of them you carry everywhere you go.
Answer: You make the shortest note value one and then go up. Quaver would be one count, crotchets two, minims four, semi-breves eight. Always make the shortest note value equal to one and then make the divisions make sense upwards. Start slowly and then start counting faster as you get the hang of it.
Playing a presto piece, andante, but in time, is less of a fault, than playing a piece's tempo correctly, but getting the rhythms wrong.
This is just to understand how to play the rhythm, it is not an explanation out of a theory point-of-view
Answer: Only use the "And" count in the measure where you need it, it'll be tiering if you do otherwise.
You'll be counting 1-2-3-4 most of the time, then use 1-2-3-4-an or 1-2-an-3-4 whenever needed.
In time, rythms like this will become natural, so you don't need to do this anymore


Criei um gridView. Agora como faço para chamar um intent a partir dos icones do gridview?

Question: Segue o código do gridview. É nessa mesma activity que quero chamar as ações do grid.
<code>//Grid VIew...set grid view item
Bitmap homeIcon = BitmapFactory.decodeResource(this.getResources(), R.drawable.teste_cor);
Bitmap userIcon1 = BitmapFactory.decodeResource(this.getResources(), R.drawable.teste_fest);
Bitmap userIcon2 = BitmapFactory.decodeResource(this.getResources(), R.drawable.teste_paris);
Bitmap userIcon3 = BitmapFactory.decodeResource(this.getResources(), R.drawable.teste_news);
Bitmap userIcon4 = BitmapFactory.decodeResource(this.getResources(), R.drawable.teste_media);
Bitmap userIcon5 = BitmapFactory.decodeResource(this.getResources(), R.drawable.local_gar);

gridArray.add(new Item(homeIcon,"menu1"));
gridArray.add(new Item(userIcon1,"menu2"));
gridArray.add(new Item(userIcon2,"menu3"));
gridArray.add(new Item(userIcon3,"menu4"));
gridArray.add(new Item(userIcon4,"menu5"));
gridArray.add(new Item(userIcon5,"menu6"));

gridView = (GridView) findViewById(R.id.gridView);

customGridAdapter = new CustomGridViewAdapter(this, R.layout.row_grid, gridArray);
gridView.setAdapter(customGridAdapter);
</code>
Answer: geralmente eu faço dessa forma:
<code>gridView.setOnItemClickListener(new GridView.OnItemClickListener(){
            @Override
            public void onItemClick(AdapterView parent, View view, int position, long id) {
                // chamar sua intent

            }

        });
</code>


Sync emails over imap with a local database

Question: For an email history in a php erp system i would like to sync emails with imap with a local mysql database.
It's faster than i thought to download all messages with attachments (about 5 minutes for 6000 messages), but after downloading all messages it would be great to only download the changes. The problem is that i can't rely on the date of the email because the flaggs (seen, flagged, answered, deleted) can change over time.
Is there a possibility to get only the changed emails, what is the best practice for local email clients like thunderbird?
As far as I understand the protocol, there is a server "push" method, but the php script should not be connected all the time, just as a local email client.
This answer was quite useful, so I have to download the headers (without the body) and compare all messages in every request?
Edit
I know that such tools like imapsync already exists, but I would like to do it without external dependencies.
Comment: Messages are immutable *Except* for FLAGS.  You can assume any UID does not change, only resync the FLAGS when you connect.  Download a list of UIDs and FLAGs when you start (if you care about the flags).  Compare to the UIDs you have.  Tada.
Comment: Each email has a unique ID. Use that as as index to test for existence.
Comment: Yes, in the first run i save all headers with the body (without attachments) in the database, also the uid. But how can i check BEFORE downloading all messages again which email changed and sync the changes?
Comment: Have you looked at the `SEEN` header. You can set this flag for each message as you receive each message
Comment: But the seen header is only a boolean and I don't know if the email has changed later, for example someone replied to this mail.
Answer: If the purpose is to download all mails from imap server and then periodically check if there is any new email, which was not previously downloaded and saved to db, then I would recommend using PEAR Net_IMAP class.
PHP native IMAP has not the capability to set/get custom flags but with Net_IMAP class you can do as follows.

Connect to IMAP server
Read message.
Save to DB.
Using Net_IMAP method setFlags, set a custom flag on that message on server. for example
<code>setFlags('savedToDb')
</code>

Thats it.
Next iterations
connect to IMAP and check all those messages which do not have the flag that you have set.
This way you will be able to handle all emails successfully synched between your IMAP and db.
Comment: is setting costom flags widely supported by most mail servers?
Comment: But if another flag changes, the problem still persists. I would like to be able to sync the flags, too.
Comment: As I am not aware of the full context of the application, but I assume that what you are trying to achieve is to have two identical mail apps. The solution provided above was based on the assumption that you are doing it one way. You fetch emails from server, you keep the server copy for sometime and then delete it to reclaim space. However same solution can be used to check all flags as well, and you can see if your custom flag exist, then only get flags, and update your local db but if it doesn't then get full email.
Comment: Sry I thought i mentioned it somewhere. Nevertheless this is a great solution, the flags are not that important.


Access violation with Drupal 7 multisite using shared DB

Question: I'm building a Drupal 7 multisite and I want to have a separate database for each site but with some shared tables. So, I have two sites called siteone and sitetwo. I also have 3 databases - oxone (for first site), oxtwo (for second one) and oxshared (where I want to keep shared tables for both sites.
I used the following config for settings.php (for siteone), but it shows me a PDO exception... What I'm doing wrong? Will be very appreciate for any help or advice.
$databases = array (
  'default' =>
  array (
    'default' =>
    array (
      'database' => 'oxone',
      'username' => 'oxone',
      'password' => '*********',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => array(
         'default'   => 'oxone.',
         'users'     => 'oxshared.',
         'users_roles'     => 'oxshared.',
         'sessions'  => 'oxshared.',
         'role'      => 'oxshared.',
         'role_permission'      => 'oxshared.',
         'authmap'   => 'oxshared.',
     ),
    ),
  ),
  'oxshared' =>
  array (
    'default' =>
    array (
      'database' => 'oxshared',
      'username' => 'oxshared',
      'password' => '**********',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => '',
    ),
  ),
);

The exception is:

PDOException: SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user 'oxone'@'localhost' for table 'users': SELECT u., s. FROM {users} u INNER JOIN {sessions} s ON u.uid = s.uid WHERE s.sid = :sid; Array ( [:sid] => _5UERN3eEWligNxBusEYCq3Lnf48pfvWDu9FDQDZUI8 ) in _drupal_session_read() (line 102 of /home/qsoul/public_html/siteone.dev.lo/includes/session.inc)

Why doesn't Drupal use the oxshared user when accessing the tables in the oxshared database, as defined in the prefix array? It uses the oxone user instead.
Comment: What's the exception?
Comment: This one: `PDOException: SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user 'oxon'@'localhost' for table 'users': SELECT u.*, s.* FROM {users} u INNER JOIN {sessions} s ON u.uid = s.uid WHERE s.sid = :sid; Array ( [:sid] => _5UERN3eEWligNxBusEYCq3Lnf48pfvWDu9FDQDZUI8 ) in _drupal_session_read() (line 102 of /home/qsoul/public_html/siteone.dev.lo/includes/session.inc).`
Comment: Pretty self-explanatory error I'd say...see https://stackoverflow.com/questions/4767055/error-select-command-denied-to-user-useridip-address-for-table-table. You may also have a typo (`oxon` v `oxone`)
Comment: But why drupal doesn't use `oxshared` user to get those shared data, but tryes to do it via `oxeone`? p.s. 'oxon' typo is only here (just made a type while replacing real data), in setings.php its 'oxone'.
Comment: No worries, glad you got it sorted :) I've edited the info from the comments into the question, it'll help people with the same problem in the future to find your answer
Answer: The question is closed. It seams to be that to have some tables in a separate database:
1) that second db should be placed at the same host as default one.
2) the user who has access to the default db should also has access to the second db.
Adding one more db to settings.php is not needed. It my case the correct settings.php db config has to look like:
$databases = array (
  'default' =>
  array (
    'default' =>
    array (
      'database' => 'oxone',
      'username' => 'oxone',
      'password' => '*********',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => array(
         'default'   => 'oxone.',
         'users'     => 'oxshared.',
         'users_roles'     => 'oxshared.',
         'sessions'  => 'oxshared.',
         'role'      => 'oxshared.',
         'role_permission'      => 'oxshared.',
         'authmap'   => 'oxshared.',
     ),
    ),
  ),
);


Como realizar consulta com alguns valores nulos?

Question: O SQL abaixo retorna os valores corretamente somente se o campo data_emissao estiver como não nulo na minha tabela do BD (independente se o parâmetro cliente foi passado ou não), porém desejo retornar os valores onde os campos data_emissao também estiverem nulos (caso o parâmetro cliente tenha sido passado).
Como devo utilizar o operador lógico OR sem deixar de obedecer a condição caso o campo data_emissao seja diferente de nulo?
$sql  = " SELECT * FROM nfe WHERE (cliente LIKE :cliente OR :cliente_ IS NULL)";

$sql. = " AND (data_emissao IS NOT NULL AND data_emissao <= :dt_final
OR :dt_final_ IS NULL)";

Simulação do problema:
------------------------------
-- Tabela `nfe`
------------------------------

1 | Aarco A | NULL
2 | Barco B | 13-01-2017
3 | Carco C | 14-01-2017
4 | Darco D | 14-01-2017
5 | Earco E | 15-01-2017
6 | Farco F | NULL
7 | Garco G | 16-01-2017

SQL:
SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NOT NULL AND data_emissao <= '15-01-2017')

Resultado obtido:
NULL | NULL | NULL

Resultado esperado:
1 | Aarco A | NULL

Eis aqui como a tabela foi criada:
-- ----------------------------
-- Table structure for nfe
-- ----------------------------
DROP TABLE IF EXISTS `nfe`;
CREATE TABLE `nfe` (
  `id` int(9) NOT NULL AUTO_INCREMENT,
  `cliente` varchar(255) DEFAULT NULL,
  `data_emissao` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;

-- ----------------------------
-- Records of nfe_teste
-- ----------------------------
INSERT INTO `nfe` VALUES ('1', 'Aarco A', null);
INSERT INTO `nfe` VALUES ('2', 'Barco B', '13-01-2017');
INSERT INTO `nfe` VALUES ('3', 'Carco C', '14-01-2017');
INSERT INTO `nfe` VALUES ('4', 'Darco D', '14-01-2017');
INSERT INTO `nfe` VALUES ('5', 'Earco E', '15-01-2017');
INSERT INTO `nfe` VALUES ('6', 'Farco F', null);
INSERT INTO `nfe` VALUES ('7', 'Garco G', '16-01-2017');

SQL Fiddle do BD: sqlfiddle.com/#!9/790cb5/1
Comment: SELECT * FROM nfe_teste WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NULL OR data_emissao <= '15-01-2017')
Comment: @MurilloGoulart, ainda está retornando `NULL`.
Comment: Tente usar o operador OR
data_emissao IS NOT NULL OR data_emissao <= '15-01-2017')
Comment: Olá @olavooneto a consulta retornou todas as linhas da tabela.
Answer:
O SQL abaixo retorna os valores corretamente somente se o campo data_emissao estiver como não nulo na minha tabela do BD [independente se o parâmetro cliente foi passado ou não], porém desejo retornar os valores onde os campos data_emissao também estiverem nulos [caso o parâmetro cliente tenha sido passado].

Ou seja:

Se o campo data_emissao estiver não nulo OU onde o campo data_emissao estiver nulo E o cliente tenha sido passado.

E portanto:
data_emissao IS NOT NULL OR (data_emissao IS NULL AND cliente IS NOT NULL)

Isso pode ser simplificado como:
data_emissao IS NOT NULL OR cliente IS NOT NULL

Uma prova disso é se olharmos os casos que você tem:

data_emissao é nulo e cliente é nulo. - Não deve vir no resultado.
data_emissao é nulo e cliente não é nulo. - Deve vir no resultado.
data_emissao não é nulo e cliente é nulo. - Deve vir no resulado.
data_emissao não é nulo e cliente não é nulo. - Deve vir no resulado.

O único caso que não interessa é quando os dois são nulos.
Entretanto, há mais duas restrições também:

As datas não nulas que importam são aquelas que são menores que a data informada (que pode ser nula).
Os clientes não-nulos que importam são aqueles que estão de acordo com o LIKE, se isto tiver sido informado.

A regra número 1 (que trás as datas de emissão não nulas que importam) fica assim:
data_emissao IS NOT NULL AND (data_emissao <= :data_final OR :data_final_ IS NULL)

Já a regra número 2 (que trás os clientes não nulos que importam) fica assim:
cliente IS NOT NULL AND (cliente LIKE :cliente OR :cliente_ IS NULL)

E portanto sua SQL fica assim:
SELECT * FROM nfe
WHERE (data_emissao IS NOT NULL AND (data_emissao <= :data_final OR :data_final_ IS NULL))
OR (cliente IS NOT NULL AND (cliente LIKE :cliente OR :cliente_ IS NULL))

Comment: Olá @VictorStafusa, excelente explicação. As duas regras estão corretíssimas, porém um detalhe que provavelmente não deixei claro é que, ao unir as duas, a que deverá prevalecer é a _segunda_, ou seja, ao realizar a consulta por `cliente` e um valor **não nulo** para `data_emissao`, os resultados deverão ser menores que a data estipulada e com base na palavra inserida (mesmo que na tabela do BD a `data_emissao` esteja nula).
Answer: Basta usar UNION:

UNION is used to combine the result from multiple SELECT statements
  into a single result set.
  Retirado do Manual (link)

Esse comando combina dois resultsets em um, mas, não realiza Distinct, ou seja, se existirem linhas repetidas, elas vão aparecer.
Alterei seu conjunto de dados para incluir dois registros para o mesmo cliente, so que um contem a data e o outro não.
-- ----------------------------
-- Table structure for nfe
-- ----------------------------
DROP TABLE IF EXISTS `nfe`;
CREATE TABLE `nfe` (
  `id` int(9) NOT NULL AUTO_INCREMENT,
  `cliente` varchar(255) DEFAULT NULL,
  `data_emissao` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;

-- ----------------------------
-- Records of nfe_teste
-- ----------------------------
INSERT INTO `nfe` VALUES ('1', 'Aarco A', null);
INSERT INTO `nfe` VALUES ('2', 'Barco B', '13-01-2017');
INSERT INTO `nfe` VALUES ('3', 'Carco C', '14-01-2017');
INSERT INTO `nfe` VALUES ('4', 'Darco D', '14-01-2017');
INSERT INTO `nfe` VALUES ('5', 'Earco E', '15-01-2017');
INSERT INTO `nfe` VALUES ('6', 'Farco F', null);
INSERT INTO `nfe` VALUES ('7', 'Garco G', '16-01-2017');
INSERT INTO `nfe` VALUES ('8', 'Aarco A', '13-01-2017');

A query final:
SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NOT NULL AND data_emissao <= '15-01-2017')
UNION
SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NULL)

O resultado:
8 | Aarco A | 13-01-2017
1 | Aarco A | NULL

Para adequar ainda mais, você pode testar o parametro do nome do cliente antes de montar a instrução SQL para execução e executar apenas o SQL indicado. Algo como:
if(cliente != null)
{
   .sql = "SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NOT NULL AND data_emissao <= '15-01-2017')
UNION
SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NULL)";
}
else
{
   .sql = "SELECT * FROM nfe WHERE (data_emissao IS NOT NULL AND data_emissao <= '15-01-2017')
UNION
SELECT * FROM nfe WHERE (data_emissao IS NULL)";
}

Edit: só vi depois que você pede para usar OR, então, desculpe ai. Se quiser eu retiro.
Answer: Alterei o nome das colunas para testar no meu ambiente, a query abaixo retornou o que você precisa:
SELECT * FROM #usuario WHERE (nome LIKE '%Aarco%')AND (data IS NULL or data <= '01-15-2017')

Comment: Olavo, novamente não funcionou. SQL Fiddle do meu BD: http://sqlfiddle.com/#!9/790cb5/1
Comment: SELECT * FROM nfe WHERE (cliente LIKE '%Aarco%')
AND (data_emissao IS NULL or data_emissao <= '15-01-2017')
Comment: Desta forma funcionou, testei no Sqlfiddle
Comment: Olá @olavooneto, pode publicar seu SQL Fiddle?
Comment: Segue link http://sqlfiddle.com/#!9/790cb5/2/0
Comment: Obrigado pela atenção e ajuda Olavo. Sim, sua resposta resolve parte do meu problema, porém ainda não atende a situação inicial ["O SQL ... retorna os valores corretamente somente se o campo `data_emissao` estiver como não nulo na minha tabela do BD "], ou seja, preciso que a consulta também retorne os dados **não nulos** com base no intervalo de data especificado. @olavooneto
Comment: Você quer as linhas tanto do intervalo de data quanto do cliente procurado?
Se entendi corretamente então vc precisa trocar por "OR" apenas:
SELECT * FROM nfe WHERE cliente LIKE '%Aarco%' OR (data_emissao <= '15-01-2017' OR data_emissao IS NULL)
Mas no resultado esperado da sua pergunta não é isso que da a entender.
Comment: @olavooneto podemos continuar essa discussão no chat? http://chat.stackexchange.com/rooms/54701/como-realizar-consulta-com-base-em-valor-null
Answer: A forma mais direta e óbvia que consegui pensar pro seu problema foi usando um IF. Se data_emissao não for nulo e se data_emissao for menor que a data fornecida ou se data_emissao for nulo.
SELECT
  *
FROM
  nfe
WHERE
  cliente
  LIKE '%Aarco%'
  AND IF(data_emissao<>NULL, (data_emissao<='15-01-2017'), TRUE)

No caso, como você quer com um OR, podemos usar
SELECT
  *
FROM
  nfe
WHERE
  cliente
  LIKE '%Aarco%'
  AND ((data_emissao IS NOT NULL AND data_emissao<='15-01-2017') OR data_emissao IS NULL)


Import text file from website to powerbi using panda dataframe

Question: TEXT OUTPUT EXAMPLEI need to import csv text files I import using python request. My code can import the data from a website with my own API key. But to get it imported to POwerBI it has to be converted to a dataframe. But I cant workout how to do this. Any help would be much appreciated.
response = requests.get('https://my url websaite', headers=headers)
wrapper = csv.reader(response.text.strip().split('\n'))
for record in wrapper:
    A = str(record[0])
    B = str(record[1])
    C = str(record[2])
    D = str(record[3])
    E = str(record[4])
    F = str(record[5])
    G = str(record[6])
    H = str(record[7])
    I = str(record[8])
    J = str(record[9])
    K = str(record[10])
    L = str(record[11])
    M = str(record[12])
    N = str(record[13])
    O = str(record[14])
    P = str(record[15])
    Q = str(record[16])
    R = str(record[17])
    S = str(record[18])
    T = str(record[19])
    U = str(record[20])
    V = str(record[21])
    W = str(record[22])

    print (A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W)

Comment: Can you give a printout of response at list first couple of lines and we can analyze from there. Some observations, csv reader expects to read from a file not a string object and it needs a delimiter defined (usually a comma).
Answer: So I understand the input data has the following structure:
['A', 'B', 'C', 'D']
['DEV', 'J', 'BBBB', '2019-08-28']
['5R', 'J', 'EMAIL', '2019-05-21']

and you want a script to convert this data in a pandas data frame to be uploaded with Power Bi.
As a first step you have to make this text data csv-readable by converting it to text that just contain lines with the values separated by comma's like:
A, B, C, D
DEV, J, BBBB, 2019-08-28
5R, J, EMAIL, 2019-05-21

Then as csv.reader reads an IO object (similar to a file) rather than just a text string you need to use io.StringIO to change the text to an IO object. Once all (non empty) rows are read, it is straightforward to create the data frame, whereby it is assumed that the first row is the header.
script:
import io
import csv
import pandas as pd

raw_input_data = '''
['A', 'B', 'C', 'D']
['DEV', 'J', 'BBBB', '2019-08-28']
['5R', 'J', 'EMAIL', '2019-05-21']'''

# condition the input data to make it a readable csv
raw_input_data = raw_input_data.replace('[', '').replace(']', '').replace('\'', '')
raw_input_file = io.StringIO(raw_input_data)
csv_reader = csv.reader(raw_input_file, delimiter=',')

# append each non empty row to the list input_rows and convert to a
# pandas dataframe, assume first row is the header
input_rows = []
for row in csv_reader:
    if row:
        input_rows.append(row)

input_df = pd.DataFrame(input_rows[1:], columns=input_rows[0])
print(input_df.head(10))
input_df

Note I believe you have to add the input_df at the end to have it recognized by Power Bi
In your case I take it that
response = requests.get('https://my url websaite', headers=headers)

has the same structure as raw_input_data.
Comment: Thanks Bruno. The output file printed is grouped together by [ ] , plus I also have some commas in text , which is being shown as , 'Y, Weekly' ,  the rest of data is separated like 'YR' , how do I get this into a dataframe so I can import to PowerBI
Comment: is there also a way to exclude the header row , as I don't need to import that data
Comment: my main issue is that to get this imported to PowerBI , it says it has to be in a DataFrame. I cant find a way to import the string into a Dataframe. There are Columns A to W
Comment: Can you update the question and give me a few rows of response so I can see the structure of the data ...
Comment: I have added a link at the top TEXT OUTPUT EXAMPLE
Comment: By the way I am a bit surprised the ulr does not give a json object as then things would be very easy just do input_data = json.loads(requests.get(url).text) and input_df = pd.DataFrame(input_data[1:], columns=input_data[0])
Comment: Thanks the code works - raw_input_file = io.StringIO(response.text)
csv_reader = csv.reader(raw_input_file, delimiter=',')
Comment: To add a column to the dataframe just do input_df['X'] = [elements], where elements must be a list of n values where n is the size (number of rows) of the data frame. I am afraid I cannot help you with r and really this should asked in another question.
Comment: As you say you can now read the response, I would appreciate if you can click 'Answered' to my answer. Thanks.


Erro ao efetuar soma Javascript

Question: Tenho um script de cálculo que adaptei de outra página.
Lá, tudo certo, mas nessa, mesmo conferindo os mesmos ids e afins, não funciona.
Eu peguei o essencial da tela, para demonstrar o que fiz até o momento.

$(".real").maskMoney({
  prefix: 'R$ ',
  thousands: '.',
  decimal: ',',
  affixesStay: true
});

function rePlace(i) {
  return parseFloat(i.replace(/[^\d.,]/g, "").replace('.', '').replace(',', '.'));
}

Number.prototype.formataReal = function(c, d, t) {
  var n = this,
    c = isNaN(c = Math.abs(c)) ? 2 : c,
    d = d == undefined ? "," : d,
    t = t == undefined ? "." : t,
    s = n < 0 ? "-" : "",
    i = parseInt(n = Math.abs(+n || 0).toFixed(c)) + "",
    j = (j = i.length) > 3 ? j % 3 : 0;
  return s + (j ? i.substr(0, j) + t : "") + i.substr(j).replace(/(\d{3})(?=\d)/g, "$1" + t) + (c ? d + Math.abs(n - i).toFixed(c).slice(2) : "");
};

//any time the amount changes
$(document).ready(function() {
  $('.qtde_orc, .valor_orc').change(function(e) {
    var valor_total_orc = 0;
    var $row = $(this).parent();
    var valor_orc = rePlace($row.find('.valor_orc').val());
    var qtde_orc = $row.find('.qtde_orc').val();
    valor_total_orc = parseFloat(valor_orc * qtde_orc);
    //update the row total
    $row.find('.valor_total_orc').val('R$ ' + (valor_total_orc).formataReal(2, ',', '.'));

    var total_geral = 0;
    $('.valor_total_orc').each(function() {
      //Get the value
      var tg = rePlace($(this).val());
      //if it's a number add it to the total
      if (IsNumeric(tg)) {
        total_geral += parseFloat(tg);
      }
    });
    $('.total_geral').val('R$ ' + (total_geral).formataReal(2, ',', '.'));
  });
});

function IsNumeric(input) {
  return !isNaN(parseFloat(input)) && isFinite(input);
}
.pedido_desktop,
.nome_desktop {
  display: block;
}

.pedido_smartphone,
.nome_smartphone {
  display: none;
}

@media (min-width: 992px) and (max-width: 1199px) {}

@media (min-width: 769px) and (max-width: 991px) {}

@media (min-width: 481px) and (max-width: 768px) {
  .pedido_desktop,
  .nome_desktop {
    display: none;
  }
  .pedido_smartphone,
  .nome_smartphone {
    display: block;
  }
}

@media only screen and (max-width: 480px) {
  .pedido_desktop,
  .nome_desktop {
    display: none;
  }
  .pedido_smartphone,
  .nome_smartphone {
    display: block;
  }
}

        <input name="id_item_up[7]" id="id_item_up[7]" class="checkbox-custom" type="checkbox" value="7" style="width:50%;" tabindex="18" checked>

Black Bass

Black Bass

        <input type="text" name="quantidade[7]" id="quantidade[7]" class="form-control input_qtde qtde_orc" placeholder="Quant." />

          <select name="tamanho[7]" id="tamanho[7]" class="s_pedido">
              <option value="" selected="selected" disabled="disabled">Tamanho</option>
              <option value="Alevino">Alevino</option>
              <option value="Juvenil">Juvenil</option>
              <option value="Adulto">Adulto</option>
             </select>

        <input type="text" name="preco[7]" id="preco[7]" class="form-control input_qtde real valor_orc" placeholder="Preço" />

        <input type="text" name="total[7]" id="total[7]" class="form-control input_qtde  valor_total_orc" placeholder="Total" />

        <input name="id_item_up[8]" id="id_item_up[8]" class="checkbox-custom" type="checkbox" value="8" style="width:50%;" tabindex="18" checked>

Cachara

Cachara

        <input type="text" name="quantidade[8]" id="quantidade[8]" class="form-control input_qtde qtde_orc" placeholder="Quant." />

          <select name="tamanho[8]" id="tamanho[8]" class="s_pedido">
              <option value="" selected="selected" disabled="disabled">Tamanho</option>
              <option value="Alevino">Alevino</option>
              <option value="Juvenil">Juvenil</option>
              <option value="Adulto">Adulto</option>
             </select>

        <input type="text" name="preco[8]" id="preco[8]" class="form-control input_qtde real valor_orc" placeholder="Preço" />

        <input type="text" name="total[8]" id="total[8]" class="form-control input_qtde  valor_total_orc" placeholder="Total" />

Total

        <input type="text" name="total_geral" id="total_geral" class="form-control input_qtde" style="background-color: #c0c0c0;" placeholder="Total" />

Answer: A sua variavel $row está retornando a
 errada e por isso você não consegue encontrar os elementos dentro dela com o .find().
Mude de var $row = $(this).parent()
para var $row = $(this).parent().parent();
Outra coisa no final você está buscando o elemento de classe total_geral mas não existe nenhum elemento com essa classe e sim com este id.
Mude de $('.total_geral').val('R$ ' + (total_geral).formataReal(2, ',', '.'));
Para $('#total_geral').val('R$ ' + (total_geral).formataReal(2, ',', '.'));

$(".real").maskMoney({
  prefix: 'R$ ',
  thousands: '.',
  decimal: ',',
  affixesStay: true
});

function rePlace(i) {
  return parseFloat(i.replace(/[^\d.,]/g, "").replace('.', '').replace(',', '.'));
}

Number.prototype.formataReal = function(c, d, t) {
  var n = this,
    c = isNaN(c = Math.abs(c)) ? 2 : c,
    d = d == undefined ? "," : d,
    t = t == undefined ? "." : t,
    s = n < 0 ? "-" : "",
    i = parseInt(n = Math.abs(+n || 0).toFixed(c)) + "",
    j = (j = i.length) > 3 ? j % 3 : 0;
  return s + (j ? i.substr(0, j) + t : "") + i.substr(j).replace(/(\d{3})(?=\d)/g, "$1" + t) + (c ? d + Math.abs(n - i).toFixed(c).slice(2) : "");
};

//any time the amount changes
$(document).ready(function() {
  $('.qtde_orc, .valor_orc').change(function(e) {
    var valor_total_orc = 0;
    var $row = $(this).parent().parent();
    var valor_orc = rePlace($row.find('.valor_orc').val());
    var qtde_orc = $row.find('.qtde_orc').val();
    valor_total_orc = parseFloat(valor_orc * qtde_orc);
    //update the row total
    $row.find('.valor_total_orc').val('R$ ' + (valor_total_orc).formataReal(2, ',', '.'));

    var total_geral = 0;
    $('.valor_total_orc').each(function() {
      //Get the value
      var tg = rePlace($(this).val());
      //if it's a number add it to the total
      if (IsNumeric(tg)) {
        total_geral += parseFloat(tg);
      }
    });

    $('#total_geral').val('R$ ' + (total_geral).formataReal(2, ',', '.'));
  });
});

function IsNumeric(input) {
  return !isNaN(parseFloat(input)) && isFinite(input);
}
.pedido_desktop,
.nome_desktop {
  display: block;
}

.pedido_smartphone,
.nome_smartphone {
  display: none;
}

@media (min-width: 992px) and (max-width: 1199px) {}

@media (min-width: 769px) and (max-width: 991px) {}

@media (min-width: 481px) and (max-width: 768px) {
  .pedido_desktop,
  .nome_desktop {
    display: none;
  }
  .pedido_smartphone,
  .nome_smartphone {
    display: block;
  }
}

@media only screen and (max-width: 480px) {
  .pedido_desktop,
  .nome_desktop {
    display: none;
  }
  .pedido_smartphone,
  .nome_smartphone {
    display: block;
  }
}

        <input name="id_item_up[7]" id="id_item_up[7]" class="checkbox-custom" type="checkbox" value="7" style="width:50%;" tabindex="18" checked>

Black Bass

Black Bass

        <input type="text" name="quantidade[7]" id="quantidade[7]" class="form-control input_qtde qtde_orc" placeholder="Quant." />

          <select name="tamanho[7]" id="tamanho[7]" class="s_pedido">
              <option value="" selected="selected" disabled="disabled">Tamanho</option>
              <option value="Alevino">Alevino</option>
              <option value="Juvenil">Juvenil</option>
              <option value="Adulto">Adulto</option>
             </select>

        <input type="text" name="preco[7]" id="preco[7]" class="form-control input_qtde real valor_orc" placeholder="Preço" />

        <input type="text" name="total[7]" id="total[7]" class="form-control input_qtde  valor_total_orc" placeholder="Total" />

        <input name="id_item_up[8]" id="id_item_up[8]" class="checkbox-custom" type="checkbox" value="8" style="width:50%;" tabindex="18" checked>

Cachara

Cachara

        <input type="text" name="quantidade[8]" id="quantidade[8]" class="form-control input_qtde qtde_orc" placeholder="Quant." />

          <select name="tamanho[8]" id="tamanho[8]" class="s_pedido">
              <option value="" selected="selected" disabled="disabled">Tamanho</option>
              <option value="Alevino">Alevino</option>
              <option value="Juvenil">Juvenil</option>
              <option value="Adulto">Adulto</option>
             </select>

        <input type="text" name="preco[8]" id="preco[8]" class="form-control input_qtde real valor_orc" placeholder="Preço" />

        <input type="text" name="total[8]" id="total[8]" class="form-control input_qtde  valor_total_orc" placeholder="Total" />

Total

        <input type="text" name="total_geral" id="total_geral" class="form-control input_qtde" style="background-color: #c0c0c0;" placeholder="Total" />

Erro ao efetuar soma Javascript


Como resolver problemas na chamada de arquivos no HTML?

Question: Estou com uma estrutura de pastas onde tenho a seguinte arquitetura
page
    php
        arquivos.php
    tpl
        arquivos.tpl(html)
    style
        arquivos.css

Mas tenho um problema em chamar os arquivos nas tpl's...
quando uso o caminho
<link type="text/css" rel="stylesheet" href="../style/style.css" />

Ele volta para fora da raiz do projeto
estou tentando usar URL's amigaveis o .htaccess está assim
<IfModule mod_rewrite.c>
    RewriteEngine On
    Options -Indexes
    ErrorDocument 404 /Errors/404.html
    ErrorDocument 403 /Errors/403.html
    RewriteRule ^/?$ page/php/Home.php [NC,L]
    RewriteRule ^contato/?$ page/php/Contato.php [NC,L]
</IfModule>

Answer: Um dos passos é acrescentar estas RewriteCond para fazer o rewrite apenas quando não existir arquivo ou pasta com o nome solicitado:
RewriteEngine On
RewriteCond %{SCRIPT_FILENAME} !-f
RewriteCond %{SCRIPT_FILENAME} !-d
RewriteRule ^/?$ page/php/Home.php [NC,L]
RewriteRule ^contato/?$ page/php/contato.php [NC,L]

Adicionalmente, nos casos em que o CSS for geral, usar caminhos relativos à raiz:
<link type="text/css" rel="stylesheet" href="/style/style.css" />

Para CSS relativo ao documento corrente, só olhando o caso específico mesmo. Lembre-se que o que vale é o caminho final de onde está sendo servida a página, e não onde o script/template se encontra de fato. Quem vai resolver este caminho é o browser, e não o apache, então leve em consideração o caminho que aparece ao navegar nas páginas.
Veja um pouco mais sobre rewrite nesta questão.
Answer: Resolvi esse problema criando uma variável que guarda o Diretório do projeto, faço isso comparando o caminho do servidor com o da url e e gravando as pastas semelhantes, após isso apenas coloco a variável como prefixo de todas as URL's do site.
$UrlSite = explode(DIRECTORY_SEPARATOR,trim(parse_url($_SERVER['REQUEST_URI'],PHP_URL_PATH), '/\\'));
$DirSite = explode(DIRECTORY_SEPARATOR,__DIR__);
if(in_array($UrlSite[0], $DirSite) AND $UrlSite[0] != ''){
    $Directory = "/$UrlSite[0]/";
    $smarty->assign('Directory',$Directory);
}else{
    $Directory = "/";
    $smarty->assign('Directory',$Directory);
}


Prove that $\exists a, b\in(0,1)$ such that $\int_0^{a} xf(x)dx=0\text{ and }\int_0^bxf(x)dx=\frac{b^2f(b)}{2}.$

Question:
Question: Let $f:[0,1]\to\mathbb{R}$ be a continuous function such that $f(0)=0$ and $$\int_0^1f(x)dx=0.$$ Prove that $\exists a, b\in(0,1)$ such that
$$\int_0^{a} xf(x)dx=0\text{ and }\int_0^bxf(x)dx=\frac{b^2f(b)}{2}.$$

My approach: Let $g:[0,1]\to\mathbb{R}$ be such that $$g(x)=x\int_0^xf(t)dt-\int_0^xtf(t)dt, \forall x\in[0,1].$$
By the first fundamental theorem of calculus we can conclude that $g$ is differentiable on $[0,1]$ and $$g'(x)=\int_0^xf(t)dt, \forall x\in[0,1].$$
Also, observe that $g(0)=0$ and $g(1)=-\int_0^1tf(t)dt$. Thus, by applying MVT to the function $g$ on the interval $[0,1]$, we can conclude that, $\exists c\in(0,1)$ such that $$g'(c)=\int_0^cf(t)dt=-\int_0^1tf(t)dt.$$
Observe that clearly three cases are possible, i.e, either$$\int_0^cf(t)dt<0\text{ or }\int_0^cf(t)dt=0\text{ or }\int_0^cf(t)dt>0.$$
Now let $h:[0,1]\to\mathbb{R}$ be such that $$h(x)=\int_0^xtf(t)dt, \forall x\in[0,1].$$
Please note that the part highlighted below is wrong, but still I have included it to just demonstrate my thinking process, as it might be of some help to others trying out this problem.

Observe that if $\int_0^cf(t)dt<0$, then, $h(1)>0$. This also implies that $\exists$ an open interval $(d,e)\in[0,c]$, such that $f(t)<0, \forall t\in(d,e)$. Now select any point $c_1\in(d,e)$. Applying MVT to the function $h$ on the interval $[0,c_1]$, we can conclude that $\exists c_2\in(0,c_1)$ such that $$h'(c_2).c_1=f(c_2).c_2.c_1=h(c_1)-h(0)=h(c_1)<0.$$
Now $h(c_1)<0$ and $h(1)>0$. Thus, by IVT we can conclude that $\exists a\in(c_1,1)\subseteq(0,1)$, such that $$h(a)=\int_0^af(t)dt=0.$$
A similar reasoning for the case when $\int_0^cf(t)dt>0,$ shows that $\exists a\in(0,1)$, such that $$h(a)=\int_0^af(t)dt=0.$$
Now finally if $\int_0^cf(t)dt=0$, then we will have $h(1)=0$. Now if $f$ is identically equal to $0$ on $[0,c]$, then clearly $tf(t)=0, \forall t\in[0,c]\implies h(x)=0, \forall x\in[0,c].$ Thus choosing any point $x\in(0,c]$ and setting it as $a$, we will have $h(a)=0$ and we will be done in that case.
Now if $f$ acquires both positive and negative values on $[0,c]$, then we can conclude that $\exists c_1,c_2\in(0,c)$, such that $f(c_1)>0$ and $f(c_2)<0$. Also, let us assume WLOG that $c_2>c_1$.

I have not been able to make any significant approach other than this. Can someone help me out with this problem? Please note that a solution using integration by parts might not be possible, since $f$ is not a differentiable function.
Comment: The integration by parts in that solution is done by *integrating* $f$ and differentiating $x$: $\int xf = xF - \int F$. So that is not a problem.
Comment: The first part is answered here: https://math.stackexchange.com/questions/34808/olympiad-calculus-problem.
Comment: @MartinR, but the solution given there has used integration by parts. And, I do not think that integration by parts is applicable here, since, $f$ is not a differentiable function.
Answer: The first part is answered in Olympiad calculus problem. We can build on Christian Blatter's answer to solve the second part.
Define $F(x) = \int_0^x f(t) \, dt$ and $\phi(x) = \frac 1x \int_0^x F(t)\, dt$, $\phi(0) = 0$. In Christian's answer it is demonstrated that
$$
 \phi'(x) = \frac{1}{x^2} \int_0^x tf(t) \, dt
$$
has a zero $a \in (0, 1)$, so that $\int_0^a tf(t) \, dt = 0$.
In our case we additionally have $f(0) = 0$, which implies that $\phi$ is differentiable at $x=0$ with $\phi'(0) = 0$. Applying the mean-value theorem to $\phi'$ gives that
$$
 \phi''(x) = -\frac{2}{x^3} \int_0^x tf(t) \, dt + \frac{f(x)}{x}
$$
has a zero $b \in (0, a)$, so that $\int_0^b tf(t) \, dt = \frac{b^2f(b)}{2}$.


Android Fragment onClick button open new Fragment with same activity

Question: I have an Activity which launches a Fragment called main page. This page has cards with buttons and each button opens a new fragment. The problem is that when it opens a new fragment it seems to start a new Activity, which is not the main one, launched at first. For this reason the toolbar I have on the main Page doesn't appear on other pages which are opened on the click from the Main fragment.
Does anybody knows how to make it not change the first Activity or at least how to access it from other fragments. When I try getActivity() or getContext() from these Fragments it is never the same Activity loaded initially.
Btw, the first Activity extends AppCompactActivity
AppActivity activity = (AppActivity) v.getContext();
activity.getSupportFragmentManager().beginTransaction().replace(R.id.drawer_layout, myFragment).addToBackStack("string").commit();

(Appears in the main fragment on click)
Comment: please post your ccode
Comment: Unfortunately I cant post the whole code from other reasons but I can show what I have in the main fragment :
Comment: yeah show that your button click and your xml
Comment: Updated the question
Comment: show your mainactivity xml
Comment: Sorry Im not allowed to, but I think the question is clear
Comment: sorry !!! then cant help you out :)
Comment: thank you, but don't you have any hint ? the question why does it happen when you start a new fragment from another fragment ?
Comment: The problem is that when it opens a new fragment it seems to start a new Activity. Which activity its starting?? is it AppActivity?
Comment: Exactly!! So how can I prevent it form happenning?
Comment: Oh I think you pointed to something, So the main Activity is lunched from another Activity, lets say the Applications Activity is lunched from and Start Activity, but the fragments they start new Application activities...
Answer: Use below code i hope this will solve your problem...
FragmentTransaction fragmentTransaction = fragmentManager.beginTransaction();
      FragmentEntityManagementPage  fragmentEntitymanagementPage = new FragmentEntityManagementPage();
        fragmentTransaction.add(R.id.rl_container1, fragmentEntitymanagementPage);
        fragmentTransaction.addToBackStack("FragmentEntityManagementPage");
        fragmentTransaction.commit();

Comment: Thanks, What exactly is this fragmentEntitymanagementPage? Where can I find it in my code ?
Comment: It is your target fragment name or second fragment name where you want to share data after button click.
Comment: Thanks, So I did add this to my code :FragmentTransaction fragmentTransaction = activity.getSupportFragmentManager().beginTransaction();
                        fragmentTransaction.add(R.id.drawer_layout, myFragment);
                        fragmentTransaction.addToBackStack("FragmentEntityManagementPage");
                        fragmentTransaction.commit();   But the problem it is still connected to the Activity I create before but it its not the same Activity as I have on the Start...
Comment: Cause I have before this :                         AppActivity activity = (AppActivity) v.getContext();
The question how do I get the first Activity ?
Comment: The Problem I am not allowed to share it, but I can say that the AppACtivity has getApplicationContext() - MyApp context but the Fragment has AppACtivity context...
Comment: https://stackoverflow.com/questions/20464273/get-the-application-context-in-fragment-in-android
Comment: Yeah I try to use getActivity() in the fragment but it is not the same as I do getActivity() in the AppActivity...


Python for each row, while column A value is the same, add to list. When column A value changes, separate list

Question: I'm trying to assess a CSV/Excel file that is organized like this:
172 Afghanistan AFG 1961    0
173 Afghanistan AFG 1962    0
174 Afghanistan AFG 1963    0
175 Afghanistan AFG 1964    0
176 Afghanistan AFG 1965    0
177 Afghanistan AFG 1966    0
178 Afghanistan AFG 1967    0
179 Afghanistan AFG 1968    0
180 Afghanistan AFG 1969    0
181 Afghanistan AFG 1970    0
182 Afghanistan AFG 1971    0
183 Afghanistan AFG 1972    0
184 Afghanistan AFG 1973    1
185 Afghanistan AFG 1974    0
186 Afghanistan AFG 1975    0
187 Afghanistan AFG 1976    3
188 Afghanistan AFG 1977    0
189 Afghanistan AFG 1978    0
190 Afghanistan AFG 1979    0
191 Afghanistan AFG 1980    0
192 Afghanistan AFG 1981    0
193 Afghanistan AFG 1982    0
194 Afghanistan AFG 1983    0
195 Afghanistan AFG 1984    0
196 Afghanistan AFG 1985    0
197 Afghanistan AFG 1986    0
198 Afghanistan AFG 1987    0
199 Afghanistan AFG 1988    0
200 Afghanistan AFG 1989    0
279 Albania     ALB 1961    0
280 Albania     ALB 1962    0
281 Albania     ALB 1963    0
282 Albania     ALB 1964    2

Essentially, I'm trying to put the 2nd column (country names) in one list with no repeats (country_names = ['Afghanistan', 'Albania',…]), and another list of lists of the last column into a list like this:
list_of_regime_lists=[[0,0,0,0,0,0,0,…],[0,0,0,0,0,0,0,…], etc.]
Is there a way to write a line of code WITHOUT USING NUMPY OR PANDAS along the lines of "while column remains same value, append line[4] to list?"
Here's what I have so far:
def read_file(fp):

reader = csv.reader(fp)
next(reader, None)

country_names = []
regime_list = []
list_of_regime_lists = []

for line in reader:
    country = line[1]
    regime = int(line[4])

    if country != "":
        if country not in country_names:
            if regime_list != []:
                list_of_regime_lists.append(regime_list)
            regime_list = []
            country_names.append(country)
            regime_list.append(regime)
            if country in country_names:
                regime_list.append(regime)

return country_names, list_of_regime_lists

Comment: Is this a homework problem? See https://stackoverflow.com/questions/71622725/python-how-to-separate-different-values-in-same-column-in-csv-file
Comment: i saw that post earlier but with my assignment, I'm supposed to use lists and only lists, not dictionaries. when i did try that code you provided however, it would print each value correctly but add "\n" to each number in the list of lists.
Comment: im ultimately trying to write a command where, when the country name changes,  it will separate each list inside the master list
Comment: Your code is almost correct. Get rid of `if regime_list != []:` and move the following line *after* `regime_list = []`. Then remove one indent from `regime_list.append(regime)` and get rid of the following two lines. Basically, you just need to create a new `regime_list` every time the country changes and otherwise keep appending `regime` to the current list.
Answer: Thanks to @ekhumoro
def read_file(fp):

reader = csv.reader(fp)
next(reader, None)

country_names = []
regime_list = []
list_of_regime_lists = []

for line in reader:
    country = line[1]
    regime = int(line[4])

    if country != "":
        if country not in country_names:
            regime_list = []
            list_of_regime_lists.append(regime_list)
            country_names.append(country)
        regime_list.append(regime)
        if country in country_names:
            regime_list.append(regime)

return country_names, list_of_regime_lists

Comment: Get rid of `if country in country_names: regime_list.append(regime)`. It's not needed, and it will add the current `regime` *twice*, which you obviously don't want.


Is it a good/bad idea to use test features to strengthen learning parameters?

Question: I am fairly new to machine learning and have started implementing a few classification algorithms.
From whatever I have read so far, I understand that data is absolutely crucial for training the classifier model - more the data, better it is at generalizing.
Now a classifier can predict correctly or wrongly. So what I  wish to know is this:
Is it a good idea to use a new test feature set to modify the training parameters every time the test feature set gets correctly classified (with the intention being to strengthen the model)?
What could be the problems if I tried this approach? Would there be the presence of some sort of bias or preference in the classification?
Also, am I treading upon the domain of reinforcement learning here?
Any thoughts upon this matter would be most appreciated...
Comment: By "new test feature set" are you referring to adding a new independent variable coded for each correct prediction?
Comment: @ArunJose Not exactly. Let's assume that the classification problem is to identify writers from their handwriting samples using a training model that has been trained using different features like stroke length, curvature etc. and that the model used is a simple naive bayes probability model. Say a new handwriting sample is provided to be tested i.e. to classify it as belonging to a particular writer and my model classifies it correctly. So now I wish to add this test sample into my training set and reevaluate all the training parameters (probabilities in this case).
Comment: If I understood your question, I'd say it depends on the goals of the analysis. I recommend you to look up "Semi-supervised learning".
Comment: @Firebug Thanks for the insight :) But would it not apply to any supervised learning situation? And yeah I am fairly sure what I am trying to achieve here is not really Reinforcement Learning. It is more inclined to semi-supervised learning (which I shall most certainly be delving into) like you mentioned or something like Online Learning (sort of)
Answer: This is a topic of deep interest to me, but I have not achieved a strong enough grasp of statistics to explain its intuition mathematically. However, I can paraphrase a strategy for solving such problems.
While we discuss this topic, my only request would be to treat it as a guide for how one might choose to solve the problem, rather than figuring if your solution was the most comprehensive one possible. Data science is infinitely complex. It is futile to assume we would have a clue what the actual distribution is. Sample size governs all.
So if I'm faced with a business problem and the solution of which can gradually shift me towards a more optimal one then I have conceptually succeeded.
Now, long story short. In your case if your data gave you each day a new experiment that could help you answer one question - What is the best way I can predict what will happen tomorrow? This is in the most generic sense the objective of any data science.
So in effect, everything we ever will use statistics for is just this single truth. The moment a simple learning algorithm can somehow get better at just guessing on your behalf what happens tomorrow slightly better than you could with a calculator, you have shifted into an optimal mode of operation with autopilot.
(This also has all sorts of oversight and operational issues of its own, but bear with me here.)
If in each subsequent day we perform a gradual  percentage on error lower we are better off. The longer we keep this journey, the more straight forward it is to see that eventually the models will evolve and possibly reduce the errors on most targets.
Now imagine this ability on steroids: if rather than trying to figure out what will happen next day, we can make slightly easier guesses about what will happen in the next 10 seconds, or 1 second for instance. Suddenly you can think of a massive number of things that we can easily learn without being too strict about the methodology.
If at the end of this path, for assume a week or two one of two things happen. Either you can see a clear evidence of change for better or you would find that the results aren't all the good. If it isn't good something is off. OR the system is inherently unpredictable in some way i.e. the stock market. Not "learnable".. yet, entirely.
I have no clue however if this answer actually addresses your question, but I've personally been able to see some actual change and often it didn't require much more complicated models than logistic regression, cluster analysis, linear regression. And you have neural networks to throw at at the drop of a hat for any big data problem you can think of. Be it genetic research or space exploration. You obviously have way more exciting discoveries to make than you have time for knowing one more type of probability distribution, for most human beings at least.
My two cents.
Comment: Thank you so much for taking the time to share the learning you have gleaned from your own experience! It was a beautiful and sensible read :) And while it may not have been a direct answer to my question, there are some key takeaways for me here viz. (1) Give the solution a shot and observe how it performs (2) Occam's razor applies even to opaque fields like ML (3) Utility beats perfection as far as real world problems are concerned


Getting "is not a function " errors with RxJS

Question: Seemingly out of the blue I am getting errors with RxJS.
My project is Angular, Typescript and I use RxJS as part of NGRX redux.
My code seemed to work absolutely fine with just this as an import:
import { Observable } from 'rxjs/Observable'
Then, from nowhere I started getting errors like this...
Uncaught (in promise): TypeError: this.appStateStore.withLatestFrom is not a function
Uncaught (in promise): TypeError: this.appStateStore.take is not a function
I was able to solve the error about withLatestFrom by adding an import of import 'rxjs/add/operator/withLatestFrom'; but then the error moved to complaining about .take.
I'm guessing I have an import wrong somewhere as I understand with RxJS you need to import the bits that you need. However I've reviewed my source control changes and I can't see that any changes (e.g. node module version, import statements) that would cause this to just start happening.
What I am doing wrong?
Comment: Then add `import 'rxjs/add/operator/distinctUntilChanged'; import 'rxjs/add/operator/take';`
Answer: import 'rxjs/add/operator/distinctUntilChanged';
import 'rxjs/add/operator/take;

Rxjs v 5.4.1
1) import { Rx } from 'rxjs/Rx;
This imports the entire library. Then you don't need to worry about loading each operator . But you need to append Rx. I hope tree-shaking will optimize and pick only needed funcionts( need to verify ) As mentioned in comments , tree-shaking can not help. So this is not optimized way.
public cache = new Rx.BehaviorSubject('');

Or you can import individual operators .
This will Optimize your app to use only those files :
2) import { _______ } from 'rxjs/_________';
This syntax usually used for main Object like Rx itself or Observable etc.,
Keywords which can be imported with this syntax
 Observable, Observer, BehaviorSubject, Subject, ReplaySubject

3) import 'rxjs/add/observable/__________';
These are usually accompanied with Observable directly. For example
Observable.from()
Observable.of()

Other such keywords which can be imported using this syntax:
concat, defer, empty, forkJoin, from, fromPromise, if, interval, merge, of,
range, throw, timer, using, zip

4) import 'rxjs/add/operator/_________';
These usually come in the stream after the Observable is created. Like flatMap in this code snippet:
Observable.of([1,2,3,4])
          .flatMap(arr => Observable.from(arr));

Other such keywords using this syntax:
audit, buffer, catch, combineAll, combineLatest, concat, count, debounce, delay,
distinct, do, every, expand, filter, finally, find , first, groupBy,
ignoreElements, isEmpty, last, let, map, max, merge, mergeMap, min, pluck,
publish, race, reduce, repeat, scan, skip, startWith, switch, switchMap, take,
takeUntil, throttle, timeout, toArray, toPromise, withLatestFrom, zip

FlatMap:
flatMap is alias to mergeMap so we need to import mergeMap to use flatMap.

Note for /add imports :
We only need to import once in whole project. So its advised to do it at a single place. If they are included in multiple files, and one of them is deleted, the build will fail for wrong reasons.


How to add v1 when specifying the URL of an API?

Question: I'm creating an API service using Services (Drupal 7) and would like to create an URL like /api/avu/v1/{id}.
I have api in Path to endpoint and in hook_services_resources() I have $resources['avu'] with
<code>"args" => array(
   array(
     "name" => "id",
     "optional" => false,
     "source" => array("path" => 0),
     "type" => "int",
     "description" => "The id of the course to retrieve."
   )
),
</code>
So /api/avu/{id} works but I can't figure out how to add v1.
Answer: All I had to do was to add another argument and clear the cache.
<code>"args" => array(
   array(
     "name" => "version",
     "optional" => false,
     "source" => array("path" => 0),
     "type" => "string",
     "description" => "The API version."
   ),
   array(
     "name" => "id",
     "optional" => false,
     "source" => array("path" => 1),
     "type" => "int",
     "description" => "The id of the course to retrieve."
   )
),
</code>


Asp.net Identity com muitos request sem ações do cliente

Question: Comecei a utilizar o Identity no ASP.NET MVC e percebi que a página fica dando vários requests, mesmo sem o usuário executar nenhuma ação. Me parece que isso é para validar os cookies, estou certo? Se sim, tem como aumentar o tempo entre esses requests? Pelo que percebi, a cada 6 segundos ele faz um novo request.
Comment: Que tipo de *requests*? Em desenvolvimento, o ASP.NET MVC envia para a sua página um recurso chamado *Browser Link*, que faz o VS se comunicar com a página gerada. Seria disso que estamos falando?
Answer: No arquivo Startup.Auth.cs, adicione as opções para configura-las:
public void ConfigureAuth(IAppBuilder app)
    {
        // Enable the application to use a cookie to store information for the signed in user
        app.UseCookieAuthentication(new CookieAuthenticationOptions
        {
            AuthenticationType = DefaultAuthenticationTypes.ApplicationCookie,
            LoginPath = new PathString("/Account/Login"),
            ExpireTimeSpan = new System.TimeSpan(0,30,0),
            SlidingExpiration = true

        });
}

É só mudar o tempo da propriedade ExpireTimeSpan para um tempo que considere adequado.
Da documentação:

ExpireTimeSpan:
Controls how much time the cookie will remain valid from the point it is created. The expiration information is in the protected cookie ticket. Because of that an expired cookie will be ignored even if it is passed to the server after the browser should have purged it
SlidingExpiration:
The SlidingExpiration is set to true to instruct the middleware to re-issue a new cookie with a new expiration time any time it processes a request which is more than halfway through the expiration window.


GAE service with internal ingress control won't process task

Question: My question is very similar to this one.  My task creation code is the same as the answer there except that I add a payload.
I have a GAE app with a second service.  I would like this service to have internal ingress only for extra security and I also want this service to process a GAE task.
According to the answer of the linked question, this should work.
When the ingress setting is set to all, everything works fine.  But when the ingress setting is set to internal only, I get a PERMISSION DENIED error.  I've copied the task queue error message below.
Any idea why this doesn't work?
<code>{
insertId: "xxx"
jsonPayload: {
@type: "type.googleapis.com/google.cloud.tasks.logging.v1.TaskActivityLog"
attemptResponseLog: {
scheduleTime: "2023-12-06T00:06:45.550274Z"
targetType: "APP_ENGINE_HTTP"
maxAttempts: 0
responseCount: "0"
status: "PERMISSION_DENIED"
attemptDuration: "0.002211s"
targetAddress: "POST /xxx"
dispatchCount: "1"
}
task: "projects/xxx/locations/us-central1/queues/default/tasks/xxx"
}
logName: "projects/xxx/logs/cloudtasks.googleapis.com%2Ftask_operations_log"
receiveTimestamp: "2023-12-06T00:06:46.807046074Z"
resource: {
labels: {
target_type: "APP_ENGINE_HTTP"
location: "us-central1"
queue_id: "default"
project_id: "xxx"
}
type: "cloud_tasks_queue"
}
severity: "ERROR"
timestamp: "2023-12-06T00:06:45.563937621Z"
}```

</code>
Comment: Have you mentioned the correct stackoverflow link in your question? If yes, can you point out which part of the code you are referring to in the [Link](https://stackoverflow.com/questions/2530/how-do-you-disable-browser-autocomplete-on-web-form-field-input-tags)? And also have a look at this [Stackoverflow link](https://stackoverflow.com/a/65911327/18265702) which might help.
Comment: Hi @SandeepVokkareni, I pasted the wrong link and meant to paste the one in your comment.
Comment: It seems like you are facing an issue with permissions? Can you check whether you have enabled the required permissions as stated in this [Documentation](https://cloud.google.com/tasks/docs/reference-access-control#roles). You need to remember that each permission is applicable to a particular resource type.
Comment: Hi @minou, Have you managed to check the Cloud Tasks IAM permissions?
Comment: Also are you using cloud tasks with VPC connections?
Comment: @SandeepVokkareni, I haven't tried updating permissions yet but will do that.  It is a GAE task and not an HTTP target task.  I do have a VPC because I also have a reddis instance.
Answer: As I stated in the comments, It seems like you are facing an issue with permissions? Can you check whether you have enabled the required permissions as stated in this Documentation. You need to remember that each permission is applicable to a particular resource type.
Also as you confirmed that you have a VPC connection, It seems Cloud Tasks do not support VPC connections (Internal access to GAE) currently. This is because Cloud Tasks runs in a Google Managed project outside your VPC.
To use Cloud Tasks with VPC network, there is already a feature request filed for it as you can refer to the public issue tracker and you can mark <code>+1</code> or add your concern over there.
As a work around for now, you can try to create HTTP Target tasks with the load balancer endpoint instead of GAE endpoint in Cloud Tasks by referring to link. Please be aware that since this solution uses public internet via Load Balancing, it will incur extra charges of Load balancing.


Prove that the lines parallel to the angle bisectors through the midpoints of sides of a triangle are concurrent at the Spieker center

Question: I've been learning about triangle centers, and I have been able to prove that most of them exist. However, this one that I've come across has stumped a bit, and I haven't been able to find any proofs or hints towards how to prove this specific point always exists. I believe the point is called the Spieker center. The problem prompt goes a little like this.

Given a triangle $ABC$, construct lines parallel to the interior angle bisectors of angles $A$, $B$, and $C$, such that they pass through the midpoints of sides $BC$, $CA$, and $AB$ respectively. Prove that these lines are concurrent.

Any ideas on how I should approach this proof?
Comment: To confirm: This is indeed called the [Spieker center](https://mathworld.wolfram.com/SpiekerCenter.html).
Answer: Rotate the original triangle $ABC$ by $180^\circ$ about its centroid $G$ and scale down to half its size. It will coincide with the triangle formed by its midpoints $D,E,F$.
The angle bisectors of $ABC$ would go to angle bisectors of $DEF$. Since any line rotated by $180^\circ$, a half turn, remains parallel to itself, the angle bisectors of $DEF$ are the lines to be constructed in the question.
We know that angle bisectors of any triangle concur at a single point, namely its incenter. Hence the lines through midpoints and parallel to the angle bisectors of $ABC$ concur at the incenter of $DEF$. The incenter of $DEF$ is simply the image of incenter of $ABC$ under our transformation (a half turn about centroid followed by scaling by half). The proof is complete.
Comment: @Blue But your answer and the links therein were quite useful. Thankyou for writing those previous answers! :)
Comment: It's worth noting that angle-bisector-ness, *per se*, isn't required for this argument. The rotate-and-scale transformation guarantees that *any* trio of cevians through $A$, $B$, $C$ transfers to correspondingly-arranged parallel cevians through $D$, $E$, $F$; and, thus, the first trio's point of concurrence (if it exists, which it does in the case of the angle bisectors) transfers to the second's.
Comment: That is very nice! So we have two-sided correspondence : parallel lines through $A,D \iff $ corresponding cevians.
Comment: +1. I got so caught-up in the bisector-parallels generalization that I looked right past the obvious "the Spieker center is the incenter of the medial triangle" approach for this specific case. :) ... Well, perhaps OP will still find the Extended Ceva Theorem helpful in exploring more circle centers.
Answer: The concurrency is evident using the medial triangle property as stated in the answer by cosmo5. Here is wiki link that you can refer to for more on it (wiki).
Here is a more elementary proof simply using, i) the midpoint theorem, ii) alternate interior angles theorem which states that if two parallel lines are cut by a traversal, the alternate interior angles are congruent, and iii) Concurrency of angle bisectors theorem.

$D, E$ and $F$ are midpoints of the sides of $\triangle ABC$. Line through $AM$ is the angle bisector of $\angle A$. Line through $EN$ is parallel to $AM$ and through midpoint $E$ of side $BC$.
Using midpoint theorem, $ADEF$ is a parallelogram and $\angle DEF = \angle A$.
Using property of parallel lines, $\angle FEN = \angle FMA = \angle MAD = \frac{\angle A}{2}$. So line through $EN$ is angle bisector of $\angle DEF$ in $\triangle DEF$. By the same logic, parallel lines through midpoints to other two angle bisectors of $\triangle ABC$ must be angle bisectors of $\angle DFE$ and $\angle EDF$.
Hence the proof is complete by concurrency of angle bisectors of a triangle.
Answer: I'll exploit a result that I presented in this answer, with different notation:

Extended Ceva's Theorem. Defining ratios of signed lengths,
$$\alpha^+ := \frac{|BA^+|}{|A^+C|} \qquad \beta^+ :=\frac{|CB^+|}{|B^+A|} \qquad \gamma^+ := \frac{|AC^+|}{|C^+B|}$$
$$\alpha^- := \frac{|CA^-|}{|A^-B|} \qquad \beta^- :=\frac{|AB^-|}{|B^-C|} \qquad \gamma^- := \frac{|BC^-|}{|C^-A|}$$
the lines $\overleftrightarrow{A^+B^-}$, $\overleftrightarrow{B^+C^-}$, $\overleftrightarrow{C^+A^-}$ concur iff
$$\alpha^+\beta^+\gamma^+ + \alpha^-\beta^-\gamma^- + \alpha^+\alpha^- + \beta^+\beta^- + \gamma^+\gamma^- \;=\; 1\tag{$\star$} $$

If $A^+$, $B^+$, $C^+$ are midpoints, then $\alpha^+=\beta^+=\gamma^+=1$, and $(\star)$ reduces to
$$\alpha^-\beta^-\gamma^- + \alpha^-+\beta^-+\gamma^- = 0$$
However, we'll find that the angle bisector context makes it natural to express the "$-$" ratios in terms of the "$+$" ones; so I won't invoke the midpoint property prematurely.

Focusing on the angle bisector from $A$ aspect, let $\overleftrightarrow{A^+B^-}$ be parallel to that bisector, and let it meet the extended side $\overline{AB}$ at $V$.

With $u$, $v$, $\theta$ as labeled in the figure, we can use the Law of Sines on $\triangle BA^+V$ and $\triangle CA^+B^-$ to write
$$\frac{\alpha^+ u}{c+v} = \frac{\sin\frac12A}{\sin\theta} = \frac{u}{b-v} \quad\to\quad v = \frac{\alpha^+b-c}{\alpha^++1} \quad\to\quad
\beta^-:=\frac{v}{b-v}=\frac{\alpha^+ b-c}{b+c}$$
Likewise,
$$\gamma^-=\frac{\beta^+ c-a}{c+a} \qquad \alpha^-=\frac{\gamma^+ a-b}{a+b}$$
Substituting into $(\star)$, we can write
$$\frac{(1 + \alpha^+)(1 - \beta^+ \gamma^+)}{a}
+ \frac{(1 + \beta^+ )(1 - \gamma^+\alpha^+)}{b}
+ \frac{(1 + \gamma^+)(1 - \alpha^+\beta^+ )}{c} = 0\tag{$\star\star$}$$
This general condition for concurrence of bisector-parallels through $A^+$, $B^+$, $C^+$ is clearly satisfied when $\alpha^+=\beta^+=\gamma^+=1$ (ie, when $A^+$, $B^+$, $C^+$ are midpoints). $\square$