Analysis of proportions in repeated experiments

Question: What would be the best solutions to analyse proportions of multiple classes.
The classes are mutually exclusive, with each observation falling into one of the classes (i.e. no other class is possible) and are unordered.
For example, having the following data, where observations in the same experiments are dependent (i.e. for the same Exp A+B+C = 100% in each condition)

Percentage
Counts
Class
Group
Exp

80
2475
A
Control
1

81
2906
A
Control
2

76
5102
A
Control
3

11
351
B
Control
1

11
405
B
Control
2

14
938
B
Control
3

9
280
C
Control
1

8
282
C
Control
2

10
639
C
Control
3

75
2380
A
Treated
1

76
2584
A
Treated
2

69
2210
A
Treated
3

15
480
B
Treated
1

15
510
B
Treated
2

16
530
B
Treated
3

10
310
C
Treated
1

9
290
C
Treated
2

15
480
C
Treated
3

prop_data <- data.frame(Percentage = c(80, 81, 76,
                                       11, 11, 14,
                                       9, 8, 10,

                                       75, 76, 69,
                                       15, 15, 16,
                                       10, 9, 15
            ),
            Counts = c(2475, 2906, 5102,
                       351, 405, 938,
                       280, 282, 639,
                       2380, 2584, 2210,
                       480, 510, 530,
                       310, 290, 480
            ),
            Class = factor(rep(c("A", "B", "C"), each = 3)),
            Group = factor(rep(c("Control", "Treated"), each = 9)),
            Exp = 1:3)

I would like to see whether the proportion of A,B and C is dependent on the Group
I thought of doing something like
model <- glm(Percentage ~ Class * Group, data = prop_data, family = poisson)

I am not, however, completely sure whether this is correct. For example, I am not dealing with counts but rather with percentages (so, not integers) here. Also, should I include the experiments as a random factor (I tried using glmer but I get a singular fit)?
I have also considered using a multinomial model but am not quite sure how to set it up.
EDIT:
I have added raw counts to the R code, as suggested in the comments
Comment: If you could post those instead, I think that would help a lot more.
Comment: How is the percentage calculated?  Do you have the raw counts or data from which this was computed?
Comment: Is there an inherent ordering to the `Class` values "A," "B," and "C," (like one is preferable to the next etc.) or are they simply 3 potential outcome classes without any order? As the answer depends on that, please edit the question to provide that information; comments are easy to overlook and can be deleted.
Comment: Yes, I could can get raw counts
Comment: OK, I'll post them tomorrow (don't have them with me right now!)
Comment: The `Class` labels in your original table don't agree with the annotations of the `Percentage` lines in your construction of `prop_data`, but they do agree with the way the `Class` values are set up in `Class=factor(rep(c("A", "B", "C"), 3))`. To line up instead with the annotations of the `Percentage` values, you would need `Class=factor(rep(c("A", "B", "C"), each=3))`. Which do you want? Makes a big difference in the results.
Comment: @EdM oh, sorry, yes forget about the comments my bad. This is not the real data anyway, I just want to get a general answer, don't care which way the results go in this particular example.
Comment: There is still something that doesn't make sense in the sample data. You say "for the same Exp A+B+C = 100% in each condition" in the question, but what you have in your table is the sum of A+A+A = B+B+B = C+C+C =100%  within each `Group` and `Exp`. I think that what you intend is for `Class` to be a multinomial outcome, but your example data aren't consistent with that. Why not show the real data?
Comment: @EdM Because the real data has other complications which are irrelevant to the question and I just want a theoretical answer on how to approach this type of question. Now the data is fixed, apologies for the mixup, I edited it in a hurry and did the wrong thing.
Comment: @EdM: added to the question, no specific ordering and each element is in one of the 3 classes (there is no other possible options, nor missing values)
Answer: With "A," "B," and "C" as mutually exclusive and unordered outcome classes in the trials of each combination of experiment (Exp) and treatment Group, this is a standard application of multinomial logistic regression.
One Class is taken as the reference and regression coefficients then express predictor-associated differences in log-odds of each of the other classes with respect to that reference class. This differs from a set of binomial regressions against the reference class, as the probabilities among classes are constrained to sum to 1. The principles are nicely outlined in this UCLA web page, which recommends the multinom() function in the R nnet package
As your data are already aggregated, you use the Counts as weights for each of the outcomes. Treat Exp as a fixed effect after making sure it's interpreted as a factor rather than numeric. The following allows for the effect of the treatment Group to differ among experiments.
library(net)
prop_data$Exp <- factor(prop_data$Exp)
mnMod <- multinom(Class~Group*Exp,data=prop_data,weights=Counts)
## some output about convergence is omitted
summary(mnMod)$coefficients
##   (Intercept) GroupTreated        Exp2      Exp3
## B   -1.953216    0.3521456 -0.01742925 0.2595691
## C   -2.179205    0.1409320 -0.15352828 0.1017433
##   GroupTreated:Exp2 GroupTreated:Exp3
## B      -0.004179719       -0.08637392
## C       0.004549808        0.40957042

The overall reference is for Class "A" under Group "Control" in Exp 1. Intercepts for "B" and "C" are log-odds differences from "A" at those reference predictor values. Regression coefficients are further log-odds differences associated with Group, Exp, and the interactions between them.
For these sample data you would need to pay attention to systematic differences among the experiments, as some Exp and Group:Exp (interaction) coefficients are quite significant:
summary(mnMod)$coefficients/summary(mnMod)$standard.errors
##   (Intercept) GroupTreated       Exp2     Exp3
## B   -34.24556     4.641310 -0.2237779 3.862921
## C   -34.56243     1.614329 -1.7310543 1.343344
##   GroupTreated:Exp2 GroupTreated:Exp3
## B       -0.04000224        -0.8928776
## C        0.03672621         3.7516681

The coefficient estimates are assumed to have a limiting normal distribution, so these ratios to their standard errors are z-scores. A ratio with absolute value > 1.96 would be considered significant at p < 0.05.
You might wonder what happened to all of the information about Class "A." That's implicit in the constraint that the probabilities among all classes sum to 1.
Post-modeling software like the emmeans package allows you to extract that information and perform appropriate post-hoc tests. For example, if one could ignore the experiment-specific outcomes and average over the experiments, you would get the following report of Class probabilities and standard errors as a function of Group:
library(emmeans)
emmeans(mnMod,~Class|Group)
## Group = Control:
##  Class   prob      SE df lower.CL upper.CL
##  A     0.7898 0.00368 12   0.7818   0.7979
##  B     0.1221 0.00295 12   0.1156   0.1285
##  C     0.0881 0.00257 12   0.0825   0.0937
##
## Group = Treated:
##  Class   prob      SE df lower.CL upper.CL
##  A     0.7336 0.00446 12   0.7239   0.7433
##  B     0.1556 0.00367 12   0.1476   0.1636
##  C     0.1109 0.00317 12   0.1039   0.1178
##
## Results are averaged over the levels of: Exp
## Confidence level used: 0.95

That suggests that "Treated" leads to an increase in probabilities of "B" and "C" with a corresponding decrease in probability of "A."
Comment: Thank you for the extensive answer @EdM. I am wondering, though, why you would choose to treat the experiment as a fixed effect rather than a random effect, given the nested nature of the data (and also the fact that the experiment is not really a predictor, but rather a nuisance factor)?
Comment: @nico First, one typically only uses random effects when there are about 6 or more clusters. The sample data only had 3 clusters (experiments). Second, I don't think that `multinom()` in `nnet` allows for random effects. I thought it best to use the function explained by the UCLA web page, to get at the main point of the question--how to deal with a multinomial outcome.
Comment: Thanks EdM, I think I will have to dig a bit more into `glmer` for the random effects bit. I am not quite sure I agree with using random effects only if there are 6 or more clusters. I would expect the effects of pseudoreplication to be more problematic when there are fewer groups as there is likely to be a large inflating of degrees of freedom.
Comment: @nico see [this page](https://stats.stackexchange.com/q/37647/28500) and its links. The difference is whether you fit intercepts/slopes individually per cluster (fixed effects) or as a group with Gaussian distributions (random effects). Random effects will be poorly estimated with few clusters. To avoid pseudoreplication problems, set up statistical tests properly in a way that respects the hierarchy of the experimental design. For example, if different clusters receive different treatments, the treatment effects need to be compared against the among-cluster variance, not within-cluster.


Playing matching chords to melody by ear

Question: I'm a beginner on piano who plays be ear using pentatonic scale and I'm having hard time accompanying the melody with a chord with the left hand. I can easily play the melody to any song but if I want to accompany someone singing, I don't know what chords to use.
The way I play chords is for example in key of C if the melody rests on note G, I would play G major chord if it rests on C play c major, so I follow the melody and play matching chords to the note of the melody. The problem I'm facing is that It's hard to determine the key by ear where the melody rests when someone is singing and then playing a matching chord to that key.
Is there an easier way to do this? Writing out the chords to the song would work out but I was wondering if it could be done just by listening
Comment: I think you should unlearn the strange idea of thinking that only G chord is OK when G note is sounding. Simple arrangment means at least defining the key (e.g. in Cmaj the chord C will be the tonic - chord I), G is chord V, F is chord IV. If you hear that the song is developing from I to IV to V than back to I, you see the structure and choose these chords C-F-G-C. So the choice of chords is more about seeing how the song develops.
Comment: If you want a magic trick, add a sixth (e.g. in C major, for a C melody note, add an E harmony note below, for D add F) below the melody. Whatever the melody note is, add a sixth below it. MAGIC! It's completely mechanical, you don't even have to know about any chords, just the scale. But don't call yourself a musician then. :) The idea of matching chords is a misconception. For any melody, there's any number of plausible, suitable chords that could be used, and they'll have different feelings and possibilities for voice leading etc. Some chord choices might feel unexpected or unconventional.
Answer: For finding chords to melodies, there's just one word: practice. There's no other way. There's no substitute for hearing and feeling. Forget about the "this note, so this chord" magic cheat idea. In some cases tricks like that can be applied, but it would still only improve your hit rate from "completely random" to "not completely random" level. Would you be happy if you could recognize your friend's face in a crowd even as much as 60 % of time, using some trick?
Learning the skill consists of two parts: (1) producing i.e. playing, and (2) listening. To play "by ear" means listening to an example and trying to answer the question "what would I play to make it sound like that".
To answer that question successfully immediately after hearing something, you have to have played many things and heard what they sound like many, many times, in many different contexts. When you're not very skilled and experienced yet, you'll have a narrower and less familiar palette of things to try, so there'll be more trial and error. If it doesn't sound like what you wanted, try something else. Keep practicing and you will get better. Your musical palette will expand, and the "hit rate" will improve.
Here are some exercises and hints to try:
Find the melody and chords to simple songs by ear. "London Bridge is Falling Down", "Twinkle, Twinkle, Little Start", "Happy Birthday", "Hit the Road, Jack", etc. First in a familiar key, like C major, and then do the same in other keys.
For finding the key, locate the home base note ("tonal center" or whatever). Some people try to explain what the tonal center feels like, but you have to learn to feel it. You gradually learn what it feels like by playing simple songs in different keys. In the key of C, what does the C note feel like? In the key of C, the home base is C. In the key of F, the home base is F. Learn to know what F feels like, when you're playing a song in F.
To establish a key, play a I - IV - V - I cadence.

To establish the key of C major, play the chords: C - F - G - C.
To establish the key of F major (key signature: 1 flat), play the chords: F - Bb - C - F.
To establish the key of G major (key signature: 1 sharp), play the chords: G - C - D - G.
To establish the key of A minor, play the chords: Am - Dm - E - Am.
etc.

(That's also a clear way to start a song. First play a I - IV - V - I cadence, and play the song's starting pitch and chord, before the singer starts singing. ;))
To play a simple song in a new key, first establish the key (see above), and then find the melody and accompanying chords by ear. Repeat. At first it is hard, but you'll get better at it. No gain without pain.
To find a melody by ear, you have to have a reference to compare to, so it has to be a song you know well. Play, listen, compare.
To find chords by ear: trial and error, lots of trial and error. Play simple songs that can be played with e.g. I/IV/V i.e. C, F and G major chords in the key of C major. Try each chord and listen. Maybe someone should pick "safe" songs for you to practice with, but in any case, there has to be a lot of "babbling" like what babies do when they start learning to speak. When you're a grown-up, you'd like to skip that embarrassing phase, and jump straight to respected legitimate things, but no can do.
You have to know what a "I chord" feels like, what a "IV chord" fees like, what a "V chord" feels like, etc. Does something feel like a V chord? Does it feel like a I chord? If you don't get any such feelings, then you haven't practiced enough. Practice more.
Like Tim says in his answer, a good bread-and-butter set of chords is, three chords of the major key and three chords of the minor key. For songs in the key of C major / A minor (key signature: no sharps/flats, white keys) they would be, C, F, G, Am, Dm, E (i.e. G sharp in that chord, E-G#-B). Babble with those six chords, and try to accompany melodies you know with them.
To learn to identify and analyze the things you're hearing, alter the components (notes) and listen to what the changes do. Play the notes of chords one at a time, leave out a note, move a note up or down, etc.
To get more ideas about things to try, check out song books (lead sheets) and try to understand the role of each chord. I, IV, V, ii, vi, etc. When you see an accidental in the melody, there's probably a temporary scale change that's also reflected in the chords. For example, in a song that's in A minor key (key signature: white keys), there might be an A major chord before going to a Dm, and the melody must take that into account and play a C# instead of C. Or there might be a Gm and A7 before going to the Dm, both of which have "non-white keys". Tricks like this are common in songs, but trying those chords in that context might not cross your mind by just randomly poking at chords.
Answer: Working on a very simple (but effective, and often accurate!) premise that a lot of songs have three main accompanying harmonies, it's not too onerous.
Most songs will start on the key chord; if there's an anacrucis, the first main bar will be on that. Call that the I chord, in whatever key. Usually, we can feel a change coming, and with two other chords to choose from, IV and V, there's a 50:50 chance just by guesswork. But a hint is if it sounds like it's going up from that I chord, rather like a 12 bar song does in bar 5, then IV is a better bet. When the next change is felt, there's still 50:50 - back to I, or onwards to V. That should be an easier one to forecast - back home, or ever onwards.
Be aware that just because there's a G note sung, doesn't automatically mean it's over a G chord - in whatever key, in reality. In a key, there's a very good chance a main note sung also belongs in the harmony. That's fine. G note is obviously in G chord. But it's also in C chord. 50:50 again! But it's not in F (the IV of C), so there's little chance of that harmony being an F chord.
Also be aware that sometimes a chord can and will continue for three or four bars - rarely longer in most songs - so even though the melody may be changing, it doesn't mean the chord has to, or will.
Taking the harmony further, using diatonic chords (those using only the notes from the key), there are also three minor chords which come into play. ii, iii and vi, the minors. Again, after a while using all six, there's a good chance of guessing. If experience tells you it's going to be a minor, there's a 33% chance of hitting the right one.
Private practice with more simplistic tunes - Happy Birthday, Christmas songs, national anthem, etc., will help. A lot of songs can be simplified down to three chords, and won't sound too bad, but you'll hear things like there's one note from a chord that isn't quite right. Have a delve into triads, but also 4 note chords, and also look at the circle of fifths, as sometimes tunes modulate, and use extra chords which are often very next door neighbours looking at that circle.
Answer: Study how songs HAVE been harmonised.  Sometimes the 'three chord trick' is appropriate, sometimes not. (For a whole load of current popular music, it isn't.).  By far the easiest way of doing this is by reading notated music.
Comment: Looking at ready-made answers is of course _easy_, and it's an excellent thing to do, if you have no idea what an answer might look like, or if you've tried but failed to produce a satisfactory answer. But IMO it's very important to try and err. :)
Comment: It's not a matter of cheating by looking up the answer for one particular song.  More of being familiar with a wide range of music, getting familiar with how songs in general are harmonised, in various styles.
Answer: Don't stare too hard at a tree, look at the whole forest.
When you were a baby, there was no trick, shortcut or method for teaching you to speak.  You did your best to imitate the baby noises your parents made until you finally latched onto one and they rewarded you with affection and joy whenever you said it.  Eventually you learned that imitation yielded results and putting words together was a science and art.  Here you are today, improvising sentences.
Likewise, there are no tricks, shortcuts or methods for learning harmonization. Harmonization is what YOU want it to be.  There is no wrong, only what you wish to communicate.
To discover those you need to do five things:  Practice, study theory, sing everything, listen to everything, experiment. The same things you did as a baby.
Here is a little experiment for you.  With your right hand play a C.  Just an octave lower play an Eb chord.  Now play the C and in the left play a D chord.  Now a Db.  Resolve it to the C.  Put all four together in a sequence.  Try the C with other chords such as an F# (I love b5 chords).
Now you can see that multiple chords can fit over one note.  Try this: Play a simple pentatonic melody in your right hand in the key of C. In your left hand, alternate between a C and Bb chord every bar.  Don't like it?  Try a C and F.  C and G?
Try experimenting with modes. This IS a cheat trick method but you should learn the modes properly.  Play a C note in your left hand with any rhythmic pattern you like. Now play with a few scales in your right  hand.  They don't have to be SCALES but notes of that scale.  Try the G scale (lydian) with a focus on the F# (flat fifths again).  Try a Bb (dorian).  One of my favorites is the locrian, Db.  I use Locrian as a passing melodic harmonization for either shock, color or I don't know what else to do.
So, there are no cheats, tricks or methods.  Only practice, study, experiment and practice some more.  Oh, and listening to other people, then steal their ideas.
Everyone steals.  I remember as a kid listening to the band BOSTON as they played their song FOREPLAY and I immediately recognized a chord progression from Sinfonia in Bach's Cantata 29.  It is at the 1:28 mark of the song.
Whenever I improvise classically I throw in that progression and everyone thinks I'm a genius but I'm really just a thief.  As you were reading this, I took your wallet.


Is there a self-correcting iterative method for approximating pi without using transcendental functions?

Question: The Newton-Raphson method is an iterative method for finding a root of a function, and it is self-correcting in the sense that any error in the initial input is reduced with each iteration so that it will still converge to the correct answer.
Is there a method to compute pi to higher and higher precision that has the same self-correcting property but also does not use transcendental functions? I have only been able to find methods that use transcendental functions, like applying Newton-Raphson to $x - \tan(x)$ or iterating the formula $x = x + \sin(x)$ over and over.
Comment: If you make your question precise, then I think it would require pi to be an algebraic number and it is not.
Comment: @noumenon28 Do you consider the following quadratically convergent iteration based on the AGM (from Borwein & Borwein, *Pi and the AGM*, Wiley 1987) as an iteration of the desired type? On p. 46: $x_{0}:=\sqrt{2}$,
$\pi_{0}:=2+\sqrt{2}$, and
$y_{1}:=2^{1/4}$.
Define
$x_{n+1}:=\frac{1}{2}\left(\sqrt{x_{n}}+\frac{1}{\sqrt{x_{n}}}\right)$,
$y_{n+1}:=\frac{y_{n}\sqrt{x_{n}}+1/\sqrt{x_{n}}}{y_{n}+1}$,
$\pi_{n}:=\pi_{n-1}\frac{x_{n}+1}{y_{n}+1}$. The $\pi_{n}$ converge monotonically to $\pi$.
Comment: There are many infinite sums and infinite products for $\pi$ where the error is reduced each time you include a new term.
Comment: [Take your pick](https://en.wikipedia.org/wiki/Approximations_of_%CF%80#Development_of_efficient_formulae).
Answer: I was given this one
$$a_n=a_{n-1}+\sin (a_{n-1}) \qquad \text{with} \qquad a_0=3$$
$$\left(
\begin{array}{cc}
n & a_n \\
 0 & \color{red}{3.}0000000000000000000000000000000000000000000000000 \\
 1 & \color{red}{3.141}1200080598672221007448028081102798469332642523 \\
 2 & \color{red}{3.1415926535}721955587348885681408797467429928172150 \\
 3 & \color{red}{3.14159265358979323846264338327950}9759271524572210 \\
 4 &\color{red}{ 3.1415926535897932384626433832795028841971693993751} \\
\end{array}
\right)$$
Comment: Note that the iteration of $x \mapsto x + \sin(x)$ is already mentioned in the question, and the author explicitly asks for methods which do "not use transcendental functions."
Comment: I also doubt that the method is "self-correcting." If some $a_n > \pi$ due to rounding errors, then the next itererates increase further towards $2\pi$.
Comment: @MartinR. I do not agree. Use Darboux theorem and you avoid all problems.
Comment: I am not sure what you mean. $(a_n)$ is strictly increasing except at the points $k \pi$, so any rounding error which leads to some $a_n > \pi$ will not be "corrected." Perhaps it depends what "self-correcting" exactly means. – In any case, my first comment still applies.


Limit of vectors and limit of components

Question: This question is related to the (closed) topic :
Closed subspace of vector space
Let $(u_n)\in F^{\mathbb{N}}$ be a sequence converging to $l$ and $(e^1,\ldots,e^p)$ be a basis of $F$ (vector space).
There exists $(x_n)_n\in (\mathbb{R}^p)^\mathbb{N}$ so that for all $n$,
$$
u_n = x_n^1e^1+\ldots x_n^p e^p.
$$
Also $(u_n)$  converges if and only if all the real valued sequence of its components converge.
Then, denote by $x_\star^i$ the limit of $x_n^i,\, i=1\ldots p,$ and $u_\star$ the limit of $u_n$.
Then $u_\star = x_\star^1e^1+\ldots +x_\star^pe^p$.
What is wrong here ?
Comment: What does $x_n^1e^1$ mean?  $x_n^1$ and $e^1$ are vectors, aren't they?  How are you multiplying them?
Answer: $\def\R{\mathbb R}$
I see by looking at your other question that you intend $F$ to be a linear subspace of a real Euclidean space $E$.  $F$ then has a metric $||\cdot||$ induced by the metric on $E$, and you also give $F$ a fixed basis, which induces a vector space isomorphism $\phi :\R^p \to F$, where $p = \dim F$.
You ask in effect whether the following statements are true:

$F$ is closed in $E$.
$F$ is complete with respect to the metric inherited from $E$.
The vector space isomorphism  $\phi :\R^p \to F$ is a homeomorphism, where $\R^p$ is given the standard topology.
A sequence in $F$ converges if and only if its coordinate sequences with respect to the given basis converge.

Yes, all four statements are true and they are more or less equivalent.  There are several ways to go about proving this.  For example, you can go about it this way:  The norm on $F$ gives you a norm $||\cdot||'$ on $\R^p$ via $||x||' = ||\phi(x)||$.  It is a theorem that any two norms on $\R^p$ are equivalent, and convergence with respect to any norm on $\R^p$ is equivalent to co-ordinatewise convergence. This theorem implies statements 2, 3 and 4, and statement 1 is a consequence.
Addendum:  For the equivalence of all norms on a finite dimensional space, see https://math.stackexchange.com/q/2702434.  This uses compactness of the unit sphere with respect to one of your favorite norms.  Another approach to the whole problem would be to show statement (1) first, as follows.  Show that $E = F \oplus F^\perp$ and use finite dimensionality to show $F = (F^\perp)^\perp$.  Conclude from this that $F$ is closed.
Answer: This statement is not justified:

Also $(u_n)$ converges if and only if all the real valued sequence of its components converge.

How would you prove it?
It is related to the question whether $l \in F$, which is precisely what you are trying to prove.
What you should try to prove is that the finite-dimensional space $F$ is complete (i.e. every Cauchy sequence converges). Then it will follow that $F$ is also closed.
Comment: There is no space of set mentioned in which the limit or it's components is/are supposed to be. Without that the statement is simply true. "$u_n$ converges" is a statement about the existence of a llimit. That this does not imply additional properties like some set being closed is correct, but I do not see any of these in the (in my opinion) ill posed question.
Comment: if $u_n$ converges it has a limit $u$, which has a representation in the basis. It's quite obvious that the coordinates of the sequence will then converge to the coordinates of the limit. The other direction is the same.
Comment: @Thomas True, if the limit $u$ is in your space. That is precisely the issue here.
Comment: Ok so this statement is not justified. And the only way to prove it is to prove that the set is closed ?
Comment: @Thomas The question is vague but I understood that $(u_n)_n$ is in the finite-dimensional space $F$, which converges to $u \in X$ ($X$ being the larger space of which $F$ is a subspace) and we're trying to prove that in fact $u \in F$. The OP wrote $u_n = x_n^1e^1+\cdots +x_n^p e^p$ and then appealed to the above statement and concluded that $$u = (\lim_{n\to\infty} x_n^1) e^1 + \cdots + (\lim_{n\to\infty} x_n^p)e^p \in F$$
without additional argumentation.
Comment: @Smilia You mean complete? I don't know if there is another way.


What modules are required to make a seating chart website?

Question: I'm currently working on a site that wants to sell theater seating by section by clicking on a map overview of the seats like the one below. What drupal 7 modules are there or recipes that can be done to create this similar functionality?

Comment: Did you search for modules on drupal.org? What did you find, and what was not helpful for you?
Comment: I want to add to what @kiamlaluno meant by his comment, just in case you missed it.  your question right now is HUGE. you're asking someone to do a TON of research for you.  It's not a specific question and you're not suck on a particular technical detail.  This forum is not set up to answer a question like that.  Also, I think it's a bit dumb to expect people give you that much work for free.
Comment: no i thought there might have been a module that already exists to do something to this extent out of the box. i wasn't expecting anyone to do any work just to answer this question. im posting this question up to see if there was anyone that did a similar site and how they went about doing it. I'm not asking anyone to do the work from scratch. sorry if it came out that way as obviously that was not my intention.
Comment: FWIW I went hunting for a module to do exactly the same thing 2 days ago - didn't find anything
Comment: Clive is right. I have done a ton of searching in this realm and didn't find anything useful.
Comment: good to know. i guess this would have to be done from scratch. i'll post my solution once thats done.
Comment: Talking about doing "research" (as in some previous comments), and since the title includes "chart", I thought it might help to point to https://www.drupal.org/node/2363985 (= current result of "my" research about charting modules). Anybody interested in going over the dozens of charting modules mentioned there? To see if any if it would help to find an appropriate module to solve this "challenging" question, and then publish a real answer here? Please? As Alan Bean (Apollo 12) would say "There HAS to be a way to do this (i.e. 'a' charting module) ..."! PS: why not EDIT the question also?
Answer: Closest things I was able to find are Wysiwyg Image Map and jQuery Map Hilight modules.
Wysiwyg Image Map is image Map editor for tinymce provided by the Wysiwyg module.
jQuery Map Hilight enables the simple addition of mouseover highlighting of hotspots to HTML image maps without requiring the editing of theme files or knowledge of javascript/jquery.
Together they should give you roughly what you need - but if I were in your situation, I would prefer to code this myself.


Describing the homotopy explicitly.

Question: Here is the question I am trying to solve:
Let $X$ be a based space, and let $PX = \{ \beta: I \to X | \beta(0) = *\}.$ Show that $p_1: PX \to X$ by $p_1(\beta) = \beta(1)$ is a based fibration.
I am confused about what should be the exact homotopy that I should write, here is a trial:
We have the following commutative diagram:

and here is a trial to the homotopy I should write:
$H'(y,s)(t) = \begin{cases}
f(y) \left( \frac{2t}{2-s} \right) & 0 \leq t \leq 1- \frac{s}{2},\\
H(y, 2(t-1)+s) & 1- \frac{s}{2} \leq t \leq 1.
 \end{cases}$
Is this a correct homotopy or not? Can it be written and thought of in a more organized and systematic way.
Edit for Anne:
You draw a square like below and then you draw your homotopy as a line joining s(down) with s(up) then you determine the coordinates of the points on both the s axes and then you calculate the equation of the line joining the points on the x-axes which help in determining the point at which your homotopy changes its definition.

EDIT 2 for Anne :

Is this what you meant Anne?
Answer: I checked your homotopy and I think it is correct. The following seems to be correct also (?), and simpler:
$$H'(y,s)(t) = \begin{cases}
f(y)(s+t) & s+t\le1,\\
H(y,s+t-1) & s+t\ge1.
 \end{cases}$$
Comment: I drew a square with $f$ on the left vertical edge and $H$ on the upper horizontal edge. Then a diagonal from $(0,1)$ to $(1,0),$ and then horizontal segments on the right of this diagonal, all along, with $H$ written on them, and vertical segments underneath with $f$ written on them. All these segments were of length 1 but I cut them to retain only the part lying inside the square.
Comment: I mean how did you construct your homotopy? people usually draw a square with t as the horizontal (x - coordinate ) and s as the vertical (y - coordinate) and express the homotopy in terms of this ........... you can use quiver https://q.uiver.app/  on your laptop, I will edit my post to include an example of a picture of a homotopy like in Harpreet Bedi video "Path in space to fundamental group" https://www.youtube.com/watch?v=VVxPwbUCep0&list=PLCgncMh0TrClLOU991FDEQK5LF_Aqm4FS&index=5   min. 6
Comment: Your right vertical $f$ should be the segment from $(1,-1)$ to $(1,0)$ because above, there are the horizontal segments representing $H.$ This is only one way (among many others, like yours) to fill the square, given the left vertical segment $f$ and the upper horizontal segment $H$ (compatible at their common point $(0,1)$ since $H(y,0)=f(y)(1)$) imposed by the commuting diagram.
Comment: got it thank you very much!
Comment: Can you draw yours for me by a square with the beginning point of the line and the end point please?  ...... Also, how did you check mine? Can you draw it also with a square please?
Comment: Sorry, I am not able to draw on a PC. (Anyway, I don't understand what you want me to draw.) What I checked (on both) is that it is consistent in the limit case, and that it makes the diagram commute.
Comment: The example in your video is not of the same nature. I don't see how to adapt this way of drawing to picture usefully your $H'$ or mine.
Comment: I mean can you tell me the intuition that leads you to write your homotopy like this please?
Comment: How did you get $s+t$ in your homotopy? and how did you get $s+t-1$?
Comment: I will try to draw your homotopy square (in an hour) and add it in my post ..... please confirm if it is correct or correct me if I am wrong.
Comment: It is something like that but the right vertical $f$ should be one step below. Actually, there are an infinity of vertical $f$'s, all below the diagonal, and (between the two you drew) an infinity of horizontal $H$'s, all above the diagonal.
Comment: What do you mean by one step and why it should be like this?

Describing the homotopy explicitly.


Does Npgsql support community version of postgres?

Question: I read from the enterprise db website that only the Postgres standard server and advanced server support Npgsql.
But am not sure if
 1. Npgsql will not work with community version at all ?? or
 2. It is just that it is not packaged along with community version,but I can use npgsql to connect and work with postgresql from .Net
Can someone help me understand?
Answer:
I read from the enterprise db website that only the Postgres standard
  server and advanced server support Npgsql.

EnterpriseDB is not the same as PostgreSQL. PostgreSQL is PostgreSQL, it's not a "community version" from EnterpriseDB. EnterpriseDB Server is a fork of PostgreSQL, not the other way around.
And yes, Npgsql does work with PostgreSQL, it was created for PostgreSQL:

Npgsql is a .Net Data Provider for Postgresql. It allows any program
  developed for .Net framework to access database server. It is
  implemented in 100% C# code. Works with Postgresql 7.x and above.
Comment: Great. So this means I have to download PostgreSQL from the postgresql site. Is it from here http://www.postgresql.org/ftp/binary/ ?
Comment: If you're looking for PostgreSQL, get PostgreSQL. If you want EnterpriseDB Server, get their product. EnterpriseDB does host the one-click installers for PostgreSQL: http://www.enterprisedb.com/products-services-training/pgdownload You can find this link also on postgresql.org: http://www.postgresql.org/download/


Any performance gain from multiple nvme drives in RAID versus a single nvme drive?

Question: We've got a cluster of scylladb hosts (it's a cassandra-type database) running on i3 instances in amazon with the /var/lib/scylla/ folder mounted on a single nvme drive. I'm wondering whether there is any i/o performance gain to be expected by replacing this single drive with a two- (or multiple-) nvme drives that are configured as a RAID 0. In order words would striping give us a noticeable performance boost on this type of drive?
Comment: @Horaciux Yes I did. Tested Raid 0 and Raid 5 configurations. Raid 0 was noticeably faster. Raid 5 did not see any performance improvement, and saw drastic performance deterioriation when the raid was in a rebuilding state. Decided that Raid 5 was not suitable for our production environment. Raid 0 would be suitable to replace an ephemeral nvme drive, but unsuitable for replacing an nvme-based EBS volume (you would lose all your data if a single drive failed.)
Comment: Just to throw,in a second option, since it typically is hard to get enough parallel commands on NVMe drives (and Striping won't help,here) amother option is to run two nodes on one machine, both writing t their own disk. I dont know about Scylla, it does help with Ceph.
Comment: Thanks for that suggestion, I'll run it by the team. We are currently cpu-bound, so we wouldn't be able to do that until we mitigate the resource (scylla binds a thread to each cpu and dedicates specific keys to each core).
Comment: I found this guy compared nvme raid 0 versus single disks on his gaming PC: http://www.eteknix.com/year-nvme-raid-0-real-world-setup/6/. Summary: raid0 noticeably better for sequential r/w; raid0 worse to somewhat better for random r/w (depending which benchmark tool he used); raid0 substantially worse for access times for reads.
Comment: @MichaelMartinez did you test it yourself?
Answer: Maybe. You'll want to benchmark it yourself to find out.
If there are no other bottlenecks, then more disks is more IOPS. But that's a big if. Only by testing yourself will you find what the next bottleneck is on that configuration.
I don't really see the point of RAID 0. Zero is the amount of data you will recover from that pair if one of them fails. For scale out databases, might as well add another instance in a simpler single drive configuration.
Comment: We aren't concerned about data recovery from a single host. We can lose a host without losing data from the cluster. The point of RAID 0 is faster writes, because they can be parallelized.  This was the expected gain in the old days where your disk had a spindle that wasted time seeking. I don't know whether this is the case with nvme drives.
Answer: Even without spindles, multiple drives responsible for different data stripes can increase your performance, and Scylla being what it is (an insanely fast database) you will want to squeeze every bit of performance you can out of those instances
Comment: Yes see my own comment (the final one) to my question. Faster but unsuitable for EBS-backed nvme.
Comment: EBS is a generally bad idea for fast databases. i3 and the XXd instances are the better choice. The downside, of course, is that those fast NVMes are ephemeral, but with a replicated DB this shouldn't be too much of a concern. Are you still runing Scylla?
Comment: We had a 12-node scylla cluster. I don't recall whether I ended up with ephemeral nvme or EBS-backed nvme on those, but we also had other clusterd stuff running on EBS-backed nvme and didn't have any performance issues. I did some benchmark comparisons between the two types of storage but I don't recall the results.
Comment: EBS is not horrible, just not comparable to the ephemeral NVMes
Comment: Correct........


Como usar o While no Python 3?

Question: Minha dúvida é sobre o laço while, não consigo entender o funcionamento dele.
Escreva uma função em Python que recebe, por argumento, dois números inteiros e retorna a soma
de todos os números inteiros entre os dois (incluindo os dois).
meu código...
def somatorio(x,y):
    count = 0
    soma = 0

    while count < y:
        soma = y + (y-count)
        count += 1
    return soma

print(somatorio(1,2))
print(somatorio(1,5))

Comment: O que você quer saber especificamente? Você usou algo que não sabe para que serve?
Comment: era sobre a condição, a dúvida já foi sanada, obrigado ^^
Comment: Só pra constar que este código está errado, pois ele não retorna "_a soma de todos os números inteiros entre os dois_", deveria ser algo assim: https://ideone.com/6b1iuW <-- neste link também tem outras opções de como fazer
Answer: O while é um laço de repetição. Faz com que um conjunto de instruções seja executado enquanto uma condição é verdadeira. Quando essa condição deixa de ser verdadeira, o loop é interrompido, vide exemplo abaixo:
contador = 0
while(contador < 10):
    print(contador)
    contador = contador + 1

No exemplo acima, criamos um laço que executa o comando print por 10 vezes. Para criar uma variação, ou um laço infinito - muito utilizado em desenvolvimento de jogos - podemos fazer assim:
while(True):
    print("infinito")

Comment: Muitooo obrigado, essa parte da condição que estava me pegando...

Como usar o While no Python 3?


How to make rsnapshot backups in different locations?

Question: I have backup server that gathers backups from few other servers. I want to make directory structure like this
/backup/snapshots/<hostname>/<period>.0/<files>

Instead of default
/backup/snapshots/<period>.0/<hostname>/<files>

How to do this? I tried to specify parameter
backup  root@remote-host:/backup/snapshots/hourly.0        .       snapshot_root=/backup/snapshots/myhost/

but it doesn't work
Answer: You need to use another configuration file, where you can set different snapshot_root directory (and other backup params).
For example /etc/rsnapshot.additional.conf
After what you need to add to your rsnapshot cron file (if you use Ubuntu - you can find it /etc/cron.d/rsnapshot).
30 3    * * *       root    /usr/bin/rsnapshot daily
10 0  3    * * 1       root    /usr/bin/rsnapshot weekly
11 30 2    1 * *       root    /usr/bin/rsnapshot monthly

lines, like this
30 3    * * *       root    /usr/bin/rsnapshot -c /etc/rsnapshot.additional.conf daily
10 0  3    * * 1       root    /usr/bin/rsnapshot -c /etc/rsnapshot.additional.conf weekly
11 30 2    1 * *       root    /usr/bin/rsnapshot -c /etc/rsnapshot.additional.conf monthly

Which means that rsnapshot will be call two times with different config file and make two backups
Answer: As far as I can tell, rsnapshots only configuration for this is the "snapshot_root" directive.
Man page: "snapshot_root Local filesystem path to save all snapshots"
These are a few examples from: https://askubuntu.com/questions/20603/what-does-snapshot-root-do

snapshot_root /var/backups/me
...
backup /home/me/movies/ /media/
backup /home/me/music/  /media/
backup /home/me/drafts/ /documents/

After running rsnapshot hourly, for example, the snapshots would be
stored under

/var/backups/me/hourly.0/media
/var/backups/me/hourly.0/documents

So, pretty much what I'm saying is that I belive there's no easy way of changing that, and that you might need to change the source for this, but of course I  can be wrong and if that's the case, please ignore this answer.
Comment: Downvoting, because this answer is not helpful, stating an incorrect belief that the question can not be (easily) solved. (It can, see Dmitry's answer, which should be the accepted one.)


Comparing two logistic regressions

Question: I am working on creating a predictive model using logistic regressions. I am hoping to compare two different populations, using the same set of variables but different data sets with different sample sizes. Is there a good way to compare the two regressions in R to see if they are statistically different from each other?
Edit for clarification: My goal is to determine if the variables affect the outcome differently to a statistical degree. My analysis is about political socialization of two different populations, so I want to try to determine if xx population is socialized differently than yy population to a statistically significant degree. The two logistic models look different, but I am trying to give a more analytical answer than just "they look different".
Comment: Welcome to the site. @linog raises a valid point. You may also want to consider fitting a single model on both populations, with a variable for population and perhaps interactions with other explanatory variables. If you add to your question what the goal of this comparison is, one might conclude which method is best. You can edit your question using the button below it.
Comment: Could you be more specific regarding your objective ? You want to estimate a model on a sample and look at the performance on another one (in that case, this is basically using `predict`) or do you want to estimate two models (maybe with the same formula) on two samples and look at some performance measure (e.g. loglikelihood, BIC...) ? This does not bring the same answer.
Answer: This sounds like a job for an interaction term and a chunk test.
For two variables $x_1$ and $x_2$ in each regression, the math is as follows; extending to more than just two variables is straightforward. Let $g(p) = \log\left(\dfrac{p}{1 - p}\right)$. Let $x_3$ be an indicator variable that takes $0$ for one group and $1$ for the other.
$$
g\left(\mathbb E\left[y_i\right]\right) = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + \beta_3x_{i3} +
\beta_4x_{i1} x_{i3} + \beta_5x_{i2} x_{i3}
$$
The model includes the original two features, an indicator variable for the group, and interactions between the group indicator and the original two variables.
Nested within such a model, by setting $\beta_3=\beta_4=\beta_5=0$, is the following model that you would have used for each group separately.
$$
g\left(\mathbb E\left[y_i\right]\right) = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} +
$$
You can fit the above model to each group, or you can fit the first model to both groups simultaneously. Then the $x_3$ variable acts like an on/off switch, showing how the regression parameters differ between the two groups.
To test if the groups differ in slope, test $\beta_3=0$. To test if the groups differ in $x_1$, test $\beta_4=0$. To test if the groups differ in $x_2$, test $\beta_5=0$. You also can test $\beta_3=\beta_4=\beta_5=0$ to see if the groups differ in their regressions at all. This testing is no different from any other testing of logistic regressions.
Below, I demonstrate how to do this in R. The final line of lmtest::lrtest uses a likelihood ratio test of nested models to calculate a p-value, either for one coefficient (giving the difference between the parameters for each group regressed separately) or for the three coefficients involving $x_3$ (testing if group membership affects the regression, what I have learned to call a chunk test).
library(lmtest)
set.seed(2023)
N <- 1000
x1 <- runif(N, 0, 1)
x2 <- runif(N, 0, 1)
x3 <- rbinom(N, 1, 0.5)
z <- x1 - x2 + x3 - x1*x3 + x2*x3
p <- 1/(1 + exp(-z))
y <- rbinom(N, 1, p)
L_full <- glm(y ~ x1 + x2 + x3 + x1:x3 + x2:x3, family = binomial)
L0 <- glm(y ~ x1 + x2, family = binomial)
L4 <- glm(y ~ x1 + x2 + x3 + x1:x3)
L5 <- glm(y ~ x1 + x2 + x3 + x2:x3)
L3 <- glm(y ~ x1 + x2 + x3)
lmtest::lrtest(L0, L_full) # I get p = 0.00181 for the full chunk test
lmtest::lrtest(L0, L4) # I get p = 0.03512 for testing beta 4
lmtest::lrtest(L0, L5) # I get p = 0.9272 for testing beta 5
lmtest::lrtest(L0, L3) # I get p = 0.5902 for testing beta 3

Comparing two logistic regressions


Maxima can't solve relatively simple equation.

Question: I'm trying to solve this equation for x, with Maxima:

What I am doing wrong here?
declare([a, b, c, d], constant);
solve(a+b*sqrt((c+x)^2 - d) - x);

[x=b*sqrt(x^2+2*c*x-d+c^2)+a]

Obviously that is not a solution - x is on both sides!
Wolfram alpha solves it no problem:

Is Mathematica just much better than Maxima?
Comment: @Karl Yes, but mailing lists are terrible and Stack Exchange is amazing.
Comment: There is a maxima newsgroup here: email@example.com
Comment: That is true....
Answer: I get as answer, I have added some white space, with GNU Maxima 5.32.1
%union(
%if(?%and(
-%pi/2 < parg(
    -(b*(c+a)-sqrt(b^2*d-d+c^2+2*a*c+a^2))/(b^2-1)
),
parg(
    -(b*(c+a)-sqrt(b^2*d-d+c^2+2*a*c+a^2))/(b^2-1)
) <= %pi/2),
[x = -(-b*sqrt(b^2*d-d+c^2+2*a*c+a^2)+b^2*c+a)/(b^2-1)],%union()),
%if(?%and(
-%pi/2 < parg(
    -(sqrt(b^2*d-d+c^2+2*a*c+a^2)+b*(c+a))/(b^2-1)
),
parg(
    -(sqrt(b^2*d-d+c^2+2*a*c+a^2)+b*(c+a))/(b^2-1)
) <= %pi/2),
[x = -(b*sqrt(b^2*d-d+c^2+2*a*c+a^2)+b^2*c+a)/(b^2-1)],%union()))

The maxima code was
display2d : false;
load("to_poly_solve");
eq : a+b*sqrt((c+x)^2-d)-x;
%solve([eq],[x]);

Comment: I get `macro definition: bad argument: c` if I try to `load("to_poly_solve");`. Can you explain why this is even necessary?
Comment: the load statement makes another solver called "%solve(..)" available. Perhaps you have an old Maxima installation, which doesn't support this yet. If you have problems with that or similar ones, you may discuss these on the superb mailinglist  email@example.com
Comment: Ok. I have the latest version of Maxima. Maybe it's a bug. Thanks anyway!
Answer: This method is to complicated. Five lines are sufficient. Check the first lines of this answer. I will repair this later.
It seems that Maxima isn't able to do this without assistance.
I use the method that is shown in this answer  by @BillDubuque.
We use the variable w in our calculation and want to be sure that it has not assigned any value.
(%i1) kill(w)
(%o2)                                done

We define the equation we want to solve:
(%i2) eq1:a+b*sqrt((c+x)^2-d)-x
                                       2
(%o2)                    b sqrt((x + c)  - d) - x + a

The global variable algebraic must be set to true in order for the simplification of algebraic integers that we define with tellrat to take effect.
(%i3) algebraic:true
(%o3)                                true

The we need the expression under the square root for the tellrat and the subst statement. One can copy and paste it or type it in but I use a Maxima function to extract this part from the equation.
(%i4) t1:part(eq1,1,2,1)
                                        2
(%o4)                            (x + c)  - d
(%i5) tellrat(w^2 = t1)
                             2             2        2
(%o5)                   [(- x ) - 2 c x + w  + d - c ]
(%i6) subst(w^2,t1,eq1)
(%o6)                        (- x) + b abs(w) + a

The % references the result of the previous statement.
(%i7) solve(%,w)
                                          a - x
(%o7)                         [abs(w) = - -----]
                                            b
(%i8) %^2
                                             2
                                  2   (a - x)
(%o8)                           [w  = --------]
                                          2
                                         b

The function rattriggers the reductions defined in tellrat.
(%i9) rat(%)
                                            2            2
                      2                2   x  - 2 a x + a
(%o9)/R/            [x  + 2 c x - d + c  = ---------------]
                                                  2
                                                 b
(%i10) solve(%,x)
                       2           2            2     2
              b sqrt((b  - 1) d + c  + 2 a c + a ) + b  c + a
(%o10) [x = - -----------------------------------------------,
                                   2
                                  b  - 1
                                        2           2            2     2
                               b sqrt((b  - 1) d + c  + 2 a c + a ) - b  c - a
                           x = -----------------------------------------------]
                                                    2
                                                   b  - 1

This is the result we want.
We can combine the last few commands into one command to make the code more compact. I prefer the way were we can see the intermediate results
(%i11) solve(rat(solve(subst(w^2,t1,eq1),w)^2),x)
                       2           2            2     2
              b sqrt((b  - 1) d + c  + 2 a c + a ) + b  c + a
(%o11) [x = - -----------------------------------------------,
                                   2
                                  b  - 1
                                        2           2            2     2
                               b sqrt((b  - 1) d + c  + 2 a c + a ) - b  c - a
                           x = -----------------------------------------------]
                                                    2
                                                   b  - 1

Now we can reset the resources that we need only temporary to avoid unexpected side effects in later computations.
(%i12) kill(t1)
(%o12)                               done
(%i13) untellrat(w)
(%o13)                                []


Why does $(\sum_{i=1}^x \frac{1}{i}) - \ln(x+\frac{1}{2})$ converge to $\gamma$ a lot faster than $(\sum_{i=1}^x \frac{1}{i}) - \ln(x)$

Question: $(\sum_{i=1}^x \frac{1}{i}) -\ln(x)$
10: 0,62638316097421
100: 0,58220733165153
1000: 0,57771558156821
$(\sum_{i=1}^x \frac{1}{i}) - \ln(x+\frac{1}{2})$
10: 0,57759299680478
100: 0,57721979014049
1000: 0,57721570652656
Answer: Let $H_n=\sum_{k=1}^n\frac1 k$. The Euler-Maclaurin summation formula gives
$$H_n=\ln n+\gamma+\frac{1}{2n}+O(n^{-2}).$$
But
$$\ln(n+1/2)=\ln n+\ln\left(1+\frac1{2n}\right)=\ln n+\frac1{2n}+O(n^{-2}).$$
Therefore
$$H_n=\ln\left(n+\frac12\right)+\gamma+O(n^{-2}).$$
The error in the approximation is now $O(1/n^2)$ rather than $O(1/n)$.
Answer: This is not especially a number theory phenomenon. You are comparing a sum and an integral. Similar to this picture from a different question:
Now, before I paste in the diagram, just note that $f(n)$ is estimated by $\int_{n-1}^n f(t) dt \;,$ or by $\int_{n}^{n + 1} f(t) dt \;,$ but is estimated much better by
$$\int_{n - \frac{1}{2}}^{n + \frac{1}{2} } \; f(t) \; dt \;.$$
Also note that the advantage of using the halves happens only in that first term. After that, both asymptotic expansions are similar, just even exponents. Here we go,
$$  H_n = \gamma + \log \left( n + \frac{1}{2} \right) +\frac{1}{24\left( n + \frac{1}{2} \right)^2} - \frac{7}{960\left( n + \frac{1}{2} \right)^4} + O \left(\frac{1}{\left( n + \frac{1}{2} \right)^6} \right)   $$
Note
$$  \gamma \approx  0.57721566490153  $$
====================================================
<code>  double h = 0.0;

  for(int n = 1; n <= 105; ++n)
  {
    h += 1.0 / n;
    double half = n + (1.0 / 2);
    double below = h - log(half) - 1 / ( 24 * half * half   );
    double above = below + 7 / ( 960 * half * half * half * half   );
    cout.precision(16);
    cout << setw(3) << n <<  setw(20) << h <<  setw(20) << below <<  setw(20) << above << endl;
  }
</code>
=======================================================
<code>  n       H_n               below gamma        above gamma
  1                   1  0.5760163733733171  0.5774567025914241
  2                 1.5  0.5770426014591783   0.577229268125845
  3   1.833333333333333  0.5771690042937476   0.577217595158665
  4   2.083333333333333  0.5771983233883347  0.5772161052305335
  5   2.283333333333333  0.5772078306265884   0.577215799116901
  6                2.45  0.5772116298045227  0.5772157146288168
  7   2.592857142857143   0.577213381574137   0.577215686100886
  8   2.717857142857143   0.577214278092129  0.5772156749464131
  9   2.828968253968254  0.5772147748446763  0.5772156700700557
 10   2.928968253968254  0.5772150678554185  0.5772156677426398
 11   3.019877344877345  0.5772152496467665  0.5772156665501748
 12   3.103210678210678  0.5772153672357559  0.5772156659024226
 13   3.180133755133755  0.5772154460039576  0.5772156655328736
 14   3.251562326562327   0.577215500362116  0.5772156653130572
 15   3.318228993228994  0.5772155388494049  0.5772156651775194
 16   3.380728993228994  0.5772155667148677  0.5772156650912913
 17   3.439552522640758   0.577215587289521   0.577215665034905
 18   3.495108078196314  0.5772156027470478  0.5772156649971248
 19   3.547739657143682  0.5772156145413271  0.5772156649712566
 20   3.597739657143682  0.5772156236663827  0.5772156649531959
 21   3.645358704762729  0.5772156308153391  0.5772156649403632
 22   3.690813250217275  0.5772156364801514  0.5772156649310989
 23    3.73429151108684  0.5772156410156465  0.5772156649243132
 24   3.775958177753507  0.5772156446815149  0.5772156649192762
 25   3.815958177753507  0.5772156476703771  0.5772156649154918
 26   3.854419716215045  0.5772156501268676   0.577215664912616
 27   3.891456753252082  0.5772156521608239  0.5772156649104084
 28   3.927171038966368  0.5772156538565317  0.5772156649086969
 29   3.961653797587057  0.5772156552792992  0.5772156649073582
 30   3.994987130920391   0.577215656480186  0.5772156649063024
 31    4.02724519543652  0.5772156574994485  0.5772156649054635
 32    4.05849519543652  0.5772156583690728  0.5772156649047917
 33    4.08879822573955  0.5772156591146631    0.57721566490425
 34   4.118209990445433  0.5772156597568547  0.5772156649038104
 35   4.146781419016861   0.577215660312387  0.5772156649034516
 36   4.174559196794639  0.5772156607949213  0.5772156649031569
 37   4.201586223821666  0.5772156612156711   0.577215664902914
 38    4.22790201329535  0.5772156615838906   0.577215664902712
 39   4.253543038936376  0.5772156619072569  0.5772156649025438
 40   4.278543038936376  0.5772156621921698  0.5772156649024033
 41   4.302933282838815  0.5772156624439875  0.5772156649022848
 42   4.326742806648339  0.5772156626672178  0.5772156649021847
 43   4.349998620601827  0.5772156628656684  0.5772156649020997
 44     4.3727258933291  0.5772156630425677  0.5772156649020274
 45   4.394948115551322  0.5772156632006642  0.5772156649019654
 46   4.416687245986104  0.5772156633423058  0.5772156649019121
 47   4.437963841730785  0.5772156634695057  0.5772156649018663
 48   4.458797175064118  0.5772156635839953  0.5772156649018267
 49   4.479205338329423  0.5772156636872687  0.5772156649017924
 50   4.499205338329423  0.5772156637806188  0.5772156649017626
 51   4.518813181466678  0.5772156638651685  0.5772156649017368
 52   4.538043950697447  0.5772156639418947  0.5772156649017143
 53   4.556911875225749  0.5772156640116504  0.5772156649016947
 54   4.575430393744267  0.5772156640751811  0.5772156649016772
 55   4.593612211926086   0.577215664133143  0.5772156649016624
 56   4.611469354783229  0.5772156641861109  0.5772156649016494
 57   4.629013214432351  0.5772156642345918  0.5772156649016372
 58   4.646254593742697  0.5772156642790354  0.5772156649016271
 59    4.66320374628507  0.5772156643198376  0.5772156649016178
 60   4.679870412951736  0.5772156643573509  0.5772156649016096
 61   4.696263855574687  0.5772156643918882  0.5772156649016019
 62   4.712392887832752  0.5772156644237292  0.5772156649015958
 63   4.728265903705767  0.5772156644531214  0.5772156649015899
 64   4.743890903705767  0.5772156644802882  0.5772156649015847
 65   4.759275519090383  0.5772156645054282  0.5772156649015799
 66   4.774427034241898   0.577215664528721  0.5772156649015762
 67   4.789352407376227  0.5772156645503264  0.5772156649015726
 68   4.804058289729168  0.5772156645703888  0.5772156649015691
 69   4.818551043352357  0.5772156645890391  0.5772156649015661
 70   4.832836757638071  0.5772156646063945  0.5772156649015633
 71   4.846921264680325  0.5772156646225621  0.5772156649015612
 72   4.860810153569214  0.5772156646376377  0.5772156649015593
 73     4.8745087837062  0.5772156646517078  0.5772156649015567
 74   4.888022297219713  0.5772156646648526  0.5772156649015545
 75   4.901355630553047  0.5772156646771447  0.5772156649015531
 76   4.914513525289889  0.5772156646886492  0.5772156649015519
 77   4.927500538276902  0.5772156646994255  0.5772156649015505
 78   4.940321051097415  0.5772156647095285  0.5772156649015492
 79   4.952979278945516   0.577215664719008  0.5772156649015481
 80   4.965479278945517  0.5772156647279097  0.5772156649015472
 81   4.977824957957862  0.5772156647362753  0.5772156649015462
 82   4.990020079909081  0.5772156647441425  0.5772156649015449
 83   5.002068272680166  0.5772156647515477  0.5772156649015442
 84   5.013973034584928  0.5772156647585223  0.5772156649015433
 85   5.025737740467281   0.577215664765096  0.5772156649015425
 86   5.037365647444025  0.5772156647712965  0.5772156649015419
 87   5.048859900317588  0.5772156647771484   0.577215664901541
 88   5.060223536681224  0.5772156647826757  0.5772156649015406
 89   5.071459491737405  0.5772156647878995  0.5772156649015404
 90   5.082570602848516  0.5772156647928391  0.5772156649015399
 91   5.093559613837527  0.5772156647975133  0.5772156649015394
 92   5.104429179054918  0.5772156648019391  0.5772156649015392
 93   5.115181867226961  0.5772156648061318  0.5772156649015389
 94   5.125820165099301  0.5772156648101061  0.5772156649015384
 95   5.136346480888775  0.5772156648138752  0.5772156649015376
 96   5.146763147555442  0.5772156648174528  0.5772156649015376
 97   5.157072425905957  0.5772156648208494  0.5772156649015373
 98    5.16727650753861  0.5772156648240766  0.5772156649015374
 99   5.177377517639621   0.577215664827144  0.5772156649015374
100   5.187377517639621  0.5772156648300606   0.577215664901537
101   5.197278507738631  0.5772156648328358  0.5772156649015368
102   5.207082429307258  0.5772156648354776  0.5772156649015365
103   5.216791167171336  0.5772156648379941  0.5772156649015368
104   5.226406551786721  0.5772156648403919   0.577215664901537
105   5.235930361310531  0.5772156648426773  0.5772156649015369
  n       H_n               below gamma        above gamma
</code>
=========================================================


Concentration of a bounded random variable, with monotonically decreasing PDF

Question: Let $X$ be a non-negative random variable bounded by $X\le C^2$ and mean $\mathbb{E} X = C$ for some constant $C>1$. Moreover, we know that its PDF is monotonically decreasing in the $[0, C^2]$. Can we upper bound $P(X>\varepsilon C)$, with a quantity that only depends on $\epsilon$?
A trival bound by Markov is
$$P(X>\varepsilon C)\le \frac{E X}{\epsilon C} = \frac{1}{\varepsilon}$$
However, because the variable is bounded by $C^2$, for $\varepsilon\ge C$ we have $P(X>\varepsilon C)=0$. Supposing this value is a continuous w.r.t. $\varepsilon$, it is interesting how does the $1/\varepsilon$ bound switches to to $0$ over the range $[0,C]$.
it is pretty clear that in general settings the worst-case behavior would happen when the distribution of $X$ is bimodal, i.e., a Dirac delta at $X=0$ and another Dirac delta $X=C^2$ with values that satisfy $E X = C$. But because of monotonicity, the variable cannot be bimodal. What is the worst-case scenario in this case?
Comment: yes that's right, i mean a bound which is not based on $C$.
Comment: By "can we bound [...] regardless of $C$" do you mean find a bound independent of $C$? Because if not, since $f$ is monotonically decreasing on $[0,C^2]$ we have for any $0 (1+\epsilon C)) < f(0)C[C-1-\epsilon]$ (is a trivial bound) right?
Comment: @Henry, Yes the conclusion on $C$ is correct. Actually, you can assume it's arbitrarily large if necessary.
Actually, there is a trivial bound by Markov, which is $P(X>(1+\epsilon)C) \le 1/(1+\epsilon)$, I'm trying to see if monotonicity here could be of help or not
Comment: @Henry I actually suspect that when $\epsilon\gg 1$, the $\mathcal{O}(1/\varepsilon)$-type decay is not tight at all,. The reason is that $P(X>(1+\epsilon)C)=0$ for $\epsilon=C$ which is much less than $\frac{1}{C}$ or $\frac{1}{2C}$.  supposing that $P(X>(1+\epsilon)C)$ is continuous in the range $\varepsilon\in[0,C]$, the question is how fast does the small-$\varepsilon$ regime switch to the large-$\varepsilon$ regime
Comment: @Henry I added these to the post with some minor changes to make the question more clear
Comment: can you explain how the tag "concentration of measure" applies to this post ... not because i think it doesn't but because i want to learn more about the tag
Comment: also, what do you mean by "worst case" ... is there a more quantitative term that captures what you intend here
Answer: I think you need $C \ge 2$ for this to work at all with the requirement for a non-negative random variable with a monotonically decreasing density (i.e. a mode at $0$) bounded above by $C^2$ and with a mean of $C$.  Then I think you have
$\mathbb P(X>\varepsilon C) \le \left\{\begin{array}{lrr}
        1, & \text{for }& \varepsilon \leq 0\\
        1- \dfrac\varepsilon 2, & \text{for }& 0 \le \varepsilon \leq 1 \\
        \dfrac{1}{2\varepsilon}, & \text{for }& 1 \le \varepsilon \leq \frac C2 \\
        \dfrac{2(C-\varepsilon)}{C^2}, & \text{for }& \frac C2 \le \varepsilon \leq C \\
        0, & \text{for }& C \le \varepsilon \qquad
        \end{array}\right.$
and that this is tight if you consider a distribution which for some $0 \lt p \le 1$ has a cumulative distribution function $\mathbb P(X \le x)= (1-p) + x\frac{p^2}{2C}$ when $x \in \left(0,\frac{2C}{p}\right]$.  The optimal values of $p$ seem to be to use $p=1$ when $\varepsilon \leq 1$, $p=\frac{1}{\varepsilon}$ when $1 \le \varepsilon \leq \frac C2$, and $p=\frac{2}{C}$   when $\frac C2 \le \varepsilon $.


capture multiple column names as value in single column pandas

Question: I just want to capture the subject name (column name) as value in new column where there is some improvements in the students marks after re-evaluation.
I have the dataset before re-evaluation:

name
class
exam
maths
physics
chemistry
joining date
result

John
Grade - 10
model 1
98
78
35
2000-01-15
fail

Bob
Grade - 06
mid term
65
52
92
2001-08-16
pass

Rose
Grade - 06
model 2
91
70
54
2001-09-14
pass

Michael
Grade - 07
model 1
10
90
45
2010-01-10
fail

Now I have the dataset after re-evaluation, there are some improvements in some students marks, and there are new data on other students marks who took their exam recently,

name
class
exam
maths
physics
chemistry
joining date
result

John
Grade - 10
model 1
98
78
87
2000-01-15
pass

Bob
Grade - 06
mid term
65
91
92
2001-08-16
pass

Rose
Grade - 06
model 2
91
70
54
2001-09-14
pass

Michael
Grade - 07
model 1
100
90
45
2001-01-10
pass

Sam
Grade - 08
mid term
43
62
80
2000-08-10
pass

James
Grade - 10
model `
76
66
96
2000-09-07
pass

Henry
Grade - 09
model 1
34
91
70
2000-01-04
fail

Now, we need to concat these two datasets, and mark which row is updated, and which column got updated, so, the concatenated dataset looks like this,

name
class
exam
maths
physics
chemistry
joining date
result

John
Grade - 10
model 1
98
78
35
2000-01-15
fail

Bob
Grade - 06
mid term
65
52
92
2001-08-16
pass

Rose
Grade - 06
model 2
91
70
54
2001-09-14
pass

Michael
Grade - 07
model 1
10
90
45
2010-01-10
fail

John
Grade - 10
model 1
98
78
87
2000-01-15
pass

Bob
Grade - 06
mid term
65
91
92
2001-08-16
pass

Rose
Grade - 06
model 2
91
70
54
2001-09-14
pass

Michael
Grade - 07
model 1
100
90
45
2001-01-10
pass

Sam
Grade - 08
mid term
43
62
80
2000-08-10
pass

James
Grade - 10
model `
76
66
96
2000-09-07
pass

Henry
Grade - 09
model 1
34
91
70
2000-01-04
fail

Now, the final output should look like this, with 2 new columns, I was able to eliminate the duplicates and added the new columns any improvement, but I got stuck on adding the other new column improved subject
Expected Output:

name
class
exam
maths
physics
chemistry
anyimprovement
improvedfield

John
Grade-10
model 1
98
78
87
Yes
chemistry,result

Bob
Grade-06
mid term
65
91
92
Yes
physics

Rose
Grade-06
model 2
91
70
54
No
no improvement

Michael
Grade-07
model 1
100
90
45
Yes
maths,result,joining date

Sam
Grade-08
mid term
43
62
80
New Entry
new entry

James
Grade-10
model `
76
66
96
New Entry
new entry

Henry
Grade-09
model 1
34
91
70
New Entry
new entry

Below is the code, I used for this,
added primary key column by concatenating name, class, exam and secondary key column by concatenating maths,physics,chemistry.

dupedf = concatdf.loc[concatdf.duplicated(subset=['PrimaryKey', 'SecondaryKey'],keep=False)]

dupedf1 = concatdf.loc[concatdf.duplicated(subset=['PrimaryKey'],keep=False)]

for i,j in dupedf.iterrows():
    for k,l in dupedf1.iterrows():
        if l['PrimaryKey'] == j['PrimaryKey']:

            dupedf = dupedf.drop_duplicates(subset=['PrimaryKey','SecondaryKey'],keep='last')
            dupedf['any improvement'] = 'No'
            # dupedf['improved subject'] = ' '

        else:
            dupedf1 = dupedf1.drop_duplicates(subset=['SecondaryKey'],keep=False)

            dupedf1 = dupedf1.drop_duplicates(subset=['PrimaryKey'],keep='last')
            dupedf1['any improvement'] = 'Yes'
           # dupedf1['improved subject'] = 'column name'

in the above code, I am iterating only the rows which exists in both before & after re-evaluation datasets. iterating row by row to have fill the 2 new columns anyimprovement & improvedfield. I was able to achieve for anyimprovement column, but I need help with improvedfield column.
Comment: Kindly add the expected final output dataframe
Answer: You shouldn't use a loop with iterrows, try vectorial code instead.
Assuming df1 and df2 the two inputs:
# define ID columns
cols = ['name', 'class', 'exam']

# ensure having dates (optional)
df1['joining date'] = pd.to_datetime(df1['joining date'])
df2['joining date'] = pd.to_datetime(df2['joining date'])

# comparing the values
tmp = (df1.assign(**{'new entry': False}).set_index(cols)
          .lt(df2.set_index(cols).assign(**{'new entry': True}))
          .eval('`new entry` = ~`new entry`') # reverse flag
      )

# producing output
out = (df2.merge(tmp.mul(tmp.columns).where(tmp).stack()  # identifying columns that changed
                    .groupby(level=cols).agg(','.join)
                    .rename('improvedfield'),
                 left_on=cols, right_index=True, how='left')
          # adding anyimprovement column from the NA or new entry status of improvedfield
          .assign(anyimprovement=lambda d: np.select([d['improvedfield'].eq('new entry'),
                                                      d['improvedfield'].notna()],
                                                     ['new entry', 'yes'], 'no'
                                                    ))
          .fillna({'improvedfield': 'no improvement'})
      )

Output:
      name       class      exam  maths  physics  chemistry joining date result     improvedfield anyimprovement
0     John  Grade - 10   model 1     98       78         87   2000-01-15   pass  chemistry,result            yes
1      Bob  Grade - 06  mid term     65       91         92   2001-08-16   pass           physics            yes
2     Rose  Grade - 06   model 2     91       70         54   2001-09-14   pass    no improvement             no
3  Michael  Grade - 07   model 1    100       90         45   2001-01-10   pass      maths,result            yes
4      Sam  Grade - 08  mid term     43       62         80   2000-08-10   pass         new entry      new entry
5    James  Grade - 10   model `     76       66         96   2000-09-07   pass         new entry      new entry
6    Henry  Grade - 09   model 1     34       91         70   2000-01-04   fail         new entry      new entry


Como faço para select multiple permitir que selecione somente um item?

Question: Possuo um select:
<select multiple="multiple">
<option value="0">teste</option>
<option value="1">teste 1</option>
<option value="2">teste 2</option>
<option value="3">teste 3</option>
<option value="4">teste 4</option>
<option value="5">teste 5</option>
</select>

Nesse select gostaria de poder selecionar somente 1 item da lista, porém quero que apareça todos os itens em formato lista, por isso o uso do multiple, como poderia resolver isso?
Obs: eu nao vou poder usar a propriedade size pois o tamanho da lista é dinâmica, a lista vem do back-end.
Answer: Meu colega, você está usando o atributo multiple para poder selecionar várias opções na lista suspensa, esse é o real objetivo do atributo multiple(selecionar vários elementos). Para você poder fazer com que os elementos <option> sejam todos exibidos você pode usar o atributo size com o valor definido para 6 que é a quantidade de elementos <option> no elemento <select>, isso é para poder amostrar um número determinado de elementos visíveis na lista suspensa:
Exemplo

<select size="6">
     <option value="0">teste</option>
     <option value="1">teste 1</option>
     <option value="2">teste 2</option>
     <option value="3">teste 3</option>
     <option value="4">teste 4</option>
     <option value="5">teste 5</option>
</select>
Comment: eu dei um exemplo ao colocar 6 itens no select, eu nao vou poder usar o size pois o tamanho da lista é dinamica, a lista vem do back end.
Comment: @Stand Alone, se isso também era o que você deseja, então tinha que ter colocado isso na pergunta também.
Comment: Ok irei editar adicionando isso obrigado.


Nested functions are disabled; use f-nested functions to re-enable

Question: I am just learning Objective C and I am having great difficulty.  This is what is typed and it is giving me an error.  I typed the text that is bold.  What is wrong with it.  It gives me the nested function error right after int main(void)
#import <Foundation/Foundation.h>

int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool = [NSAutoreleasePool alloc] init];

// **#include <stdio.h>
int main(void)
int amount = 1000000;
printf("The amount in your account is $%i\n", amount);
return 0;
}**
NSLog(@"Hello, World!");
[pool drain];
return 0;
}

Answer: Your problem is that C and it's brethren do not like functions within functions (putting aside gcc extensions for now).
What you seem to be trying to do is to declare a whole new main inside your main. That's a big no-no. What I suspect is that you've cut-and-pasted an entire C program into the middle of your existing main.
Start with:
#import <Foundation/Foundation.h>
#include <stdio.h>

int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [NSAutoreleasePool alloc] init];
    int amount = 1000000;
    printf("The amount in your account is $%i\n", amount);
    NSLog(@"Hello, World!");
    [pool drain];
    return 0;
}

and work your way up from there.


How to always show poll results view instead of vote view on poll single node?

Question: The default poll runtime settings don't fit our needs. We need to have a time window with a specific start date and a specific end date. When it's too early the button and radio inputs will be hidden. I solved that by a simple hook_form_alter(). But when it's too late the results view should be displayed and I can't find out how.
All anonymous users.
I know I could hook_init() everybody to the node/%nid/results URL. Problem is, this URL gets unavailable when the user already voted. How does Drupal know if someone already voted? How can I call the clean URL alias of a node and ensure that always the results view will be displayed even when the poll settings still say the poll is open?
Answer: It's template_preprocess_node() to hook in :D
function MYTEMPLATE_preprocess_node(&$vars) {
  // check if we are in the right place
  if ($vars['type'] == 'poll') {
    if ( /* add other conditions here if you need to */ ) {
      // then unset the voting view first
      unset($vars['content']['poll_view_voting']);
      // and add the results view instead
      $node = node_load($vars['nid']);
      $vars['content']['poll_view_results'] = array('#markup' => poll_view_results($node, TRUE, FALSE));
    }
  }
}


Exception from Tracker recompute function ( Invariant Violation )

Question: Weird behavior that I can't explain:
Here's a react component that shows a stringified object - data from google account.

If I change props.user[0].services to
                      props.user[0].services.google property

I get an error:
Answer: It might be that the services property is loaded after the function call.
The way I would solve this is using lodash to get default properties when there are none. It also prevents reference errors:
const googleProps = _.get(props, 'user[0].services.google', {
    picture: 'https://some-source/default-picture.png',
    email: 'unknown'
});


using twig | replace filter in a view

Question: I'm using textbox replace in a view for an ICAL and I need to get the special characters to render. Right now calendars like Google Calendar just shows the encoded characters instead of the decoded characters. I want ' instead of this ' .
right now it's not doing anything. I've tried multiple ways to pull this off using  |render or using replace | but nothing works. It still just shows up with the encoded characters. example below
{% set special_1 = ''' %}
{% set result_1 = "'" %}
{% set special_2 = '&' %}
{% set result_2 = '&' %}

{{ title_1|replace({ (special_1): result_1, (special_2): result_2 }) }} - {{ field_sport|replace({ (special_1): result_1, (special_2): result_2 }) }}

Answer: You can write try custom Twig_Extention
In yourmodule create service
yourmodule.service.yml
services:
  custom_replace:
    class: Drupal\yourmodule\TwigExtension\Custom_Replace
    arguments: ['@renderer']
    tags:
      - { name: twig.extension }

src/TwigExtension/Custom_Replace.php
<?php

namespace Drupal\yourmodule\TwigExtension;

/**
 * Class Custom_Replace.
 */
class Custom_Replace extends \Twig_Extension {

  /**
   * {@inheritdoc}
   */
  public function getName() {
    return 'custom_replace';
  }

  public function getFunctions() {
    return [
      new \Twig_SimpleFunction(
        'custom_replace',
        [$this, 'custom_replace'],
        ['is_safe' => ['html']]
      ),
    ];
  }

  /**
   * {@inheritdoc}
   * @param $search can a array [`word1`,`work2`] or string `word`
   */
  public function custom_replace($string, $search, $replace = '') {
    return str_ireplace($search, $replace, $string);
  }
}

In template yourpage.html.twig
{{ custom_replace('word','',string) }}

Comment: I think this would work to get the view to use replace() extension but I've discovered for my problem even if I do replace and put & for example the view will still print the special characters & It's more of a core drupal issue here.

using twig | replace filter in a view


MySQL - How do I make my max_allowed_packet setting stick?

Question: I've edited my.ini in the directory where MySQL is installed and inserted
max_allowed_packet=50M under the [mysqld] section, but when I launch the MySQL command line and do:
show variables like 'max_a%'

It still shows a value of 1048576.  I've rebooted the box, but to no avail.
EDIT: I believe the issue was that MySQL was reading one of the (many) other config files (e.g., my-huge.ini).  I re-ran the MySQL config wizard and it appears to be honoring the settings in my.ini now.
(btw: Why so many configs?!?)
Answer: You're most likely confusing 'global' and client variables.  Try:
show global variables like 'max_a%';
Make sure you're specifying the option under "[mysqld]" as well.  Dominik's recommendation regarding modifying the correct config file is key, regardless.
Answer: The following page shows the different directories from which MySQL reads the configuration files:  http://dev.mysql.com/doc/refman/5.1/en/option-files.html


jQuery Accordion - enlarging the box

Question: I'm using jQuery accordion to implement the similar features which http://flavors.me have. I've designed the palette which consists of text links which leads to text box, dropdown box etc..etc. What I need is when I click any link in the accordion, the entire box should be enlarged according to the  text boxes and other items in the accordion link.
Here is my code:
http://jsfiddle.net/rkumarnirmal/GXAYa/
Could anyone just modify the code to get that feature for me?
Thanks!
Answer: If I have understood you correct, this should solve the problem?
<code>#boxtext {
font-family: Tahoma;
color: white;
word-wrap: break-word;
}
</code>
Comment: It should, but don't take my word for it

jQuery Accordion - enlarging the box


VBSCRIPT check same value in many row but in different place

Question: I need help for this problem, I have multiple rows in a text file (.txt) as the example

0001 0002 0005 0007
0004 0005 0009 0004
0004 0009 0006 0004
xxxx 0006 zzzz kkkk
xxxx zzzz ..........

I must check if the third element of each row (0005 in fist row) is the same of second element of next row (0005 in second row), I must continue to check for each row (0009 in second row with 0009 in third row)
How can i check if the value are equal ? maybe put in array ?
thanks for answer
Comment: well is possible have the same number in a row (First row 0005 0005 0005 0005 second row 0005 0005 0005 0005) how can check if the third element of first row is the same in the second element of second row ?
Comment: read - split - extract - check
Answer: Keeping an array is overkill. Just keep two variables, one which has the elements of one line in an array and one with has the elements of the next. Something like:
Option Explicit

Dim fso,f,problem, i, lineone, linetwo

Set fso = CreateObject("Scripting.FileSystemObject")
Set f = fso.OpenTextFile(WScript.Arguments.Item(0),1,true)

lineone = Split(Trim(f.ReadLine))
i = 1
problem = false

Do While f.AtEndOfStream <> True
  linetwo = Split(Trim(f.ReadLine))
  i = i+1
  If lineone(2) <> linetwo(1) Then
     MsgBox "Problem at line " & i
     problem = True
     Exit Do
  End If
  lineone = linetwo 'for next pass through the loop
Loop
f.close
If Not problem Then MsgBox "File is Okay"

When I feed the script the following file:
0001 0002 0005 0007
0004 0005 0009 0004
0004 0009 0006 0004
0004 0006 0006 0004
0004 0006 0006 0008

The script prints "File is Okay"
But when I feed it this file:
0001 0002 0005 0007
0004 0005 0009 0004
0004 0006 0006 0004
0004 0006 0006 0004
0004 0006 0006 0008

The script prints "Problem at line 3"
I didn't do any error checking to handle the case where there are fewer than two lines in the file or some of the lines lack the expected number of entries. Obviously, you should make this more robust.


Training twice on non-injective data

Question: I have a large dataset of 30000 points, but most my Ys are the same while all Xs are different. Ys are from different samples, so I had means of Y for each sample and I used means alongside Xs to train an RF. Then I used the predicted values by this RF as a new Y to train another RF that uses only Xs as input. This surprisingly resulted in a 20 percent decrease in RMSE (final predicted Y vs initial non-injective Y).
So to cap it:
X + mean of Y for samples vs Y using RF
then
X vs YpredictedByFirstRF using RF
Did I commit some kind of machine learning crime? My idea was to simulate noise by adding central tendency measures of output as input for first model.
Answer: IIUC, instead of training a random forest directly on your given data pairs $(x_i, y_i)$, you first transformed each of those pairs into another pair $(x_i, y^\prime_i)$ (using some sub-means of the $y_i$ and a random forest model $R_1$) and then you trained a second random forest model $R_2$ on this transformed dataset $\{(x_i, y^\prime_i)\}_i$. So one could say that you used "augmented data" for training the second random forest.
Training models on augmented data instead of the (inferior) original data is common practice and there is lots of literature on how to augment datasets.
However, it would of course be important to compute the RMSE of $R_2$ on a test dataset $\{(x^t_i, y^t_i)\}_i$, one that has not been used in your augmentation procedure and in training your random forest $R_2$. It is not clear from your question how and on what data you measured this 20% decrease of RMSE.
As to why your approach works: I am a surprised that this works and I don't see a reason why this should work in general, especially with such a huge improvement of 20%. Maybe it is a speciality of your data. I would suggest some variations to figure out what is going on:

Does this method also work with other data?
Do the same without the extra y-means, to make sure that it is really the $y$ means that you provide that make the difference.
Substitute other models for random forest, e.g. GBMs, neural networks, ... Can those models also be improved by your method?
Iterate the method several times. Does it keep improving?
Comment: Thanks for the feedback and introducing the terminology. I measured RMSE reduction on untransformed test data with the second model as you have outlined. Do you have any idea why RMSE is reduced when this approach is taken?

Training twice on non-injective data


Erro ao incluir biblioteca usando Linux

Question: Estou apenas tentando compilar exemplos do livro e obtenho fatal error: No such file or directory ao tentar incluir conio.h, io.h e até curses.h.
Li no SOen e foi sugerido justamente esta curses.h, a qual obtenho erro. Por que isso acontece e com qual biblioteca devo utilizar para fazer manipulação de arquivos bufferizados utilizando Linux (Fedora 23)?
Answer: conio.h e io.h são bibliotecas normalmente encontradas nos compiladores para Windows. Não fazem parte do ISO C. Portanto, é natural que outras plataformas não o incluam. A não ser que tenha ótimos motivos, nunca dependa de extensões específicas de uma plataforma.
A curses.h está disponível no Linux, mas é preciso instalá-la primeiro. Se estiver usando o Ubuntu, use o comando sudo apt-get install libncurses5-dev para instalá-la.
No Fedora, tente sudo yum install ncurses-devel.
Answer: Pará completar a resposta acima de mim, se você precisa mesmo de incluir todas essas bibliotecas siga estes passos:
Instale ncurses e faça download do ficheiro gconio.h(http://www.wence.vandermeersch.org/gconio/gconio.h).
Coloque gconio.h na sua INC_DIR ou na pasta dos seu projecto.
Por fim utilize estes includes
#include <ncurses.h>
#include <gconio.h>//ou "gconio.h" depende da sua localização
#include <asm/io.h>

Erro ao incluir biblioteca usando Linux


Why are these topologies not comparable?

Question: These topologies are on $\Bbb R$. The finite complement topology (T3) and the topology having the sets $(-\infty, a)=\lbrace x\,|\,x \lt a\rbrace$ as basis (T5).
I'm very confused because some papers say to

consider point $1$ and two open sets $\Bbb R − \lbrace 0\rbrace \in$ T3 and $(−\infty,2) ∈$ T5 containing it. Neither of these open sets contains an open set from the other topology that contains $1$.

My question is: isn't $\Bbb R - \lbrace 0\rbrace$ not in the finite complement topology since $\Bbb R$ is infinite, and if you get rid of one point, you still have an infinite set?
Comment: $\Bbb R-\{0\}$ _is_ an open set in the co-finte topology, because its complement is $\{0\}$, a finite set.
Answer: $O = \mathbb{R}\setminus \{0\}$ is in $T_3$, because its complement, $\{0\}$, is finite. $1 \in O$, but if we would have $(-\infty, a) \in T_5$ with with $1 \in (-\infty, a)\subseteq O$ then $a >1$ and so $0 \in (-\infty, a)$ as well ,so $(-\infty, a) \nsubseteq O$. So $1$ is not an interior point of $O$ in $T_5$, so $O \notin T_5$.
So $O \in T_3, O \notin T_5$.
On the other hand, $O' = (-\infty, 2) \in T_5$. Then $1 \in O'$ but if $1 \in \mathbb{R}\setminus F \subseteq O'$, we'd have $X \setminus O' = [2, \rightarrow) \subseteq \mathbb{R} \setminus O = F$, which is a contradiction,as the former is infinite, and $F$ is finite by definition of $T_3$. So $1$ is not an interior point of $O'$ in $T_3$, so $O' \notin T_3$.
So $O' \in T_5, O' \notin T_3$. So $T_3$ and $T_5$ are not comparable.


Probability and binomials choosing phones

Question: So assume we have 100 phones of some sort and that we choose 15 of them. We know that on average 6 of them are faulty and 94 are fine. What is the probability of us choosing 15 phones from the 100 so that non of them is faulty?
I tried writing the binomial function as:
$$ \frac{{{94}\choose{15}}{{6}\choose{0}}}{{{100}\choose{15}}} $$
But this is approximately =0.3 but I expect an answer of 0.77. What should I be doing to make it right?
Comment: The difference is whether we have a factory that produces tens of thousands of phones (*some amount sufficiently greater than our sample size*) that we randomly sampled from and so we might have potentially had more than six or less than six faulty phones in which case the binomial distribution is more appropriate... or if we have a factory that produced *exactly* $100$ phones and we know that *exactly* six are faulty in which case the hypergeometric distribution is more appropriate.
Comment: Read more on [Binomial Distribution](https://en.wikipedia.org/wiki/Binomial_distribution) versus the [Hypergeometric Distribution](https://en.wikipedia.org/wiki/Hypergeometric_distribution).  As for "*but I expect an answer of 0.77*", no interpretation of the problem that I can see will give an answer of $0.77$.
Comment: You have used the hypergeometric distribution, not the binomial distribution.
Comment: @callculus42, sorry I dont understand how I should change the answer
Comment: Thank you all! I will think more about this problem!
Comment: @Clone Why you expect $0.77$?
Comment: how about this way.  The first phone you pick, you have 94/100 odds.  Then the 2nd one  there are 99 phones left and 93 ok phones, so next time it is 93/99.  so it is (94/100)(93/99)(92/98)...(80/86) = (94! / 79!) / (100! / 85!) = 0.366891221
Comment: alternatively it's how many ways can you pick 15 from 94 (the number of successful choices you have) divided by how many ways can you pick 15 from 100 = (94! / (15! 79!)) / (100! / (15! 85 !)) =  (94! / (15! 79!))  (15! 85 ! / 100!) = 94! 85! / (79 ! 100!) = 0.366891221 again
Answer: Your formula would be appropriate if you actually have 100 phones, 6 of which are faulty, and if you pick 15 phones from those 100 (i.e. select phones without replacement).  However, given that the prompt talks about an 'average' of 6 out of 100 phones being faulty, you really should think about this in terms of probabilities. And note, as such, whether any one phone is faulty is independent of any other phone being faulty, and that is not the case when you pick phones from a group without replacement. So, the two approaches will get different answers.
Comment: I guess in that case, if the 100 phones are selected from the large population at random, then selecting the 6 phones from the 100 is equivalent to selecting any 6 phones from the original large population - thanks for pointing out that subtle point.


Regression on Auction Prices, Multiple Prices Randomness

Question: I'm currently building a model to predict internet auction sale prices of products in a marketplace. There are a lot of instances where a product goes for multiple prices but it's basically the same item. I have features such as category of item (clothes, kitchen, bathroom, etc.) color, weight, height, US state the item is currently in, condition of the item (good, bad, ok), and a number of other features. I have historical data dating back five years which allows me to see how much items have gone for in the past.
There are occasions when for example a bicycle might be basically the same item but there's a price variability of 100 dollars between the two items. Time of day, month, auction length help some bit but I can't seem to account for the seemingly random way that prices go up and down. One bicycle might sell for 150 dollars and the other bicycle sells for 250 dollars.
I am using Python to make the model.
Comment: Thank you @PeterFlom. That part has been removed.
Comment: Welcome to the site. Until your final paragraph, I was following along and thinking about solutions. But what has random walk to do with this?
Comment: @PeterFlom I don't honestly know much about random walk but from what i've read about stock prices, random walk assumes that stock prices can take an unpredictable path compared to previous trends. So what I was thinking random walk could be incorporated is if I have ten bikes and they went for ten different prices on a day, the random walk would be able to randomly assign integers to increase or decrease a price prediction. Again, this is my limited understanding of it and it's probably off base. It's probably not that relevant and I can remove it if it distorts the question too much.
Comment: I would delete it.
Answer: This doesn't really seem like it should pose a problem. If you run a regression with "price" as the dependent variable, then there will be variation in price, even given all the independent variables.
But that's common; I'd say it's almost universal.
Comment: Thank you Peter!


Quando cria um novo par num dicionário perde os dados anteriores?

Question: Estou criando um dict do zero, porém quando eu adiciono algum par novo, os pares antigos se perdem. Se tento imprimir 'nome' por exemplo, dá erro.
cidade = {
    'nome': 'São Paulo',
    'estado': 'São Paulo'
}

print(cidade)
print(cidade['nome'])
cidade = {'país': 'Brasil'}
#print(cidade['nome'])
print(cidade)

Tirem o comentário do penúltimo print() que vocês verão o erro acontecendo.
Answer: Você não está criando um novo par de chave e valor, você está criando um novo dicionário e guardando na mesma variável, por isso o valor anterior dela é perdido, já que não pode guardar dois valores diferentes na mesma variável.
Se deseja adicionar um elemento novo deve criar isso através da sintaxe padrão do dicionário dizendo que tem uma chave que antes não existia e guardando um valor na chave e não no dicionário. A chave é como se fosse um variável diferente, ou seja, o dicionário é um objeto cheio de variáveis.
Outra forma é usar um método que o objeto de dicionário tem por padrão e ele muda o objeto para você. Documentação.
cidade = {
    'nome': 'São Paulo',
    'estado': 'São Paulo'
}
cidade['país'] = 'Brasil'
cidade.update({'cep': '01000'})
print(cidade)

Veja funcionando no ideone. E no repl.it. Também coloquei no GitHub para referência futura.


Why are lax functors from the terminal $2$-category the same as monads?

Question: I use boldface (e.g. $\mathbf{C},\mathbf{D}$) for categories and fraktur (e.g. $\mathfrak{B}$ for bicategories.)
According to this blog post at nLab, with some of the notation changed a little:

If $1$ denotes the terminal bicategory, then a lax functor
  $\mathfrak{B} \leftarrow 1$ is the same as a monad in $\mathfrak{B}$.

Sounds cool! Unfortunately, I don't get it.
As I understand it, a monad in $\mathfrak{B}$ is an object $\mathbf{C}$ of $\mathfrak{B}$ (which is a $0$-cell), together with an endomorphism $F$ of $\mathbf{C}$ (which is a $1$-cell), together with morphisms $\eta : F \leftarrow \mathrm{id}_\mathbf{C}$ and $\mu : F \leftarrow F \circ F$ (which are $2$-cells) satisfying certain identities.
Now write $1$ for the terminal bicategory, $1_0$ for its sole object, $1_1$ for its sole morphism, and $1_2$ for its sole $2$-cell.
Let $\varphi : \mathfrak{B} \leftarrow 1$ denote a lax functor.
Then $\varphi(1_0)$ is an object of $\mathfrak{B}$. So define $\mathbf{C} = \varphi(1_0)$.
And $\varphi(1_1)$ is an endomorphism of $\mathbf{C}$. So define $F = \varphi(1_1)$.
And $\varphi(1_2)$ is... an endomorphism of $F$? Umm, what? Where do $\mu$ and $\eta$ come from? And where do the equations between $\mu$ and $\eta$ come from?
Answer: Because $1_1$ is the identity, there is a natural isomorphism $1_1 \circ 1_1 \cong 1_1$. The fact that $\varphi$ is a lax functor turns this into a natural transformation $F \circ F \to F$ which is not required to be a natural isomorphism; this is what it means to laxly preserve composition. Similarly, what it means to laxly preserve identities is that instead of $F$ being naturally isomorphic to the identity (which is what a real functor would do), there is a natural transformation $\text{id}_C \to F$ which is not required to be a natural isomorphism.
The rest of the lax functor axioms become the monad axioms; you can use this to guess what they are.


RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu, using Google Colab GPU environment

Question: I'm working with this notebook right now, using Google Colab GPU environment. When I execute the block containing the following code
with torch.no_grad():
    generated_images = vae.decode(generated_image_codes)

I got the following error:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-36-56287126db2f> in <module>
      1 with torch.no_grad():
----> 2     generated_images = vae.decode(generated_image_codes)

3 frames
/usr/local/lib/python3.8/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   2197         # remove once script supports set_grad_enabled
   2198         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2199     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   2200
   2201

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)

I tried to comment all the previous block and searched for solution in similar questions, but nothing helped to solve the problem. Can anyone give me help about this?
Answer: If you executed the code blocks sequentially, both generated_image_codes and vae should be on the same device, i.e. CPU.
generated_image_codes = torch.cat(generated_image_codes, axis=0).cpu()

and
torch.cuda.empty_cache()
vae.cpu()

To double check, you can run
print(generated_image_codes.device)
print(next(vae.parameters()).device)

Expected outputs are
torch.device("cpu")
torch.device("cpu")