What are the key ideas behind a good bassline?

Question: Hello fellow musicians,
I trying to figure out how to write a good bassline to a pop-ish song (no specific genre, but think Selfless by The Strokes and Lost in Time by The Living Tombstone). I have some good experience writing music, but this one's a first - I'm trying out a new experiment adding a guitar to a song. What I'd like to do is create a bassline that can complement the melody.
From what I understand, the baseline should follow the chord progressions that I have already established. However, I don't really understand much else. When I just jam, I can follow the chords, but how can I graduate from mindless fiddling around to a competent bassline? What are the fundamentals for creating basslines other than following the chords?
Comment: There is definitely a lot more to Bass than just playing root notes of chords, try studying some basslines from other songs (especially check out beattles)
Comment: If you're going to check out The Beatles (& you really really should) start with two opposing musical directions… "All My Loving", then go straight to "Come Together". There's about 6 years & a lifetime's experience between the two tracks.
Comment: @RishiNandhaVanchi the first to look into must be AC/DC.
Comment: This one simple trick bassists don't want you to know: mindlessly fiddling around is how you make a competent bassline.
Comment: @Mazura - I've heard many bass players doing this. Question is, will they ever live long enough to produce that 'competent bassline'? In most cases, no!
Comment: @Džuris Aye!, but said pop so maybe Beattles closer to pop than AC/DC
Comment: Okay. It's a poppy tune and it's similar to "Selfless" by The Strokes and "Lost in Time" by The Living Tombstone. Hard to put a genre on it, I don't write genre music like rock or metal - just short poppy tunes
Comment: Lots of people are saying it's complicated, but there's an over-simplification that will get you a long way. In metal, play 1, 5 or both. In a folk or country, play 1, 5, 3, 7 (in order of importance) on the beats and passing notes between those if you want to. In blues, play 1, 3, 7. The blues 7 is always flat. In jazz, play 1, 5, 3, 7, 9 if you're supporting but 3, 7, 9, 13 if you're more up front. Be careful to get the quality of those 3, 7, 9, 13 right. Leave 11 to the experts.
Answer: Do note that your question is really vague (and too close to the "opinion based" flag).
As many rules, the first rule is that there is no absolute rule.
Bass lines are fundamental (yes, implied joke): they give a reference to the listener, and that reference is usually a foundation to what the listener perceives. There's a reason for which they call it "root".
Consider a seventh chord: assuming the other voices (instruments) don't play the root note, the result will dramatically change if the bass plays the "actual" root of the chord, or it plays the third. Take a simple "major 7" chord:

the bass note is the tonic: the result is possibly a major chord
the bass note is the third: the result most certainly a minor chord
the bass playing the fifth or the seventh: result possibly undefined (more on this later)

What you also have to consider is the harmonic rhythm[1], and the bass line is an important aspect in that.
For example, in classical music, there is a concept that is known as "pedal point", which creates a lot of harmonic tension, which is usually very interesting.
You can also try "delays" or "anticipations" in the harmony [2].
There is no rule.
Just remember that the rhythm of the harmony (which is relative) can dramatically change the feeling of the music.
Start by paying attention to the bass line of any kind of music, and try to get what effects it does.
Do some experimentation.
Also, consider the psychological aspects in pop[ular] music: it should be "simple", but most importantly effective. There's plenty of "pop" music that is far from simple, but it's still emotively effective and can easily become an ear worm.
[1] My native language is Italian, and I'm not sure if that's how it's actually described in English.
[2] Considering the above note, in Italian they're known as "ritardo[ armonico]" and "anticipo".
Comment: Well, you're a programmer (I don't know what's your "day job", but I feel close to you, as we both have active profiles on SO). I've learnt that being a musician or a programmer are closely related things, even if you're not a composer/arranger/producer nor a professional programmer. The basic rule is that every rule has its *exception*: we have to try and learn (in the hard way too), then, one day, we look back at the code/music we wrote/played years before, with looks that are both of compassion and interest: we were so ignorant but still eager to learn and [re]do things, as we're now :-)
Comment: Thank you for your answer. I did debate about whether this question was too vague or not, but I think it's justifiable with the rules. Much like the basics of composition (melody, theme, accompaniment, etc) there are probably rules to basslines that I simply have not learned yet. Composing is of course subjective, but some fundamentals are objective :)
Answer: The important thing about the bass is that it's the bridge between the drums and the other instruments. How it fills that role is massively dependent on the musical style, the conventions of that style, and the song itself.
The most basic bassline of course is playing a low root note of the current chord on the "one". And honestly, sometimes that's exactly what you need. John Illsley does not much more than that in Money for Nothing, for example. The effect is to give an "insistent" feeling to the music.
The classic "1-5" bassline is very standard in rockabilly. It's pretty cheesy, but it works in context.
Typically the bass adds the passing notes to the next chord (think Love Letters by Ketty Lester). Again this can be somewhat predictable depending on the context, but it often works.
Disco needs a strong downbeat on the "one", but then may have more going on during the bar (think Superstition). Alternatively there's the "four on the floor" disco bassline which again has that insistent quality by its repetitiveness.
African music (and often reggae) may lead with a higher root note on the "one" and then go lower afterwards, inverting the usual rock/pop style of starting low and then doing higher fills during the bar.
Then there's timings. As the bridge between the drums and instruments, a lot depends on how the bass works with the drums. Disco and rock generally have the bass very tight to the drums - Blondie may be the tightest drum/bass pairing ever for this, with Call me being perhaps the finest example. Alternatively the bass may lag the drums slightly, which tends to also pull the other instruments slightly behind the beat, creating a more "lazy" feel.
And lastly there's just getting a good hook going. It has to serve the rhythm, but it also has to serve the song. This will usually either provide a jumping-off point for the rest of the instruments (You can call me Al by Paul Simon) or act as a counterpoint/foil to an instrumental hook (Sweet child of mine).
Or in the ultimate case, perhaps you don't need a bassline at all. When doves cry by Prince has no bass.
The main thing for bass and instruments though is that they must not tread on each others' toes. The higher notes on a bass and the lower notes on a guitar cross over; and of course a piano has a massive range too. EQ sculpting can help with this a bit, but really this comes down to your arrangement.
Like all things musical, there are many ways to do it! And as always, if it sounds good, it is good.
Comment: @AndrewTheProgrammer It's not totally open-ended, sure, but there's still an awful lot of options. You added the songs after I wrote my answer. For light pop, honestly you mostly just don't want to overdo it.
Comment: If you're not going follow the drummer, or if you're just making a mess, you're better off w/o a baseline. The sound engineer on *And Justice For All* (quite possibly the finest mix ever laid down) knew exactly what they were doing. And if you don't know, you're probably trying to lay down a beat, which is not your job; it's to fill w/o being noticed, unless you're playing the lead. Bassists competent enough to do that I can count on one hand, and they're basically playing lead guitar anyway, just on a bass.
Answer: In addition to the answers so far, which seem to focus on melodies, don't forget the rhythmic aspect of the instrument.  Sometimes you get more out of playing a groovy rhythm on just one of two notes than trying to sew together an intricate counterpoint melody. The bass can be percussive, slapped, tapped, hammered.  Not only in the Pfunk sense but think of Les Claypool (not really pop).  However, one thing is that the bass (or any instrument for that matter) can't conflict with the main act, singer, front man, etc.  If you want the bass highlighted that's another story, and that can work.  But everything has to gel and be supportive of the music as a whole.  I'd focus more on style and rhythm than specific melodic lines.
Look to some old tunes for ideas.  A few that come to mind (not necessarily "pop") are:

Ramble On by Led Zeppelin (great bass on that, melodic and complements the tune)

Long Distance Run Around by Yes

Can You See the Real Me by the Who

The Body Electric by Rush

Of course none of these is Pop and some of these bassist "solo" over the entire band (especially Entwhistle).  But they are examples in my mind of tunes where the bass does something unique and interesting while complementing everything.
Listening is the best lesson.
Comment: Thanks for providing me these. They may not be pop, but I'm not one to sell myself short on learning more complex stuff
Comment: Because the answer has merit and helps.  Pop borrows from other styles too, that's one way to achieve wide spread popularity.  Michael Jackson has brilliant at this.  And these songs have beautiful bass lines so why not draw from that?
Comment: Jpj is one of the most underrated bassists from that era.  If you subtract his bass from zep, it ain't zep.  Ramble On is beautiful
Answer: The bass has the following rhythmic elements

the START of a note
the END of a note
CHANGE of pitch

Whenever a bass note starts, end, or changes pitch, it is an important rhythmic transient, like a drum hit. So pay attention to those. If the bass sound has a fast release envelope, such as when playing muted notes, you can do bass notes that don't have a clear-cut end.
Harmonically, it is the bass instrument's responsibility to clarify what the lowest note i.e. bass note is. In this task, the rhythmic weight of the notes depends on which "beat" to note sits on. A note that's on a strong beat, especially the first beat, will be more strongly perceived as being the bass note. So a 1-5 alternating bass will be perceived as a root position chord, but a 5-1 alternating bass will be perceived as a second inversion chord, with the 5 in the bass.
What comes to instrumentation, the bass can be thought of as sharing some of the kick drum's role. Then again, a bass line with rapid repeating notes can provide a similar effect as a hi-hat or ride cymbal. The ending of a bass note provides a bit of the same effect as a snare drum hit.
For actual bass lines, where the bass plays note sequences, there are at least these basic styles

single-note, only focus on rhythm
alternating bass, swaying between e.g. 1 and 5
repeating octave jumps, like in disco
arpeggiating, playing chord notes, e.g. 1 - 3 - 5, used in latin styles
"riffs", repeating phrases which have a strong rhythmic and melodic dimension
repeating lines which emphasize notes harmonically, without really making a melody out of it
boogie-woogie bass, e.g. repeating C - A - Bb - B line
walking bass, where the bass walks along various sized steps and leaps, either scale or chromatic, to target tones
non-repeating melodic lines, James Jamerson style. These are at least partly improvised so that they don't repeat in exactly the same pre-written form like riffs do

I should probably add examples for all of these styles, but the OP already accepted an answer. I had this text written already so I'll rather post it than not.
Answer: The main idea should be to make the bass an independent melody but still support both the harmony and the melody. The (at least I think so) easiest bass line is the "oom-pah-pah" or "boom-chick" bass. The simplest indication is (in 4/4) to play the root of the underlying harmony on 1 and the fifth on 3. (Similarly for 3/4 but often the root and fifth alternate between measures as many 3/4 pieces change their harmony in two-measure cycles.)  To continue with the 4/4 case; one can also try a "walking" bass. It more complicated but sounds very good when done well. First, one can use the third (instead of the root) of a chord at times. This will produce a first inversion (or 63 or 6, depending on who's writing) chord. This can allow for smoother bass movement. The walking bass usually plays on each beat. Trivially (but still sounds good)one can walk from the I to the IV chord by playing notes 1,7,6,5 which puts one in position to play the 4 on the first beat of the next measure. (from the I to the V, one can go 1,2,3,4.) In 3/4 one hears 1,7,6 going I to 5 and 1,2,3 going I to IV. A foxtrot often uses alternating 1 and 5 on beats 1 and 3 for 16 measures (or 32 or whenever the second strain arrives) a switch to a 4-beat bass occurs.
Things can get more complicated (if appropriate) by chromatically altering some notes (often using 1,2,3,#4 moving from I to V). In a minor key, one by (using C minor for example) one may use things like C, Bb, Ab, G  moving from I to V and C, A, B, C, when moving from V to I. (My observation has been that the melodic minor scales are used more for bass lines than for melody lines. Lots of classical pieces even use all forms essentially at the same time.)
Another variation is to use other rhythmic patterns; this happens in Latin music. A common pattern is dotted-quarter, eighth, quarter, quarter (rumbas, boleros, tangos, and others), also dotted-quarter, dotted-quarter, quarter. Older Cuban styles tie the last quarter to the ensuing first beat dotted-quarter.
Comment: Playing notes 1, b7, 6, 5 often sounds better going from I to IV.
Comment: Another of my "I must be getting old" anecdotes;) Friend of mine, one of the best bassists I've ever heard, went from a million-a-year session job to touring with the Everly Bros. His comment, "I'm bored with all that clever sh*t, just gimme 1,5,1,5. That'll do me."
Answer: Frequency modulation and "chipping" (on-off patterns) around the intended chord.
The most basic form of bassline is a chord, maybe 2-3 notes stacked and played in each bar.
Now make it more interesting. Instead of a solid chord, have the instrument play and pause intermittently in tandem with the percussion track. In electronic communications, this concept is sometimes known as "chipping" (https://en.wikipedia.org/wiki/Code-division_multiple_access) where a signal is split into multiple timeslots per unit of data, and then the signal is "on" in certain timeslots and "off" in other timeslots in a repeating pattern.
But why stop with a chipping code? We're writing music, not a communication system. Make it smoother with amplitude modification - an extended chipping code with fractional coefficients.
Finally, who said a bassline has to be a chord? Make it a pattern. Make it interesting with a repeating note pattern built around a chord.
The best examples of a really good bassline are found in "spacesynth" music (look that up if you don't know what it is).


integer field select list with min and max value

Question: I would like to have integer field that ranges from 0 to lets say 100000 with "select list" widget. It doesn't have to have keys i just don't want to type 10000 numbers in field settings of "List integer" field. If I select integer field type it offers min and max but there is no option to have "select list" widget. Am I missing something obvious?
Comment: Maybe [this one](http://drupal.org/project/range) suits you?
Answer: It may not be the best way to do this, but it might work for you... you can set up the field as a list and leave it empty, then use a PHP script to populate it, EXAMPLE being your field name:
for ($i=0; $i<100000; $i++) {
      // Get the field info
      $info = field_info_field('field_EXAMPLE');
      // Get a reference to the values
      $values = &$info['settings']['allowed_values'];
      // Manipulate the values in some way, e.g.
      $values[$i] =  $i;
      // Save the field
      field_update_field($info);
}

Then you should be able to see the value listed under your field's values (under admin/structure/content/types/YOURAPGETYPE/fields). You can also use this in the future if you need to add more values, just remove/modify the for loop. I can't take credit for this code, I found it here on SE but just can't remember where.
Comment: clever. i was trying to avoid custom php as it seems such an obvious function to have.
Comment: took 4min to create 500 options with that script. Works but very poor performance
Answer: I went with creating these options in Excel with simple
B1=A1+1
and then copy and paste that column - works well
Answer: Where can I put that code? Is it in hook_form_alter?
This is for to turn a text field into a drop down on view.
Can be seen here for the ages from and to:
age to and from drop down needed: can be seen at www.vuxi.com


Multiplicação em Python usando recurção sem interações do tipo `while` e `for`

Question: Estou a escrever um código de uma calculadora de multiplicação que deve usar apenas operadores aritméticos, no intervalo de  -10 >= x <= 10 e deve sempre fazer as contas sob a melhor performance possível, ex.: 3x2 = 3 + 3 e não 3x2 = 2 + 2 + 2. O meu problema é que eu não posso usar de maneira alguma os comandos while e for, só posso usar os operadores + e -. A minha ideia era primeiro fazer usando while e for e depois retirá-los, mas não estou conseguindo ver como isso seria possível. Além disso, preciso dar print nos intermediários da conta, por exemplo: 3 x 10 eu teria que dar um print de algo parecido com: 10; 20; 30 e não faço ideia de como fazer isso, talvez seja algo que se abra a possibilidade ao conseguir fazer sem while e for, mas não sei. Agradeço qualquer ajuda, estou bem enrolado com isso.
def erro_numero():
    print(' ')
    print('O número deve ser igual ou estar entre -10 e 10.')

def erro_letra():
    print(' ')
    print('O valor deve ser um número inteiro.')

def erro_0():
    print(' ')
    print('Não é possível dividir por zero.')

def primeiro_numero():
    try:
        x = int(input('Digite o valor do primeiro número: '))
        if x < -10 or x > 10:
            erro_numero()
            return primeiro_numero()
        else:
            return x

    except:
        erro_letra()
        return primeiro_numero()

def segundo_numero():
    try:
        z = int(input('Digite o valor do segundo número: '))
        if z < -10 or z > 10:
            erro_numero()
            return segundo_numero()
        else:
            return z
    except:
        erro_letra()
        return segundo_numero()

def produto(p, s):
    if p == 0 or s == 0:
        return 0
    else:
        sinal = True
        if (p < 0) != (s < 0):
            sinal = False

        if p < 0:
            p = 0 - p

        if s < 0:
            s = 0 - s

        resultado = 0
        if p > s:
            for i in range(s):           #Aqui o for e a função range() precisam ser removidas
                resultado = resultado + p
        else:
            for i in range(p):           #Aqui o for e a função range() precisam ser removidas
                resultado = resultado + s

        if sinal == False:
            return 0 - resultado
        else:
            return resultado

def print_do_valor(pro):
    print('O valor do produto é:', pro)

while True:                   #Essa parte do código é só para fazer rodar
    p = primeiro_numero()
    s = segundo_numero()
    pro = produto(p, s)
    print_do_valor(pro)

Comment: é possível fazer "sem while e for" - mas não é "saudável" - não si se a intenção de quem passou o exercídio era fazer vocês exercitarem um monte de "if" e "elif" e copiar colar código - mas defintiivamente, não é uma boa forma de aprender a programar - estruturas de repetição são algo MUITO fundamental em programação - e é mito mais importantante dominar as mesmas do que copiar e colar código igual 10 vezes fazendo "if"s.
Comment: tem uma forma que exercíta sim um bom "pensar sobre programação" sem for nem while, que é simular o laço usando recursão. - se essa é a intenção pelo menos não fica um monte de repetições. Seu professor falou em funções e função chamando ela mesma?
Comment: Falou sim, ele disse que era pra fazer de maneira recursiva, tanto que boa parte do código está escrito de maneira recursiva. Como que, nesse caso, eu faria pra fazer a conta e o print dos resultados usando a recursividade?
Answer: Esclarecidas as dúvidas no comentário, dá pra escrever uma resposta que pode ser produtiva:
Poder executar tarefas de forma repetitiva é possivelmente a mais importante característica da computação digital. Por exemplo, só para "pintar" a tela onde esta resposta está sendo lida, o computador está transferindo as informações de cerca de 2 milhões de pixels (~6 milhões de bytes) entre áreas de memória que representam o buffer de vídeo, e isso 30 vezes por segundo (pelo menos). Imagine fazer isso sem repetição?
Dito isso, as estruturas de repetição que existem em todas as linguagens modernas (no caso do Python while e for), já são abstrações em cima do que a CPU realmente "faz" por baixo dos panos: tudo que ela faz são saltos condicionais - no código de máquina a CPU subtrai um valor de um número, e decide se salta para a execução continuar dentro do laço ou se continua em frente, para outro trecho do programa.
Linguagens chamadas "funcionais" - que de forma subjetiva podemos dizer que são mais próximas da codificação usada na matemática moderna, podem não ter "while" e "for", escolhendo usar a construção de "funções" - que todas as linguagens modernas também tem, e são importantíssimas, para representar também todas as repetições.
Isso não ajuda a "aprender python". Não ajuda a "aprender programação" - mas ajuda a entender "o que é possível fazer com programação" e ajuda a entender melhor funções.
Então primeira coisa: esqueça a parte da "impressão" - o que você precisa é primeiro calcular o valor da multiplicação chamando funções.  Quando já tiver o valor calculado , aí sim, você imprime só com um "print".
Então, qual a ideia de "repetição usando recursão"? Simples - uma função recebe seus parâmetros, e verifica se chegou ao fim dos cálculos que precisa fazer - se sim, retorna o valor encontrado ().  Se não, faz uma interação dos cálculos - o equivalente ao código que estaria no corpo de um 'for' ou 'while' e na última linha retorna o valor que ela mesma retorna, com os valores modificados, e indicando que falta uma interação a menos para terminar o processo.  Quando chegar na última interação, aquele "retorno" que indiquei com () vai atravessando e passando pelo "return" de todas as chamadas anteriores.
Há duas formas de escrever isso, equivalentes - uma que fica mais parecida com um "for" normal:
def multiplica(op1, contador, resultado_parcial=0):
    if contador == 0:
        return resultado_parcial
    resultado_parcial = resultado_parcial + op1
    return multiplica(op1, contador - 1, resultado_parcial)

E, refatorando um pouco, temos uma forma que fica mais "simples e elegante" e mais parecida com o uso de funções em matemática :
def multiplica(op1, op2):
    if op2 == 1:
        return op1
    return op1 + multiplica(op1, op2 - 1)

o resultado é exatamente o mesmo, e são muito parecidas, mas nessa última a operação é feita na ordem "inversa": a soma mais "interna" é feita antes, e, na hora de retornar, uma última operação é executada em cada chamada da função ao longo da cadeia. Isso é, numa multiplicacao de "4 X 3", depois da terceira chamada com "4, 1", a linha do return da chamada com "4, 2" é executada, faz a soma de "4" com o resultado interno  e retorna para a chamada mais externa "4, 3".
Em Python isso não faz nenhuma diferença - mas em algumas linguagens funcionais, quando não tem nenhuma operação a mais para ser feita com o valor do retorno da chamada recursiva (como está escrito na listagem de cima - o valor retornado mais interno só é retornado direto para quem chamou "multiplicar" pela primeira vez), a linguagem pode fazer uma otimização chamada "tail call optimization", em que todas as informações das chamadas intermediárias são "limpas" - o último return, mais interno, já devolve valor direto para quem chamou "multiplicar" de fora.
Em Python, código recursivo não fica muito otimizado - é tranquilo usar quando melhora a legibilidade, por exemplo, para buscar arquivos em árvores de diretórios, ou dados em estruturas JSON de listas e dicionários aninhados - mas cada chamada de função recursiva consome de 100 a 1000 vezes mais recursos computacionais que voltar ao início de um "for" "while", criando um objeto que se chama "Frame"  com o estado da função chamada. Então esse tipo de conta, só é bom para fins didáticos mesmo.
Dada a explicação, e como funciona a mutiplicação, você deve conseguir terminar seu exercício: tem que lidar com os operando negativos, que eu não trato acima, nem com otimizar a execução para fazer menos chamadas.


Pythagorean triplets of the form $a^2+(a+1)^2=c^2$ and the space between them

Question: I was searching for pythagorean triples where $b=a+1$, and I found using a python program I made the first 10 integer solutions:

$0^2+1^2=1^2$
$3^2+4^2=5^2$
$20^2+21^2=29^2$
$119^2+120^2=169^2$
$696^2+697^2=985^2$
$4059^2+4060^2=5741^2$
$23660^2+23661^2=33461^2$
$137903^2+137904^2=195025^2$
$803760^2+803761^2=1136689^2$
$4684659^2+4684660^2=6625109^2$

Now what's so interesting? I discovered that any $c$, divided by the previous (for example $5/1$ or $29/5$) limits to $5.828427...=\left(\frac{1}{\sqrt2-1}\right)^2=\sqrt8+3$. My question: why?
Comment: This is quite a nice observation! The explanation is closely related to the solutions of the Pell Equation. I'll let someone else write up the details, but the essential ingredients can be found in the comments of this OEIS entry: http://oeis.org/A001652
Comment: As the others have pointed out, this has to do with the _Pell equation_ $x^2-2y^2=-1$. To make the ratio you observed even more interesting, maybe you should know there is the nice continued fraction, $$\sqrt{-1+\sqrt{2}} =\frac{\sqrt{2}\,e^{-2\pi/8}}{1+\cfrac{e^{-2\pi}}{1+e^{-2\pi}+ \cfrac{e^{-4\pi}}{1+e^{-4\pi}+\cfrac{e^{-6\pi}}{1+e^{-6\pi}+\ddots}}}}$$
Comment: The difference between (n+1)^2 and n^2.  Is 2n+1 so these triplets are found by taking odd squares and setting that as the difference between squares. So the trips are (2m+1)^2,2m^2+ 2m, and c=2m^2+2m+1. So limit is m+2+1/2m over m-1.
Comment: We have $a^2+(a+1)^2=c^2$ if and only if $4a^2+4a+2=2c^2$ if and only if $(2a+1)^2-2c^2=-1$.  There is a nice formula for the solutions of the Pellian $x^2-2y^2=-1$, they are  obtained in a simple way from the odd powers of $1-\sqrt{2}$.
Comment: This question has been answered here:  https://math.stackexchange.com/questions/2830138/explain-this-convergence-among-pythag-triplets
Comment: each time you to go next line you are adding 5 times more if you divide previous you  will of course approach 5.XXXXx  for example   696 x 5.828427 = 4056.585192 , so you will always stay within the range because each time you are multiplying every result with  5.XXX
Comment: you can see wolfram result you are always within 5.XXX range 696 /119 = 5.848739495798319327731092436974789915966386554621848739495 https://www.wolframalpha.com/input/?i=696+%2F119
Answer:
I. Silver ratio

What you have discovered is the square  of the silver ratio,
$$S=1+\sqrt{2} = 2.414213\dots$$
It is a cousin of the golden ratio, $\phi=\frac{1+\sqrt{5}}{2}$, and share similar properties. Your Pythagorean triple can be expressed as,
$$\Big(\frac{b-1}{2}\Big)^2+\Big(\frac{b+1}{2}\Big)^2=c^2\tag1$$
and factoring (WA link), we get the condition,
$$b^2-2c^2=-1$$
This is a Pell equation, and given one integer solution we can find an infinite more (this link, eqn. 35,36). Since this is the negative Pell equation, we use odd powers $n$,
$$\begin{aligned}
b_n &= \frac{S^n+(-S)^{-n}}{2} = 1,\, 7,\, 41,\, 239,\dots\\
c_n &= \frac{S^n-(-S)^{-n}}{2\sqrt{2}} = 1,\, 5,\, 29,\, 169,\dots
\end{aligned}\tag2$$
The $b_n$ is always odd, so $(1)$ are integers.

II. Answer:

Note that the contribution of $(-S)^{-n} \approx (-0.4142)^n$ in $(2)$ rapidly diminishes as $n$ becomes large. Thus,
$$c_n\approx \frac{S^n}{2\sqrt{2}}$$
Taking the ratio of $c_n$ for successive odd powers $n$,
$$
\frac{c_{2m+3}}{c_{2m+1}} = \Big( \frac{S^{2m+3}}{2\sqrt{2}} \Big)\Big(\frac{2\sqrt{2}}{S^{2m+1}}\Big)= S^2=(1+\sqrt{2})^2 = 5.828427\dots
\tag3$$
and we recover your observation.

III. Neat stuff

Just like the golden ratio can be found in the pentagon, the silver ratio $S$ is in the octagon,
$\hskip2.8in$
It is also an infinitely nested radical,
$$S = 1+\sqrt{2} = 2\sqrt{\frac{1}{2^2}+\sqrt{\frac{1}{2^2}+\sqrt{\frac{1}{2^2}+\dots}}}$$
and in that nice continued fraction above using $e^{-\pi}$, as well as in a pi formula, etc, etc.
Answer: This explains why the ratios converge to the square of the silver ratio.

This shows how all of the primitive Pythagorean triples can be generated by
Pellian sequences, and therefore the ratios of consecutive hypotenuses from any of those sequences would converge to the square of the silver ratio.
All of the primitive triples can be listed by sequences with the same recursion relation, just different initial values. The initial values will only affect the coefficients in the explicit forms for those sequences. This will not affect what their ratios converge to as n goes to infinity for each sequence.
THEOREM: All of the primitive Pythagorean triples can be generated, ordered and
largely sorted without redundancy by substitution into {v^2 – u^2, 2uv, v^2 + u^2 } of consecutive pairs of terms of the Pell numbers and similar sequences formed by the same recursion relation, P(n + 2) = 2P(n + 1) + P(n), and initial values n, n + m and n – m, 3n – 2m, where n and m are positive integers, n > m, gcd(m,n) = 1,
and m is odd.
(The leg difference of triples generated from both sequences is d = 2n^2 – m^2.
And the Pell numbers can be considered the special case of a singleton sequence:
n, n + m, . . . , where n = m = 1.)
Comment: interesting references (+1). as per the comment above pls. report the essential lines.
Comment: I don't understand why it's written .  I have given a formula of connection of solutions of this equation with solutions of Pell's equation. Just rewrote my answer in a slightly different form.  That this equation can be reduced to the Pell equation. This has been known for a long time. And it is enough to open the brackets.
Comment: I'm trying to show exactly how to construct the closed form of the Pell numbers, use it to demonstrate exactly why those hypotenuses must converge to the square of the silver ratio and extend all of this to all of the Pythagorean triples.
Comment: Great original question/observation!!! Since the triples represents the rational coordinates of the unit circle ⭕ and all of the triples can be generated from sequences involving the silver ratio, this connects the silver ratio to the unit circle ⭕! That may be a big, big deal! And the fact that the sequences lead ratio of hypotenuses to converge means geometrically that we can use all of the triples to approximate the irrational-coordinate point (1/sqrt2, 1/sqrt2) AND the convergence involves the square of the silver ratio!
Answer: The values $\; 1,5,29,169,\cdots\;$ are subset of pell number as shown in the OEIS sequence A000129. You are in fact using the odd-value numbers in the sequence.
If we define Euclid's formula for generating Pythagorean triples as:
$$ \qquad A=m^2-k^2\qquad B=2mk \qquad C=m^2+k^2\qquad$$
the $(m,k)$ values needed to generate these are also Pell numbers and the $n^{th}$ triple can be generated by
\begin{equation}
 m_n= \frac{(1 + \sqrt{2})^{n+1} - (1 - \sqrt{2})^{n+1}}{2\sqrt{2}}\qquad
 k_n= \frac{(1 + \sqrt{2})^n - (1 - \sqrt{2})^n}{2\sqrt{2}}
 \end{equation}
If you plug these into $\quad C=m^2+k^2\quad $ and take the ratios between adjacent $C$-value polynomials, it should explain the result you have observed, much as ratios of consecutive Fibonacci number converge to $\phi$.
\begin{align*}
\frac{(1 + \sqrt{2})^{2} - (1 - \sqrt{2})^{2}}{2\sqrt{2}}=2 \qquad&
     \frac{(1 + \sqrt{2})^1 - (1 - \sqrt{2})^1}{2\sqrt{2}}=1 \\
&\qquad\qquad  F(2,1)=(3,4,5)\\
 \frac{(1 + \sqrt{2})^{3} - (1 - \sqrt{2})^{3}}{2\sqrt{2}}=5 \qquad &
     \frac{(1 + \sqrt{2})^2 - (1 - \sqrt{2})^2}{2\sqrt{2}}=2 \\
&\qquad\qquad  F(5,2)=(21,20,29)\\
  \frac{(1 + \sqrt{2})^{4} - (1 - \sqrt{2})^{4}}{2\sqrt{2}}=12 \qquad &
      \frac{(1 + \sqrt{2})^3 - (1 - \sqrt{2})^3}{2\sqrt{2}}=5 \\
&\qquad\qquad  F(12,5)=(119,120,169)\\
 \frac{(1 + \sqrt{2})^{5} - (1 - \sqrt{2})^{5}}{2\sqrt{2}}=29 \qquad &
     \frac{(1 + \sqrt{2})^4 - (1 - \sqrt{2})^4}{2\sqrt{2}}=12 \\
&\qquad\qquad  F(29,12)=(697,696,985)
 \end{align*}
Answer: Actually, you should write a general formula, and these issues are constantly emerging.
Solutions of the equation:  $$x^2+(x\pm{a})^2=z^2$$
Can be written using the solutions of Pell's equation: $p^2-2s^2=\pm{a}$
And have the form:
$$x=2s(p+s)$$
$$z=p^2+2ps+2s^2$$
If we introduce a replacement:
$$q=p^2+2ps+2s^2$$
$$t=2ps$$
Solutions have the form:
$$x=t^2-q^2$$
$$z=q^2+t^2$$
Solutions have the form:
$$x=2qt$$
$$z=q^2+t^2$$
It must be remembered that the number of $p,s$ can be of any sign.


Max Hard Drive Size for SATA 2

Question: Recently acquired a 1U Supermicro E3-1230 with two 3.5" drive bays, and thinking of buying two 18TB SATA-3 hard drives for it.  Before I make this investment, would anyone know if SATA-2 (3 Gbps) would support 18TB drives with their full 18TB capacity, with a motherboard BIOS date from 2011?
All SATA docs talk about the speed difference between SATA-2 (3 Gbps) and SATA-3 (6 Gbps), and little about capacity compatibility.  OS will be Linux 64-bit which is 18TB compatible, if the SATA bus supports it.  Motherboard is a Supermicro X9SCL.
Answer: That's not up to the SATA version but to the BIOS, firmware and drivers.
LBA-48 introduced in 2002 (prior to SATA 1.0) supports up to 128 PiB. So, that machine should support 16+ TiB HDDs but it still might not due to some shortcoming or bug - likely, it's not been tested.
You will not find that information in the SATA specs but rather in the system specs, forums or from support. Alternatively, you have to just try. If all fails you could simply add a more up-to-date SAS/SATA controller and hang the drives from there.

Max Hard Drive Size for SATA 2


In one note played on a piano, say C4, what can we say about the phases of the different frequencies heard?

Question: Say C4 is played.  There are the harmonics, and then other frequencies of lesser amplitude.  Based on how a string vibrates, can we describe a pattern to the phases of these frequencies?
Comment: @Artemmm -- I didn't see the original comment, but it isn't rude to offer an opinion that a question is poorly conceived in the comments. Maybe the comment was genuinely rude, but your response certainly was. For my part, I see this question as unclear, and I have flagged it as such. It seems either Unclear or Too Broad: "is there anything we can say about how the phases of these frequencies align?" I'm not sure what this means, or what would constitute an answer. You haven't clarified what you mean by "aligned" yet.
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/92663/discussion-on-question-by-artemmm-in-one-note-played-on-a-piano-say-c4-what-ca).
Answer: The first mistake here is that most notes on the piano do not consist of a single string. It would be impossible to tune the two or three strings to identical frequencies, and in fact they are intentionally tunes slightly differently, because if the tuning accuracy is too "tight" the instrument sounds "dead".
The second mistake is that because of the flexibility of the bridges on the sound board, even a single string has two different modes of vibration, one perpendicular to the soundboard and the other parallel to it, and each of these vibration modes has its own full set of harmonics, with slightly different frequencies from each other.
The third mistake is that because the piano wire is not perfectly flexible, the harmonics are not exact multiples of the fundamental frequency. The high harmonics are progressively sharper than the fundamental.
A further issue is that the rate of sound radiation from the vibration modes perpendicular to the soundboard is higher than from those parallel to it, and the sound from the "perpendicular modes" decays faster.
The combination of all those factors makes the development of the sound very complex. An over-simplified version of what happens is that the initial sound is dominated by the "perpendicular" vibration modes. They decay faster than the "parallel" modes and are vibrating at a slightly different frequency, so the two families of vibrations "beat" against each other, and at some point (typically about 0.5 to 1 sec after the start of the note) there is a minimum in the sound amplitude cause by the first beat. After that point, the "perpendicular" vibrations have decayed to a low amplitude and the sound level then rises (the so-called "blooming" of the piano tone, named by analogy with a flower bud opening) and finally decays slowly, over several seconds, or even over several minutes for low pitched notes.
To summarize, what you learned about vibrating strings in a first "sound and vibration" physics lab doesn't even begin to explain what is really going on in real musical instruments.
If you want to understand and experiment with these effects, an acoustic guitar is a better instrument to use than a piano, because (if you learn about classical guitar technique) you can control the relative amplitudes of the "perpendicular" and "parallel" vibration modes depending on how you pluck the string, and hear what effect that has on the sound. For example, this explains why "strumming" an acoustic guitar produces a very different sound from playing it with classical guitar technique, which plucks the strings away from the sound board, not from side to side.
Comment: Thank you for your answer.  Why I asked this question was more to get insight into the phases of the frequencies rather than the amplitudes.  Thanks for pointing out the error about multiple frequencies rather than one.


What is the 'exact' difference between linear and non-linear equation?

Question: I want to know whether the following series of thought is valid.
My textbook says an equation of the form $a_1 x_1 + a_2 x_2 + ...+ a_n x_n = b$ where $a_1 , ... , a_n $ and $ b$ is some constant is linear equation.
According to the textbook, the following equations are linear
$x_1 + 5x_2 - 2 x_3 =1 $,  $x_1 = -7x_2 +3x_3$, $3x+28y-1/2z=0$
and the followings are nonlinear
$x_1 + 3x_2 + x_1 x_3 =2 $ , $xyz=x+2y$, $x^{-2} + x_2 +8x_3 =5$
I think the definition textbook gave is useful for judging linearness of nonlinearness of the equations at one sight, but it doesn't click my mind because it does not match with my intuition with linearity.
I think the story should go like this... 'these are the $\textbf{algebraic differences}$ between linear and nonlinear equation and from these, we can prove that an equation should has a form $a_1 x_1 + a_2 x_2 + ...+ a_n x_n = b$ to be linear...'
So I thought a while to figure it out, and found some algebraic differences between linear and nonlinear equation. which is I think crucial to do 'elimination'.
1. If I double (or multiply constant)the variables and constant, the equation remain balanced.
e.g. from the above examples, $(2x_1)+5(2x_2)-2(2x_3)=2(1)$ but $(2x)(2y)(2z)=8xyz\neq2x+4y=(2x)+2(2y)$.
2. If you add two equations, it still holds property 1.
So the question is : how can I demonstrate that an equation which has the property 1 and 2 (which I think is linear) should has the  form $a_1 x_1 + a_2 x_2 + ...+ a_n x_n = b$ ? Thanks for reading.
Answer: Do you have some background in linear algebra? A map $f:\mathbb{R}^n \to \mathbb{R}$ is linear if $f(x+y) = f(x) + f(y)$ and $f(\alpha x) = \alpha f(x)$ for $x,y \in \mathbb{R}^n$, $\alpha \in \mathbb{R}$. I believe these are the properties you are looking for. It is easy to see (*) that any linear map $f:\mathbb{R}^n \to \mathbb{R}$ has the form $f(x_1,\ldots,x_n) = \alpha_1 x_1 + \cdots + \alpha_n x_n$ for some scalars $\alpha_i$. Thus your textbook states that a linear equation is one of the form $f(x_1,\ldots,x_n) = b$ for some linear map $f:\mathbb{R}^n \to \mathbb{R}$.

(*) If $f:\mathbb{R}^n \to \mathbb{R}$ is linear, define $\alpha_i := f(e_i)$ where $e_i = (0,\ldots,0,1,0,\ldots,0)$ ($1$ in the $i$th position) is the $i$th standard basis vector. Then $$f(x_1,\ldots,x_n) = f(x_1 e_1 + \cdots + x_n e_n) = x_1 f(e_1) +  \cdots + x_n f(e_n) = \alpha_1 x_1 + \cdots + \alpha_n x_n.$$


Unable to reach Terminal Apps and Ping to VM server 2008

Question: We have Domain Controller Group Policy Set to Disable Firewall on All computers within Domain, because we have Syamntec end point Protection enabled. The problem now is, since Windows Firewall is Disabling automatically because of GPO, WE cannot ping or reach Terminal Apps from Local LAN within Domain.
Tried starting manually the Windows firewall on VM servers (2008 Enterprise) and as expected the Ping and Terminal Apps are reached and working flawlessly, until the Domain controller pushes Disable firewall Group policy onto VM servers. Manually given exception in windows firewall for Remote Desktop Terminal Services programs, but since GPO is forcing for Firewall disable, exceptions no longer work after certain time.
Please let me know, if Creating a OU in DC giving Exceptions for two VM servers for firewall enabling solves it or any precise solution waiting for...
Please can somebody help me understand the situation and solve this ASAP
Thanks
Answer: The issue can be solved by using the SECPOL.MSC configuration. Run as admin.
We need to allow the traffic of firewall even if firewall is disabled by GPO for public, private and local domain policies.
Allow inbound and outbound traffic for Domain, Private and Public as seen in the below Screenshot. And that's it! it will now work, even if Domain policy forces to disable firewall.

Change all the Private, Domain and Public Profiles as seen above and configure to allow incoming and outgoing traffic even if firewall is disabled or enabled.
That's it.


Can't change width or height in electron

Question: I'm making a electron application and trying to make a div a certain size, but it doesn't work.
I try putting width and height, but it doesn't set the width or height to what I set as the height, I tried looking in the DevTools, but it had a warning symbol.
My CSS looks like this:
.price {
    background-color: #272525;
    color: white;
    width: 128; /* Doesn't work */
    height: 128; /* Doesn't work */
}

Answer: Its undefined because the value (128) is not defined whether it's pixels, rem.. etc.
Try changing it to
width: 128px;
height: 128px;
Answer: Add px after the number in the height and with properties.
You can use others like em, rem, %, vw, vh


Date and time column has mixed format

Question: I am having some issue in formatting the date and time. I have data file that contains date and time. Below is the sample date that represents part of my data.
data = pd.DataFrame()
data['Date'] = ['01 Jul 2014 - Qualification','30 Sep 2014 - Group Stage','17 Mar 2015 - Play Offs',' 19:00:00']
data ['ID'] = [1,2,3,4]

I created a new columns and tried to format using datetime as follow:
data['date1'] = pd.to_datetime(data.Date,errors = 'coerce')

I got all NaT in date time. I also wanted to create two new columns such as Time column and stage to represent the time and the game stage.
How can I proceed with the issue?
Comment: can you post the sample output...confused what you actually want!!
Answer: The Date column has text that is other than just date/time. You cannot convert it to datetime object as it is. You need to isolate the date/time part of the text from the rest of it. To do this, you can split on - and expand to get the Stage text and date in separate columns of a temp dataframe df_temp and then use these columns to assign & create each in your existing dataframe:
In [27]: df_temp = data['Date'].str.split('-', expand=True)

In [28]: data['date1'] = df_temp[0]

In [29]: data['stage'] = df_temp[1]

In [30]: data
Out[30]:
                          Date  ID         date1           stage
0  01 Jul 2014 - Qualification   1  01 Jul 2014    Qualification
1    30 Sep 2014 - Group Stage   2  30 Sep 2014      Group Stage
2      17 Mar 2015 - Play Offs   3  17 Mar 2015        Play Offs
3                     19:00:00   4      19:00:00            None

In [31]: data['date1'] = pd.to_datetime(data.date1,errors = 'coerce')

In [32]: data
Out[32]:
                          Date  ID      date1           stage
0  01 Jul 2014 - Qualification   1 2014-07-01   Qualification
1    30 Sep 2014 - Group Stage   2 2014-09-30     Group Stage
2      17 Mar 2015 - Play Offs   3 2015-03-17       Play Offs
3                     19:00:00   4        NaT            None

Comment: Hi amanb, Thanks for the prompt reply and sorry for my delay to reply. How about the time column? I like to create a new column that represents the time.
Comment: You can create the new column just like the others, but I could not find time specific informaton in either of the existing columns. If you have time data as a list for eg., then just do `df['Time'] = #time data`
Answer: You can use regex here with Series.str.extract:
#https://stackoverflow.com/a/47656743
pat = r'(\d+/\d+(?:/\d+)?|(?:\d+ )?(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|June?|July?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)[.,]?(?:-\d+-\d+| \d+(?:th|rd|st|nd)?,? \d+| \d+)|\d{4})'

#https://stackoverflow.com/a/46069885
pat = r'((?:\d{,2}\s)?(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]*(?:-|\.|\s|,)\s?\d{,2}[a-z]*(?:-|,|\s)?\s?\d{2,4})'

s = data['Date'].str.extract(pat, expand=False)
data['date1']  = pd.to_datetime(s, errors = 'coerce')
print (data)
                          Date  ID      date1
0  01 Jul 2014 - Qualification   1 2014-07-01
1    30 Sep 2014 - Group Stage   2 2014-09-30
2      17 Mar 2015 - Play Offs   3 2015-03-17
3                     19:00:00   4        NaT

Comment: Hi Jez, Thanks a lot for the prompt reply to my query. Sorry for the delay in reply.
How about the game stage and time column? Thanks

Date and time column has mixed format


Using Principal Component Analysis to determine variable interaction

Question: I'm characterizing a communication channel by measuring bit error rate (ber) as a function of 15 channel settings: ber = Func(settings[1:15]). The dataset is about 100,000 samples.
I wonder if I can use PCA to explore interaction of settings and their effect on the bit error rate. Specifically:

How to prepare data for PCA; do I run PCA on {ber, settings[1:15]}, or only on settings[1:15] ?
I've seen PCA used on "symmetric" data, such as images, text, etc.
What do projections of settings[1:15] on principal components tell me in this case
What do outlier datapoints in principal component space tell me.
Answer: Principal Component Analysis is an unsupervised method -- that is, you use it on independent variables, settings in your case.
Once you have your principal components, you can partition the variance of your independent variables into orthogonal projections. That is, a certain amount of variance goes into component 1 (which is a linear combination of your variables), which is orthogonal to (uncorrelated with, but see here) the other components.
Usually, one uses PCA to replace a large number of variables by a much smaller number of principal components that are (i) uncorrelated and (ii) contain most of the variance.
However, given that you have a large number of samples and only a small number of variables, and given that you have a dependent variable, and given that you want to "explore interaction of settings and their effect on the bit error rate", I don't think PCA is the right method for you. You should consider either linear modelling, linear discriminant analysis, or maybe a method like PLS (which is to PCA what linear regression is to calculation of a mean). Or even a supervised machine learning method (like SVM, random forests, PLS-DA).


NL and other languages visible, but English site not visible and showing error after translating news item

Question: I get below error when try to reach my site (see screenshot):
The editor of the site made a news item and translated it to English. Now the English (default) site can't be reached. But the other languages are still working. For example the dutch version.

Notice: Undefined offset: 5 in tb_megamenu_insert_tb_item() (line 347
  of
  /var/www/clients/client7/web17/web/sites/all/modules/tb_megamenu/tb_megamenu.functions.inc).
  Notice: Undefined index: href in include() (line 2 of
  /var/www/clients/client7/web17/web/sites/all/modules/tb_megamenu/templates/tb-megamenu-item.tpl.php).
  Notice: Undefined index: href in include() (line 2 of
  /var/www/clients/client7/web17/web/sites/all/modules/tb_megamenu/templates/tb-megamenu-item.tpl.php).
  Notice: Undefined index: options in include() (line 2 of
  /var/www/clients/client7/web17/web/sites/all/modules/tb_megamenu/templates/tb-megamenu-item.tpl.php).
  Recoverable fatal error: Argument 2 passed to url() must be of the
  type array, null given, called in
  /var/www/clients/client7/web17/web/sites/all/modules/tb_megamenu/templates/tb-megamenu-item.tpl.php
  on line 2 and defined in url() (line 2220 of
  /var/www/clients/client7/web17/web/includes/common.inc).
Answer: From the error msgs shown, it seems that you're using the TB Mega Menu module. So think of a way to get that module disabled (eg using the Dutch admin screens to disable a module, or using Drush if you have it available). That might possibly help to get rid of those errors again.
If you then also unpublish (or delete?) that news item where your problems started from, you may have a stable situation again. And from there continue your investigation to find the real cause (root) of your issue.
Doei!


É possível integrar pagamentos no meu sistema com instituições bancárias?

Question: Olá, estou com um problema: preciso fazer uma quantia em dinheiro ser depositada na conta de uma determinada pessoa, a partir do meu sistema, que eu implementei.
Preciso saber se é possível realizar pagamentos ou transferências entre contas, a partir do meu sistema. Preciso poder fazer uma transferência bancária ou depósito, informando o valor e a conta de destino. Pode ser qualquer banco.
Caso não exista essa possibilidade, existe algum tipo de carteira online (semelhante ao Neteller, por exemplo) que eu possa efetuar transferências e checar o saldo ou o extrato via API?
Grato!
Comment: Você tem que perguntar para o banco que você opera.
Answer: Sim, basta utilizar um gateway de pagamento. Faça uma pesquisa no google, e veja qual dos que existem no mercado, o que mais atende a sua necessidade. Eles vão te oferecer uma biblioteca para você integrar a api deles em seu sistema e a maioria oferece também um suporte para implementação. Porém você que vai ter que lidar com os riscos da transação, diferente de quando se usa um integrador de pagamento (paypal, moip, boacompra).
Se você não estiver pronto para assumir esses riscos, pode contratar um integrador de pagamento em vez de um gateway. Eles assumem todo o risco das transações, e outras dores de cabeça, mas cobram uma taxa de serviço mais elevada também.
O integrador não faz depósito direto em conta, o usuário teria que criar uma conta no site do integrador, e depois resgatar o dinheiro na conta dele. Se você precisa de depósito direto na conta bancária, a melhor opção é usar o gateway de pagamento.
https://pt.wikipedia.org/wiki/Gateway_de_pagamento
Comment: Podemos conversar melhor fora daqui? Tenho muitas dúvidas. firstname.lastname@example.com
Comment: Muito obrigado.
Comment: Claro. Enviei um convite pelo google hangout


Why does this array-like object behave like this?

Question: By setting the prototypal methods of an object to Array methods, the object behaves like a hybrid between an object and an array. Below is a simple example:

function Foo() {}
Foo.prototype.push = Array.prototype.push;
Foo.prototype.forEach = Array.prototype.forEach;

var foo = new Foo();

foo.push('abc');

foo.length; // = 1 as expected. But wait, why isn't foo.length undefined? How/when did this property get attached to foo?

foo[1] = 'def';

foo.length; // still = 1. But foo={0:'abc',1:'def'}, Why not =2?

foo.forEach(function(item) {
  console.log(item)
}); //shows only'abc' and not 'def'

foo.push('ghi');

foo.length; // = 2, and now foo = {0:'abc', 1:'ghi'}. So it overwrote the key=1, which means its accessing the same location, but the first approach did not change the length ( didn't become a part of the array ) why ?

foo.forEach(function(item) {
  console.log(item)
}); //now shows 'abc' and 'ghi'

Why is all this weird behavior happening, and why isn't it good to mimic Arrays like this?
Comment: Because Array in javascript is still Object under it that mimics the Array we know of from other languages. Its like your creating your own complexity and its hard to maintain. The next guy who will look at your code will be more confused.
Answer: How is the length property set?
When you invoked the Array#push, or in this case, Foo#push method. Per the ECMAScript 2015 Specification:

22.214.171.124 Array.prototype.push ( ...items )
[...]

[...]

     [...]
     d. Let len be len+1

Let setStatus be Set(O, "length", len, true).

So when you invoke the function, it automatically sets the length property if it doesn't exist, to the value len which is incremented every time you push.
Why isn't the length 2?
Now, when you directly set an index in the array, the length property is not updated. That's because arrays in JavaScript are exotic objects which internally increment length when you set properties directly. Per the specification again:

9.4.2 Array Exotic Objects
[...]
whenever an own property of an Array object is created or changed, other properties are adjusted as necessary to maintain this invariant. Specifically, whenever an own property is added whose name is an array index, the value of the length property is changed, if necessary, to be one more than the numeric value of that array index;

By definition, an exotic object is any object that overrides internal methods that all normal objects have. In this case, the array exotic object overrides the internal [[DefineOwnProperty]] method so that when you define a property of the array such as setting an index, it takes extra steps to make sure things such as the property length is updated. Your Foo constructor does not create exotic objects, thus it doesn't override the [[DefineOwnProperty]] internal method like arrays do -- and thus doesn't update length when you directly define a value at an index.
Why does push push to the previous index?
Since Foos are not exotic objects and thus don't auto-increment length when an element is added directly with foo[1] = 'def', the length remains 1 when you try to push to it the second time. If we look at Array#push again:

184.108.40.206 Array.prototype.push ( ...items )
[...]

Let len be ToLength(Get(O, "length")).

[...]

Let items be a List whose elements are, in left to right order, the arguments that were passed to this function invocation.

[...]

Repeat, while items is not empty
a. Remove the first element from items and let E be the value of the element.
b. Let setStatus be Set(O, ToString(len), E, true).

Thus, since your array length is still 1 because foo[1] = 'def' did not modify the length property, it would set the new to-be-pushed element at index 1 because the length was 1.
The same principle applies to Array#forEach. forEach depends on length to iterate through the array. Since your length is not modified when you do foo[1] = 'def' and remains 1, forEach only iterates from indices [0, 1) causing it to only log the first element. Pushing updates the length and causes it to iterate from [0, 2) and logs both elements.
Why shouldn't I do this?
It's because arrays are exotic objects. They do not get along with regular objects, in the sense that regular objects fundamentally cannot achieve the same behavior as exotic ones. Exotic objects, by definition, override the default behavior of internal methods to achieve certain behaviors needed for functionality. Arrays, in this case, have to especially handle setting indices and managing length -- done with the internal method [[DefineOwnProperty]]. With a regular object, it doesn't override [[DefineOwnProperty]] so many basic operations do not work correctly as a result -- so you shouldn't do it.
You could, though, use an exotic object such as the Proxy object to implement your own code for [[DefineOwnProperty]] like arrays to mock the behavior. Another way, as loganfsmyth mentioned, you could use ES2015 classes to extend and subclass builtin exotic object constructors correctly thus mimicking array behavior.
Comment: Just to note, you _can_ extend `Array` with a real ES6 `class Foo extends Array` because ES6 class syntax properly returns an array exotic, it just doesn't work with standard ES5 prototype extension.
Comment: @mutenka Did another edit to address `forEach` and why you shouldn't do this.
Comment: @loganfsmyth Interesting, thanks for the info! I'll add that.
Answer: foo.push('abc');

foo.length; // = 1 as expected. But wait, why isn't foo.length undefined?
            //How/when did this property get attached to foo?

The .push() method is setting the .length, as it would with a real array. It isn't bothered that .length didn't already exist and it treated it as if it were 0 - so new element added at key 0 and .length set to 1.
foo[1] = 'def';

foo.length; // still = 1. But foo={0:'abc',1:'def'}, Why not =2?

Your object isn't an array, so although adding elements with numeric indices works (though they're converted to strings, as indeed real array object indices are) it doesn't magically set the .length for you like it would with a real array. The existing .length of 1 is thus unchanged.
foo.push('ghi');
foo.length; // = 2, and now foo = {0:'abc', 1:'ghi'}. So it overwrote the key=1,
// which means its accessing the same location, but the first approach
// did not change the length ( didn't become a part of the array ) why ?

The .push() method uses the existing value of .length to decide where to add the new element. According to .length you only had one element, so the new one was pushed in at index 1 and .length was changed to 2.
Answer: push() function updates the length of array so after foo[1] = 'def', length is still 1. foo is an object and hence assigning 'def' to key 1 works but doesn't increase the length and after foo.push('ghi') length gets incremented. Also, foo.push('ghi') uses current length of foo which is 1 at the time, as index for 'ghi'.
Also before you call the push() function for the first time foo.length is actually undefined. push() function defines it for the first time and after that increments it for each call.


Autohotkey reading a csv file with million lines

Question: My requirement is to read a CSV file with around million lines by grepping particular string and displaying the output line.
Example CSV File:
Rob,school,oxford,tennis
James,school,cambridge,cricket
lucy,college,harvard,football
lily,hotel,novotel,golf
..
..
..
1 million lines.

Requirement:
When i invoke the AHK script , it prompts a user input through InputBox and takes input as James school and it should output as cambridge. Similarly input as lily hotel and it should output as novotel.
I am able to achieve the desired output through below script , but the problem is when i search for a string which is for example at 1 millionth line it takes around 5-10 mins to give me the output.
Script i wrote:
#SingleInstance, force
#Include C:\Users\mpechett\Desktop\ahk\tf.ahk

InputBox, Name, Search for Name

StringSplit, word_array, Name, %A_Space%, .  ; Omits periods.

pattern = %word_array1%,%word_array2%

Outputline = % TF_Find("C:\Users\mpechett\example.csv", "","", pattern, 1, 1)

MsgBox,%Outputline%

Please help me in improving the performance of my script.
Comment: I would [sort](https://autohotkey.com/docs/commands/Sort.htm) the contents in alphabetical order and then [split](http://filesplit.org/) this big file into several smaller files (a.csv, b.csv etc).
Comment: That is one way of doing it.
Instead of splitting files, isnt any way to improve the speed?
Answer: If You use a RAMDISK you can speedup your Search Results. I can not see what your tf.ahk Script is, but 5-10 mins is to long, 1 - the loop code in the tf.ahk File is not good 2 - or it does do everytime a search from out your HardDisk c: and not from out your ram memory.
You can Download the Freeware From Here Imdisk

The RAMDISK is a virtual Harddisk, that is placed into Your Ram Memory.
a RAMDISK is +-100x Faster then a Harddisk.

First Install IMDISK on your Windows System, then you can simple install/put/copy any Application or put/copy any csv file to the Ramdisk [example - z:\example.csv]
z:\example.csv
Rob,school,oxford,tennis
James,school,cambridge,cricket
lucy,college,harvard,football
lily,hotel,novotel,golf
..
..
..
1 million lines.

Note : With this AHK Keyboard Shortcut Macro Script, you can type - for Example : college,harvard and then it will do a search in NotePad and it will give the result Value football (This is only a example to test it out, for bigger files you will need to Chance the code a little bit and use a other Application(SpeadSheat program) that can Handle Bigger Files)
Search.ahk
; this Script works on Windows 10 system.
; You can Click key, F1 to EXIT

#SingleInstance, force

run notepad.exe z:\example.csv
WinWaitActive,example.csv, , 2

loop
{

InputBox,Clipboard,Search for Name
sleep 100
send ^{Home} ;goto Top of the Page
sleep 100
send ^f ;goto the Find box
sleep 100
send ^v ;paste Clipboard Value
sleep 100
send {enter}
sleep 1500 ;You can change this sleep codeline - How bigger the search, how larger the sleep must be.
send {esc}
sleep 100

;If you want to Select the Whole Search Line - you can use this code.
;send {Home}
;sleep 100
;send +{End}

;If you want to Select the Rigth Site of the Line - you can use this code.
send {Right}
sleep 100
send +{End}

sleep 100
send ^c ;copy the Search LineValue to Clipboard
LineValue = %Clipboard%
sleep 100
word_array := StrSplit(LineValue, ",")
sleep 100
SearchValue := word_array[1]" "       ;word_array[2]" "word_array[3]
sleep 100
MsgBox "SearchValue",%SearchValue%
}

F1::ExitApp

Answer: This is pseudocode for a indexed database type solution:
make_index() {
  global file := FileOpen( "database.csv", "r" )
  for each line in database {
    position  := file.pos
    line      := file.readline()
    values    := StrSplit(line)
    key       := make_a_unique_key(values)
    hash[key] := position
  }
  save hash to "database.index"
}

lookup(values) {
  global file, hash
  file.seek( hash[ make_a_unique_key( values )])
  return file.readline()
}

See: Manual entry for File object


Spark Streaming 2.3.1 Type Casting: String to Timestamp

Question: I am using apsche spark streaming 2.3.1, where I am receiving a stream containing a timestamp values (13:09:05.761237147) of the format "HH:mm:ss.xxxxxxxxx" as string.
I am in need to cast this string to timestamp data type.
<code>spark = SparkSession \
    .builder \
    .appName("abc") \
    .getOrCreate()

schema = StructType().add("timestamp", "string").add("object", "string").add("score", "double")

lines = spark \
    .readStream \
    .option("sep", ",") \
    .schema(schema) \
    .csv("/path/to/folder/")
</code>
Any suggestion how to convert "timestamp" to timestamp data type?
Comment: `lines.withColumn('timestamp' col('timestamp').cast('timestamp'))` ? Since you don't have any date associated, I believe it will take today's date in the timestamp.
Answer: As per the description provided in source code of TimestampType and DateTimeUtils classes, they support timestamps till microseconds precision only.
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/types/TimestampType.scala
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala


How can I PUT a new file to a node using services 3 without overwriting the existing file?

Question: Using Services 3 I have it working fine to add a file using
endpoint/file
{"uid":"1","filesize":"99999","filename":"whatever.bin","file":"BASE64_ENCODED_DATA"}

and also to add a node with that file using
endpoint/node
{"title":"testrest1","type":"dl","field_dl_file":{"und":[{"fid":582}]},"uid":"1","language":"und"}

But how can I add a new file to the node? When doing PUT
endpoint/node/999
{"field_dl_file":{"und":[{"fid":999}]},"uid":1,"language":"und"}

It just overwrites the existing file. Trying to POST gives a whole bunch of errors.
Answer: Background
Services uses drupal_execute() with the node form to save updates to nodes, which won't entirely merge updates in with existing data.
If the update doesn't set a field at all, then it won't overwrite what's there, so in a sense it does merge data in that case. For instance, in your example, you don't set "title" in your update, and therefore it won't overwrite that field.
However, if you set a field, it will overwrite whatever was there previously. So, in your example, since you are setting "field_dl_file", it will overwrite whatever was previously there with whatever you set now. As you saw, this causes any existing files to be overwritten.
Solution
Probably the best solution (and I assume the one intended by the Services team) is to load the previous value of the "field_dl_file" in your script (via a retrieve call to the REST interface), then call the update with both the new value as well as the old value. In other words, merge the old data in with the new data on the client side (inside the script that interacts with the REST interface).
Another possible solution is to manually merge your "field_dl_file" field with existing values inside node_resource.inc (line 365 or so). I assume the Services team never intended this, and it would require changing the module directly (since there aren't any hooks there), so I can't really recommend it. It would, however, ease the work needed in the client-side script, which could be a benefit depending on your implementation.
Note: I referenced Drupal 6 code, though I'm fairly sure it applies to Drupal 7 as well (that part of Services 3 didn't change as far as I know).
Comment: Thanks, its definitely a solution worth trying but as it creates more http requests from the app and I expect these file lists to get quite large I would prefer an update and create solution. I will look into it further and see if perhaps I can store the file list in android sharedpreferences (which will save unnecessary http requests). I would have thought this was quite a regular request and find it strange that there is not already a functionality for doing so.
Comment: I agree that it's strange, since it seems inefficient to force the client to do more HTTP requests. The second possible solution would probably work for you though: manually merge the new "field_dl_file" with the existing $old_node->field_dl_file before the drupal_execute() saves the node again (line 365 or so of node_resource.inc). It would require changing a contrib module though, which has obvious downsides.


Conditional Probability of Poisson Process

Question: I have seen many questions on here regarding the conditional probability of the Poisson Process however I have not seen an option for the opposite
i.e \begin{align}
P(N_T = n | N_t = m)
\end{align}
where $T > t > 0$ and $n > m > 0.$
Could someone help me understand this please?
Am I missing something obvious?
Sorry for the poor format this is my first post.
Comment: What do you mean by "an option"?
Comment: Sorry I meant an answer
Comment: An answer to which question? Are you looking for an expression that is equal to $P(N_T=n\mid N_t=m)$?
Comment: Yes sorry. I have only seen expressions equivalent to the formula given above where t>T>0 and m>n>0. For example I would like to know P(N_2 = 5 | N_1 = 2)
Answer: Due to the memorylessness of the Poisson process,
$$
P(N_T=n\mid N_t=m)=P(N_{T-t}=n-m\mid N_0=0)=\mathrm e^{-\lambda(T-t)}\frac{(\lambda(T-t))^{n-m}}{(n-m)!}\;,
$$
where $\lambda$ is the rate parameter of the Poisson process.
Comment: Thank you so much for your answer, I had a feeling it may have been the case but wasn't sure and needed confirmation so thank you for clarifying.


Removing sinusoidal oscillation from signal Python

Question:
Any suggestion for removing sinusoidal oscillation from such a signal?
Answer: Use low-pass filter. Take a look here:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.butter.html
or even better here:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.filtfilt.html
filtfilt function applies filter twice, from the beginning, and from the end, so that delay that occurs due to filtering is removed (two opposite delays eliminate each other)
you will need to adjust filter parameters, most important - cut-off frequency (set order of the filter to 5-10), try different values around 0.05-0.2.
sig = [0, 1, 0, 1, 0, 1, 0, 1, 0] # your signal
b, a = signal.butter(8, 0.1)
filtered_signal = signal.filtfilt(b, a, sig, method="gust")


Is someone is trying to access our IP-PBX

Question: We run asterisk on a freePBX distro.
The system is only up for one day.
When I logged in to asterisk from the terminal the system was generating a lot of unexpected messages just like the following.
<code> [2016-07-14 14:34:46] NOTICE[25546]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"90" <sip:email@example.com>' failed
 for '22.214.171.124:5099' (callid: email@example.com) -
No matching endpoint found

[2016-07-14 14:34:46] NOTICE[21748]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"90" <sip:firstname.lastname@example.com>' failed for
'126.96.36.199:5099' (callid: email@example.com) - No
matching endpoint found

[2016-07-14 14:34:46] NOTICE[25546]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"90" <sip:email@example.com>' failed
for '126.96.36.199:5099' (callid: email@example.com) -
No matching endpoint found

[2016-07-14 14:34:49] NOTICE[21748]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"66666" <sip:email@example.com>'
failed for '18.104.22.168:5066' (callid: email@example.com) -
No matching endpoint found

[2016-07-14 14:34:49] NOTICE[25546]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"66666" <sip:firstname.lastname@example.com>' failed
for '18.104.22.168:5066' (callid: firstname.lastname@example.com) - No
matching endpoint found

[2016-07-14 14:34:49] NOTICE[21748]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"66666"
<sip:email@example.com>' failed for '22.214.171.124:5066'
(callid: firstname.lastname@example.com) - No matching endpoint found

[2016-07-14 14:34:49] NOTICE[25546]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"66666" <sip:email@example.com>'
failed for '184.108.40.206:5066' (callid: cd692628-58a449e-
firstname.lastname@example.com) - No matching endpoint found

[2016-07-14 14:34:49] NOTICE[21748]: res_pjsip/pjsip_distributor.c:368
log_unidentified_request: Request from '"66666" <sip:firstname.lastname@example.com>'
failed for '126.96.36.199:5066' (callid: email@example.com) -
No matching endpoint found
</code>
We have not set up any of these extensions so I do not understand how they are generating the request.
At first I blocked traffic from the IP address in the notice and the notices stopped for a short while. Then the notices began again but they included a different IP address in the body of the notice. I also blocked this IP but it has had no effect on the frequency of the message appearing.
I blocked the addresses using the Sangoma responsive firewall.
I would appreciate any help in trying to understand what is going on. Thank you.
Note: in each case xx.xx.xx.xx was the same number, it was our IP address.
Comment: Your PBX is facing directly the internet ?
Comment: @yagmoth555 It is behind a firewall but all traffic on port 5060 and 5061 is forwarded to the PBX. Dose this mean it is directly facing the internet?
Comment: @bloopiebloopie It's not unusual for your PBX (or any service) to get probed by bots/hackers/whatever out there when you are internet facing...
Comment: @bloopiebloopie Yes it does
Comment: @Ryan Babchishin Why is the IP not blocked after several failed attempts to connect to the extension. Can you help me understand those notices?
Comment: @bloopiebloopie I don't use res_pjsip but it sounds like your getting call attempts to destinations that don't exist. Look at your other logs, enable debugging, or enable debugging on the console to see what these requests are for if you want to know. As long as you're facing the Internet and allowing connections from anywhere you can expect to see things like this all the time. I personally was getting attempts to call Lebanon many times a day every day. I've also had people attempt to call every internal extension... So, if this bothers you, lock it down. Use SSL, restrict connections by IP.
Comment: @Ryan Babchishin Thank you for the perspective. I am very new to internet telephony and internet security. I thought that they had to successfully connect to the PBX using a valid extension and the corresponding password before being able to make outbound calls. Is this not the case. I will enable diagnostic tools and try again.
Comment: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/42492/discussion-between-bloopiebloopie-and-ryan-babchishin).
Comment: FreePBX automatically routes these calls nowhere by default. You don't really need to worry about anything.
Answer: Since I've been answering the question in the comments I figured it was more appropriate to put it here:
It's not unusual for your PBX (or any service) to get probed by bots/hackers/whatever out there when you are internet facing... This happens with web servers, SSH, SMTP, etc... all the time. As long as you are open to the Internet without restriction, you'll keep seeing stuff like this.
I don't use res_pjsip but it sounds like your getting call attempts to destinations that don't exist. Look at your other logs, enable debugging, or enable debugging on the console to see what these requests are for if you want to know.
At one point I personally was getting attempts to call Lebanon many times a day every day (that never went through). I've also had people attempt to call every internal extension (some went through!)...
So, if this bothers you, lock it down. Use SSL, restrict connections by IP, don't use numbers for your SIP accounts, maybe use a VPN. Whatever works. If you must leave it wide open to the Internet, make sure things are configured very securely and these attempts will go nowhere. Just like attempts to compromise servers via HTTP/SMTP go nowhere when the servers are configured securely.
As for your last question, I don't know why IPs aren't getting blocked after several failed attempts. Are they supposed to?
Answer: A lot of people think a firewall is a security system for a PBX.  It's not.  If you forward SIP and RTP into your PBX through your firewall, then your firewall is only acting as a router (from the perspective of VoIP).  There is no checking of valid users, devices, geographic locations, dialing patterns, user behaviours, etc.
Setting up basic security for Asterisk is essential - there are weaknesses in Asterisk/SIP that get exploited, and even more in the configuration generators (Elastix/FreePBX/etc).  For example, a weakness in the FreePBX GUI last year allowed attackers to rewrite dialplans allowing them to call anyone, anytime, etc.  (and the corresponding $100k+ phone bills that you are responsible for).  Again, a firewalls does nothing for you in this case.  For fun, Google $400k Asterisk PBX fraudin one weekend and watch the video (an Astricon presentation)!
Take a look at Voip Info for a good intro to securing your PBX (These are facts - the reality of VoIP security - not an opinion/perspective).
If this is a small installation (which I suspect it is), install the free version of SecAst to secure your PBX.


Missing PATH_INFO on Apache 2.4 / php5.6-fpm / proxy_fcgi

Question: I'm using Debian 9 with PHP-FPM 7.0.26-2 & 5.6.32-1 from repository deb.sury.org. The same configuration is working fine on a Debian 8 server with PHP-FPM 5.6.30

I've copied the pool.d/user.conf from that server.
Both php.ini files are having cgi.fix_pathinfo=1.
The parameter is also forced using php_admin_value[cgi.fix_pathinfo] = 1.
Both servers uses this pool similarly from <VirtualHost> context with:
ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/run/php/user.sock|fcgi://localhost/home/user/public_html

Now, there's an AJAX page that needs to read path from PATH_INFO (possibly PATH_TRANSLATED). However, while these variables have correct content on the Debian 8 server:
["SCRIPT_NAME"]=>
string(14) "/path/ajax.php"
["ORIG_SCRIPT_NAME"]=>
string(25) "/path/ajax.php/para/meters"
["PATH_INFO"]=>
string(12) "/para/meters"
["PATH_TRANSLATED"]=>
string(34) "/home/user/public_html/para/meters"

They seem to work differently on the fresh Debian 9 + dub.sury.org:
["SCRIPT_NAME"]=>
string(25) "/path/ajax.php/para/meters"
["PATH_TRANSLATED"]=>
string(22) "/home/user/public_html"

And the prameters ORIG_SCRIPT_NAME and PATH_INFO are completely missing.
Answer: It was the way I used the proxy_fcgi. As explained in the examples for mod_proxy_fcgi:

You can also force a request to be handled as a reverse-proxy request,
  by creating a suitable Handler pass-through. - - The benefit of this
  form is that it allows the normal mapping of URI to filename to occur
  in the server, and the local filesystem result is passed to the
  backend. When FastCGI is configured this way, the server can
  calculate the most accurate PATH_INFO.

So I replaced my
ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/run/php/user.sock|fcgi://localhost/home/user/...

with a Handler & Worker:
    <FilesMatch "\.php$">
            SetHandler  "proxy:unix:/run/php/user.sock|fcgi://localhost"
    </FilesMatch>
    <Proxy "fcgi://localhost/" enablereuse=on max=10>
    </Proxy>

Now all the AJAX request are working fine.
BONUS: If some knows why the ProxyPassMatch was able to calculate the the PATH_INFO before on my Debian 8 / Apache 2.4.10 / PHP-FPM 5.6.30 but not with Debian 9, that would be awesome.
Comment: I'm glad, if this was helpful.
Comment: I have exactly the same problem. `PATH_INFO` was not passed after I upgraded to Debian 9 and set my Apache config with ProxyPassMatch "^/(.*\.php(/.*)?)$" "unix:/var/run/php/php7.0-fpm.sock|fcgi://localhost/var/www/"
Answer: As it is decribed on:
https://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html#env
The reason, why the ProxyPassMatch does not set the PATH_INFO is the following:

Environment Variables
In addition to the configuration directives that
  control the behaviour of mod_proxy, there are a number of environment
  variables that control the FCGI protocol provider:

proxy-fcgi-pathinfo:
  When configured via ProxyPass or ProxyPassMatch,
  mod_proxy_fcgi will not set the PATH_INFO environment variable. This
  allows the backend FCGI server to correctly determine SCRIPT_NAME and
  Script-URI and be compliant with RFC 3875 section 3.3. If instead you
  need mod_proxy_fcgi to generate a "best guess" for PATH_INFO, set this
  env-var. This is a workaround for a bug in some FCGI implementations.
  This variable can be set to multiple values to tweak at how the best
  guess is chosen (In 2.4.11 and later only):

first-dot: PATH_INFO is
  split from the slash following the first "." in the URL.
last-dot:
  PATH_INFO is split from the slash following the last "." in the URL.
full: PATH_INFO is calculated by an attempt to map the URL to the local
  filesystem.
unescape: PATH_INFO is the path component of the URL,
  unescaped / decoded.
any other value: PATH_INFO is the same as the path
  component of the URL. Originally, this was the only
  proxy-fcgi-pathinfo option.
Comment: The answer can be improved telling how to declare that environment variable.


Amazon S3 caching does not work even if I have set metadata cache control to several days

Question: I need to cache images that are hosted on Amazon S3 but there is no way to do it.
I have set the "metadata" cache control on Amazon S3 and also added the cache control in my .htaccess file.
No way to make it works.
The website is streetstylejournal.com (under dev) and I'm trying to improve the page load speed for mobile.
This one of the images that has expiration set on S3
https://s3.eu-west-2.amazonaws.com/streetstylejournal/homepage/london-fashion-week-ss19-street-style.jpg
Thank you in advance!
Answer: Your header value is incorrect, so it would have no effect.
Cache-Control: 1728000

This should be...
Cache-Control: max-age=1728000

Also note that S3 completely ignores .htaccess files.
See also https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control.
Comment: Hi Michael, thank you for your answer. I have added the tag
Comment: That shouldn't be necessary if you correct the cache-control settings on the objects in the bucket.


How to find combined SSE?

Question: For example, if we know that for a sample includes 107 income households, the SSE is 10.8. And for another sample including 98 income households, the SSE is 9.2. How to find the combined SSE for the all 225 samples?
Comment: From the sample sizes, $R^2$, and $s_U^2$, you should be able to find the variance of the monthly expenditures.  From the additional information in (b) you should now be able to carry out an analysis of variance for the INCOME variable.
Comment: I am confused.. Why are you all talking about variance? Isn't F test only about SSR and n and k?
Comment: An $F$ test, by definition, compares variances.  The variances to compare must somehow reflect the difference in mean monthly expenditures between the low and high groups.  That can be revealed by comparing the residual variances to the total variance.  (That's why this is called "Analysis of Variance.")  One way or another, however you express it algebraically, your test *must* come down to such a comparison.  I believe that the point of this series of questions may be to help you appreciate that fact.
Answer: You need to look at the Chow statistic:
$$
F=\frac{SSR_p-(SSR_1+SSR_2)}{SSR_1+SSR_2} \cdot \frac{n-2(k+1)}{k+1}
$$
Where SSR is the sum of squared residuals (or errors if you prefer). K is the number of explanatory variables, n is the total sample. The subscript p refers to the model where you pool all observations, while 1 and 2 refer to the groups number.
The trick about this question is to realize that you can work out $SSR_p$ from variance estimate in (a). Since the variance follows from:
$$
\sum_{i=1}^n \hat{u}_i^2 / (n-k-1) = s^2_u
$$
Where n and k has the same definition as above, and the sum of squared residuals are given by $\sum_{i=1}^n \hat{u}_i^2$.
The Chow test directly compares if there is difference in all the estimated parameters between the groups, which is what the question in (b) asked you verify.
Comment: But all we know is SSE, and what is SRR?
Comment: SSE = sum of squared errors = SSR = sum of squared residuals. They are the same thing
Comment: They are not? SST= SSR+ SSE. That's what I learnt.
Comment: And what is $SSR_p$ in your answer?
Comment: See the update...
Comment: So $SSR_p$ is the sum of all of the estimated error listed in the pic? And $SSR_1+SSR_2=10.8+9.2=20$?
Comment: Then F is just equal to 0?
Comment: This doesn't seem to make sense to me...
Comment: Whoops sorry, $SSR_p$ should be worked out from the model in (a). Please ignore my comments above, you easily do this since you know the variance of the residuals. Will update the answer later to reflect what is right
Comment: Thank you very much.. but I still don't know how to approach it.
Comment: How to work out $SSR_p$ knowing the variance?
Comment: Did you see the update?
Answer: This is not a meaningful practice to calculate a combined SSE in my view- What are you trying to get out of that? You need to combine both sample first and fit one linear regression to the whole data space and then calculate the overall SSE, which accounts for the total error of your regression line to the fitted data.
Comment: Because I need to know the SSE to apply F test. And I was given the data above..
Comment: I've inserted the pic to the original question. Part b, that's the only way I can think of for solving part b.
Answer: In this context, the F-test is used to test whether the ratio of two SSE measures is equal to 1, in other words, whether the two SSE measures are equal:
$F_{(n_1-1, n_2-1)} = \frac{MSE_1}{MSE_2}$
Where $F$ is from the $F$-distribution with $n_1-1$ and $n_2-1$ degrees of freedom.
However, I don't think it is meaningful to compare $SSE_{low}$ and $SSE_{high}$ in such a way, because there would be little to learn from it and it doesn't answer the question. Usually in ANOVAs the $SSE_{variable}$ is compared to the $RSS$ (Residual Sum of Squares), but it is not immediately obvious where to find those in this problem.
Comment: The $R^2$ is given, and from that, we can learn the ratio of SST and SSR, but that's it.. I still don't know how to solve it and how to connect what I've known to SSE.
Comment: The given $R^2$ is from the regression model, but in question b they turn it into an ancova by discretizing the income variable. I am not sure whether you can still use the $R^2$ in that case.
Comment: So do you have any idea how to solve part b?
Comment: I am still stuck and lost...

How to find combined SSE?


Currency conversion confusion

Question: I buy 1 unit of stock at USD=55.48 when USD-INR=66.40, spending INR=3684.02
I sell 1 unit of stock at USD=54.60 when USD-INR=65.44, gettingbk  INR=3573.13
My Loss in USD=0.88 and in INR=110.89 giving USD-INR=126 instead of expected value of 65.44.
What is the mistake here?
Comment: The buy was $55.46\times66.40=3683.872.$

The sale was $54.60\times65.44=3573.024$

The difference is $110.848$.   I know that's not the big problem but the little math may help.
Answer: The problem lies in that the loss of 110.89 in INR is not entirely due to the stock price decline, it also includes a portion due to the currency appreciation. You need to remove that part before implying the exchange rate.
The part of the loss due to the currency change is
$$\frac{66.40-65.44}{66.40}\times 3684.02=53.36$$

Currency conversion confusion


developing rhythmical phrasing

Question: Recently i've become more and more interested in phrasing for melodic instruments, and how one could develop a strong sense for making interesting arrangements, particularly when you work with three or more voicings.
Comparing with theories of polyphonic harmony, would you say there is a similarly systematic theory for rhythmical phrasing (for non percussive instruments) and how they fit together, or is it one of those things that only can be developed through listening and practice?
Comment: Does this answer your question? [What is rhythmic counterpoint?](https://music.stackexchange.com/questions/105561/what-is-rhythmic-counterpoint)
Answer: Comparing with theories of polyphonic harmony, would you say there is an similarly systematic theory for rhythmical phrasing
In general it seems difficult to divide things with your bright line - harmony and rhythm are intrinsically bound to one another. How can you apply harmony correctly if you don't know the timing and rhythm of a piece of music? How can you study more than simple rhythms that aren't purely percussive, if you're not playing melodies with harmonies?
Having said, that I believe that your question can be answered by taking a look at this syllabus and other resources on the Berklee Site. Here's a good place to start:
Berkee College of Music -Jazz Arranging

Syllabus Lesson 1: Rhythm Section and One Solo Instrument: Smooth Jazz
  Jazz Arranging: Understanding the Writing (Arranging) Process Focus
  Levels, Melodic, Rhythmic, and Harmonic Functions Comping Functions
  Writing for the Drum Set Arrangement Considerations in the Smooth
  Jazz-Funk Style Front-Line Solo Instrument Characteristics
Lesson 2: Bebop and Hard Bop Bop and Hard Bop Jazz Arrangement
  Analysis Two-Part Writing Techniques Voicings and Intervals
  Introductions/Endings
Lesson 3: Solis and Working with Vocalists Historical Perspective
  Two-Horns and use of Pads Punches and Counter Melodies Vocal Ranges
  Melodic Manipulation Writing Solis
Lesson 4: Big Band Historical Combinations Primary, Secondary, and
  Tertiary Focus Levels Cross Sectional Possibilities Open- and
  Close-Position Chord Voicings Spread Voicings Upper Structure Triad
  Voicings Unison Arrangements
Lesson 5: Writing for Large Jazz Ensemble Choosing the Soloists Solo
  Feature Combo Within the Big Band Open Solo Sections Background
  Figures
Lesson 6: Writing for Large Jazz Ensemble (Part 2) The Saxophone Soli
  Writing the Soli Coupling The Sax Section Endings
Lesson 7: Writing for Three Horns and Rhythm Section History of the
  Three-Horn Front Line Unison-Octave Splits Independent Lead with
  Two-Voice Background Three-Part Harmonization The Harmonization
  Process Highlighting the Melody Cluster Voicing Secondary Focus Beyond
  the Head
Lesson 8: Three Horns and Rhythm Section; Funk and Fusion Modal
  Characteristics The Music of The Crusaders The Music of The Brecker
  Brothers Role Model Composition
Lesson 9: Writing for Five Horns 5-Part Voicings Close Position Double
  Lead Voicing Guidelines Substitute Double Lead 5-Part Passing Chords
  Latin Influences: Brazilian Latin Influences: Afro-Cuban
Lesson 10: Final Project: Large Ensemble Arrangement Arrangement
  Planning Hooks Working With Clients Conceptualizing the Arrangement
  The Music of Sammy Nestico The Music of Rob McConnell The Music of
  Thad Jones "Variable" Instrumentation
Lesson 11: Final Project: Large Ensemble Arrangement (Part 2) Building
  to a Climax The Music of Mark Harris Large Ensemble Arrangement
  Analysis The Music of Bob Mintzer
Lesson 12: Large Ensemble Arrangement (Part 3) Treating the
  Recapitulation Score Analysis Color Reharmonization The Music of Bob
  Brookmeyer The Music of Maria Schneider Conceptualizing the Whole
  Arrangement
Requirements Prerequisites Completion of Arranging: Advanced Horn
  Writing or equivalent knowledge and experience in basic arranging
  concepts, such as instrument ranges, chord spelling, voicing, and
  harmonization is required. You should also know modes and related
  chord scales. Knowledge of basic reharmonization techniques is
  helpful, but not required. Recommended experience in a DAW of choice.
Answer: The short answer is that it is one of those things that is normally developed through listening and practice.
Experience record producers and recording engineers will, however, stress that there are specific, separate sound frequencies that must be identified (more so than rhythmic phrasing) and layered accordingly in the mix. Vocals, guitar leads, or certain keyboard frequencies that are very close to each other will nearly cancel each other out in the end, if not separated stereophonically, or massaged with EQ.
To your original question, some say that recording artist Paul Simon brought the expression 'world music' to the Americas with the release of his 1980's album Graceland. While traveling around the globe earlier the year before recording that LP, he was exposed to exotic African rhythmic phrasing, textures, and odd time signatures. It was only by this exposure that the artist was able to open up his mind to such interesting arrangements, and it's doubtful that a folk artist (originally) would have dreamed up such rhythmic phrasing and layering on his own.
Comment: Jazz musicians were working with 'world music' by the 1940's. Dizzy Gillespie:[Dizzy-Afro-Cuban music](https://en.wikipedia.org/wiki/Dizzy_Gillespie#Afro-Cuban_music) In the late 1940s, Gillespie was also involved in ...Afro-Cuban music, bringing Afro-Latin American music and elements to greater prominence in jazz and even pop music.... Gillespie was introduced to Chano Pozo in 1947 by Mario Bauza, a Latin jazz trumpet player. Chano Pozo became Gillespie's conga drummer for his band. Gillespie also worked with Mario Bauza ... Pop music and Paul Simon? Late arrivals to the scene.
Comment: Almost all classical composers of note - people like Hadyn, Beethoven, Brahms.. have been culling themes and melodies from their own folk and indingenious traditions - dance music, popular tunes, church hymns... as long as there have been composers. See [**_What are some influences from external cultures on the development of Western classical music?_**](https://music.stackexchange.com/questions/4584/what-are-some-examples-influences-from-external-cultures-on-the-development-of-w/59339#59339) - Pop/rock claim to be "progressive", but in musical terms they are very conservative, narrow genres.

developing rhythmical phrasing


Accessing all Advanced Custom Field checkbox values

Question: Is it possible to access all values for an ACF Checkbox field available in the 'Choices' field settings?

These values are being used to populate a filter dropdown, so therefore simply associating these values with a singular $post leads to inconsistent results.
i.e. my current approach only works when there are posts associated with the custom field to populate the dropdown:
<code>$courses_fieldObj = get_field_object('course_check');

$courses = $courses_fieldObj['choices'];
foreach ($courses as $value => $label): ?>
</code>
The 'options' argument looked promising, however, this doesn't seem to make available the 'Choices' values.
Solutions outside of ACF's api (such as WPDB queries suggested here) also look promising, however, since ACF stores these vals in a serialised array, I am hoping for a simpler solution.
Many thanks in advance!
Comment: `the_field('yourchoicefield');` this returns all the values with a comma. so then you can simply split on the comma
Comment: @Mederic thanks for your response.

Have tested using _get_field_ in the past (which is believe is much the same as _the_field_, except it stores as opposed to echoing out the values).

By itself, the field seems to only reference the checkbox values for the posts present on the page, as opposed to all of the available values in the field 'Choices' setting. When passed the 'options' argument, the object doesn't return any values, i.e:
`$courses = get_field('course_check', 'options');`

Am I missing something here?
Comment: The function get_field if i'm not mistaken will work only in the current post. you want to get it for every possible post? is that what you mean? Im sorry I dont really get what you want to do.
Comment: Sorry, I'll try to clarify: I am trying to use the fields of the ACF checkbox to populate a dropdown field for a post filter. Having the dropdown being populated based on one post's fields leads to inconsistent results (as say a post might only be associated with one value of the checkbox, leading to only one option being present in the select dropdown). What I want to do is access the values from the field's 'Choices' setting, so I can get all of the potential values of the checkbox, regardless of which posts are being displayed. Hope that clarifies things.
Answer: Was not able to solve this issue. I did, however, work around it by instead creating a custom taxonomy (which really suited my use case better), allowing me easy access to all course terms via the get_terms function, i.e:
<code>$args = array( 'taxonomy' => 'courses_tax' );
$courses = get_terms( $args );
</code>
I can still see scenarios where the need to access all ACF checkbox values outside of a $post context will be required - will leave this question unsolved as this answer doesn't directly address the issue at hand.


In Python, what is the difference between the 'urllib library' and the 'socket library'?

Question: I'm learning network programming basics, and there is a question I don't know how to answer in my assignment.
How is 'urllib library' different from 'socket library' in Python?
How can I explain it clearly?
Answer: urllib is much higher level.
socket provides low-level access to many different kinds of sockets, which you can use to communicate over any port and protocol.  For example, you could use it for email, SSH, remote desktop, etc.  You can also use it for to listen on a port (for a server).  Almost all Python networking libraries, including urllib, use socket somehow.
urllib is specialized to particular uses of sockets, namely the client side of the HTTP (and optionally TLS) and FTP protocols, typically (but not always) using ports 80, 443, or 21.
Comment: Additionally, `urllib` is client-only, while `socket` supports clients, servers, and whatever else.  It's easier to just say that all the Internet-facing stuff you do in Python probably goes through `socket`.
Comment: @Mike, thanks. Those are both good points. I'll update my answer.
Comment: Wow, Thank you Mathew and Mike very much.
Answer: In my basic mindset, urllib require you less code line and more consistent to attach to the server. In fact, socket may take you much more time to access the internet and require you more code to process through your code.
Comment: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).