Is the set of primes at even positions a Large Set?

Question: If we start numbering the prime numbers from 2, and we consider the subset of prime numbers at even positions, is it known is that set is large or not?
A subset of natural numbers is called "large" if the sum of reciprocals of its elements diverges. For example it is well known that the set of natural numbers and set of prime numbers are large sets. Surprisingly I could not find any reference to whether the set of primes at alternating positions is large or not.
I tried using Bertrands Postulate but failed. I intuitively believe the set primes at odd positions must be large as the but I cant seem to be able to prove it rigorously.
Any guidance/help would be appreciated. I am also looking for amateurish resources for studies on large subsets.
Answer: This follows immediately from the fact that the set of primes is large. More generally, if $A=\{a_1<a_2<a_3<...\}$ is large, then so is $\{a_1<a_3<a_5<...\}$. The same holds for even positions (just think about $A\setminus\{a_1\}$) or indeed any arithmetic sequence of positions.
This is just because we have ${1\over a_{2n-1}}>{1\over a_{2n}}$. So we get $${1\over a_1}+\color{red}{1\over a_1}+{1\over a_3}+\color{red}{1\over a_3}+...>{1\over a_1}+\color{red}{1\over a_2}+{1\over a_3}+\color{red}{1\over a_4}+...$$ or more snappily $$\sum_{n\in\mathbb{N}} {2\over a_{2n-1}}>\sum_{n\in\mathbb{N}}{1\over a_n}.$$ If $A$ is large then the right hand side diverges, so the left hand side also diverges. Now just divide by $2$.
Comment: Thanks but what about the primes at even position?
Comment: @N.S.JOHN No different - just get rid of the first prime. This is what the "and so on" refers to - this trick works for **any arithmetic progression**. E.g. $$\sum_{n\in\mathbb{N}}{1\over a_{17n+307}}$$ diverges. I've edited to clarify this.
Comment: So can we generalise this fact about arithmetic progressions to all Large sets?
Comment: @N.S.JOHN Note that I wrote my answer in terms of arbitrary large sets, not primes specifically (see the sentence beginning "more generally"). It's fully general.
Answer: You know that $\sum_{k=1}^{\infty}\frac1{p_k}=\infty$. By Cauchy condensation, we have that $\sum_{k=1}^\infty \frac{2^k}{p_{2^k}}=\infty$ as well. But we know that $$\frac{2^k}{p_{2^k}}=4\cdot 2^{k-2}\frac1{p_{2^k}}\le 4\sum\limits_{2^{k-1}< j \le 2^k\\ j\text{ even}}\frac1{p_{2j}}$$
And therefore $$\infty=\frac14\sum_{k=1}^\infty \frac{2^k}{p_{2^k}}\le\sum_{k=1}^\infty \frac1{p_{2k}}$$
Comment: Unless I'm missing something, invoking Cauchy condensation is massive overkill here - see my answer.
Comment: @NoahSchweber You are certainly right. It was really just the first idea that came to mind.


python 3 programming lottery number generator

Question: am writing a code which like the lottery should produce a list of 6 numbers like this (20,45,11,16,09, + 12)
numbers have to be from 1 to 50
How do I do it so that it just gives me the last line all 5 randomly chosen values plus the additional number all within one list:
import random
a = list(range(1,50))

b = random.randint(1, 20)

temp = []

for i in range(5):

temp.append(random.choice(a))

print(temp, "+", b) code here

output:

[8] + 12

[8, 30] + 12

[8, 30, 42] + 12

[8, 30, 42, 21] + 12

**[8, 30, 42, 21, 14] + 12**

Comment: Before I can answer your question, I do have a few questions on my own:
 1. are the numbers randomly drawn without replacement?
 2. It seems that the drawn numbers and the additional number come from two different sets, is that correct?
 3. Is your desired result [N1, N2, N3, N4, N5, Additional]? I.e. is your question how to add a value to a list in python?
Answer: I am not familiar with how lottery works. I assume that all numbers, including the additional number, are drawn from the same set without replacement. If that is true, this code should do the trick for you:
import random
numbers = list(range(1, 51)) # include 50
random.shuffle(numbers)
print(numbers[:6]) # pick the first 6 numbers (5 + 1 additional number) in the randomize list

If your additional number comes from a separate set, do this:
import random
numbers = list(range(1, 51)) # include 50
random.shuffle(numbers)
# pick the first 5 from the randomized list:
picks = numbers[:5]
# select from remaining numbers those that are less than or equal to 20:
picks.append(random.choice([n for n in numbers[5:] if n <= 20]))
print(picks)

Comment: thank you but how do I make it so that the number from the additional list is never the same as the number form the first list eg:[11, 21, 18, 36, 10, 10]
Comment: Please accept my answer then (click the gray check mark to the left of my answer). Thank you
Answer: Expending on the previous answer based on your request to exclude previously picked numbers while generating the last number.
import random

numbers = list(range(1, 51)) # create a list containing 1 to 50
random.shuffle(numbers) # randomise the list

picks = numbers[:5] # pick the first 5 numbers from the randomised list

# select the last number from the remaining numbers in the list
picks.append(random.choice([n for n in numbers if n not in picks]))
print(picks)


Why is O(n/2 + 5 log n) O(log n) and not O(n log n)?

Question: For n/2 + 5 log n, I would of thought the lower order terms of 5 and 2 would be dropped, thus leaving n log n
Where am I going wrong?
Edit:
Thank you, I believe I can now correct my mistake:
O(n/2 + 5 log n) = O(n/2 + log n) = O(n + log n) = O(n)
n/2 + 5 log n <= 2n, for all n >= 1 (c = 2, n0=1)
Answer: Let us define the function f as follows for n >= 1:
f(n) = n/2 + 5*log(n)

This function is not O(log n); it grows more quickly than that. To show that, we can show that for any constant c > 0, there is a choice of n0 such that for n > n0, f(n) > c * log(n). For 0 < c <= 5, this is trivial, since f(n) > [5log(n)] by definition. For c > 5, we get
    n/2 + 5*log(n) > c*log(n)
<=> n/2 > (c - 5)*log(n)
<=> (1/(2(c - 5))*n/log(n) > 1

We can now note that the expression on the LHS is monotonically increasing for n > 1 and find the limit as n grows without bound using l'Hopital:
  lim(n->infinity) (1/(2(c - 5))*n/log(n)
= (1/(2(c - 5))* lim(n->infinity) n/log(n)
= (1/(2(c - 5))* lim(n->infinity) 1/(1/n)
= (1/(2(c - 5))* lim(n->infinity) n
-> infinity

Using l'Hopital we find there is no limit as n grows without bound; the value of the LHS grows without bound as well. Because the LHS is monotonically increasing and grows without bound, there must be an n0 after which the value of the LHS exceeds the value 1, as required.
This all proves that f is not O(log n).
It is true that f is O(n log n). This is not hard to show at all: choose c = (5+1/2), and it is obvious that
f(n) = n/2 + 5log(n) <= nlog(n)/2 + 5nlog(n) = (5+1/2)nlog(n) for all n.

However, this is not the best bound we can get for your function. Your function is actually O(n) as well. Choosing the same value for c as before, we need only notice that n > log(n) for all n >= 1, so
f(n) = n/2 + 5log(n) <= n/2 + 5n = (5+1/2)n

So, f is also O(n). We can show that f(n) is Omega(n) which proves it is also Theta(n). That is left as an exercise but is not difficult to do either. Hint: what if you choose c = 1/2?
Comment: Great, thank you. I think I can now show it's O(n) (updated question).
Answer: It's neither O(log n) nor O(n*log n). It'll be O(n) because for large value of n log n is much smaller than n hence it'll be dropped.
Comment: Technically, it is both O(n) and O(n log n)... and O(n!) and O(2^n) and so on.
Comment: Hi Atin yes my answer is incomplete - I'll remove (and let you know here) thanks - also upvote for your answer !
Answer: It's neither O(log n) nor O(n*log n). It'll be O(n) because for larger values of n log(n) is much smaller than n hence it'll be dropped.

Consider n=10000, now 5log(n) i.e 5*log(10000)=46(apprx) which is less than n/2(= 5000).
Comment: Technically, it is both O(n) and O(n log n)... and O(n!) and O(2^n) and so on.
Comment: @Patrick87 Agreed, but I answered considering the nearest possible upper bound.
Comment: I agree, and since a tight bound is possible here, it makes sense to go for that... just wanted to point out the technical nuance in the definition of O, not criticizing the spirit of this answer.


Entity Framework Reverse POCO Generator - Custom Navigation Property

Question: I'm currently working with a legacy database that's missing a relationship between two tables.
Basically, I have a table of 'Items' and a table of 'Vendors' with their own respective primary keys.
However, the 'Items' table contains a field 'VendorNumber' that is used to map to the 'Vendors' table
.
This relationship is not enforced in the database, but I'd still like to use 'Vendors' as a navigation property from the 'Items'.
I'm using the 'Reverse POCO Generator', but I'm unable to figure out how to 'force' it to generate the navigation property.
I've tried making another 'ItemConfiguration' partial class that includes the property, but as it's set in the 'ItemConfiguration' constructor, I'm unable to achieve the desired functionality.
Any assistance/insight would be greatly appreciated!
Comment: You don't need to use 'reverse POCO generator', just use Fluent API to setup the relationship (https://msdn.microsoft.com/en-us/library/jj591620(v=vs.113).aspx)
Comment: EF6 doesn't support relationships to non-PK fields.
Answer: Create a partial class and add the relationship to the Vendor from Item
public partial class Item
{
    public virtual Vendor ItemVendor { get; set; }
}

Create a partial class for the configuration and inplement the InitializePartial method that the generated partial class calls from the constructor
public partial class ItemConfiguration : System.Data.Entity.ModelConfiguration.EntityTypeConfiguration<Item>
{
    partial void InitializePartial()
    {
        HasOptional(a => a.ItemVendor).WithMany().HasForeignKey(a => a.VendorNumber).WillCascadeOnDelete(false);
    }
}

Your Item class will now have a ItemVendor field filled out.


formultario filtros php

Question: boas estou a fazer uma plataforma em php com html e pedidos ajax, tenho dois anos . Sempre que mudo de ano ele mantem os dados anteriores existe alguma forma de apagar os dados que estao o datalist
Comment: Dois anos? Tão jovem e já programa em PHP? lol #MomentoDescontração. Falando sério agora, poderia melhorar a pergunta e se possível colocar o trecho do código que está com problema? Me parece que seu problema é mais com o Ajax do que com PHP. Aguardo mais informações
Comment: while ($row = $statement->fetch(PDO::FETCH_ASSOC)) {
        $meiovida[] = ""; // Format for adding options
    }
    echo json_encode($meiovida); tenho algo deste genero no php no ajax so faço o append. se usar um select não acontece.
Comment: Desculpe mas ainda não consegui visualizar seu problema, se pudesse melhorar a pergunta ajudaria bastante.
Comment: na query no sql da menos resultados.. no datalist esta a por dados relativos a query anterior
Answer: Olá,
Não sei se entendi muito bem. Mas tu não pode dar um "vazio" usando um innerHtml = "" no elemento e remontar o datalist.

formultario filtros php


What is the limit to which humans can precisely time notes on an instrument or perceive notes as precisely timed?

Question: I have the tendency to either rush individual notes or let them drag behind (see this other question of mine). To cure this I'm recording myself to a click on a DAW and try to see which of the notes in a melody I usually play too early, and which ones I play too late. Of course, there is always some jitter, as I'm not a robot (and that is a good thing).
The question now is: What is the time (might be bpm dependent) that I should consider a note "too late" or "too early", compared to the beat, to work on it. A natural measure would be "when would humans feel that a note is dragging or rushing", but how much is this in ms or in percentage of (for example) one beat note?
Comment: @PiedPiper beginning music students often think they're "close enough" but aren't.
Comment: @phoog OP knows they're not close enough yet.
Comment: This seems like a straightforward and scientific question, but there are a lot of variables. (Not the least of which is that you're asking about human neurology and ability, which is a pretty un-straightforward science!) Guaranteed, individuals will differ in both their learned abilities and their maximum potential. And the nature of the musical material might affect the ability (texture, other instruments playing, counter-rhythms, etc.). It will even vary with the tempo, perhaps in a predictable mathematic way or perhaps not. Your earlier question, "what can I do about it," is much easier!
Comment: …however I don't know waveform. The interface and computer negotiate a latency, broadly, which should be mentioned somewhere in preferences. From here you can see how much delay you would expect from the vst and whether any kind of other compensation is going on. Set the latency as low as possible until the computer struggles is the usual advice, however if you are trying to investigate very accurate timing on an entry level audio recording system it may be frustrating, and a lot of money and time has been spent over the years trying to fix these issues. What's your audio interface?
Comment: Use your ears. If you've achieved the musical results you want, taking your musical style and instrument into account, then you're close enough.
Comment: How do you actually do the recording? Where does the click come from? How have you verified that what gets recorded is the same thing you heard when playing - is this MIDI? Have you measured note-to-note "jitter" when playing by yourself and not to a click? The most important thing to develop is your ear, IMO.
Comment: It is possible that what you see in the DAW differs from what actually happened in the physical world during recording. Try making an audio recording where you record both the click and your playing as audio, click on the left and playing on the right side of a stereo pair. Depending on your DAW software, PC,  operating system and MIDI interface, there can be things which mess up timing in surprising and chaotic ways, and even so that what you actually hear during recording is different from what gets recorded. An audio recording of what was actually heard during recording is the real thing.
Comment: @Quantumwhisp there's various different effects of latency in an audio setup, the one you describe here is usually compensated for in the hardware, it knows the buffer it's been allowed and starts the process of replaying early enough for it to land in time. Most daws do this for all audio it's playing back from disk, but cannot of course do this for the live audio it's processing. Logic has many settings to tailor this behaviour (and you can get in a real pickle setting them wrong on big projects, think tracking with plugin latency delay compensation on full and a lot of plugins…)
Comment: The answer is going to depend on context.  Without a metronome or other reference, people will disagree.  There will be a difference between perceiving deviations from the meter and deviations in rhythm.  Set your metronome to 56 and to 58.  Can you hear the difference?  It is a difference of about 37 ms.  It's pretty easy to perceive the difference between "swung" eighth notes and dotted-eighth-plus-sixteenth rhythm, however, even at tempos where the difference will be less than 37 ms.
Comment: @phoog the latter is the context I'm interested in.
Comment: @piiperiReinstateMonica basically that's what I'm already doing, with an additional "visual" feedback - no?
Comment: *I'm not a robot (and that is a good thing)* - yes, but only up to a point. From the link : "it just required an absolute adherence to the tempo against a stereo echo running at 16s + 8s, left to right, *for long periods* of the track" - My first drum lesson (that I ignored but was eventually able to do) was being told to open and close the HH for 8h straight because humans can only be *thinking* about 3 things at a time; left foot needs to just do that on its own. - Slide the tempo up and you have to start leaving things out that *physically* can't be done at that speed.
Comment: @piiperiReinstateMonica I'm having a DAW running (Waveform) on a laptop that produces the click, and I'm recording the midi signals from my master keyboard. By that I have a clear information of "when I hit the key". I assume that the click will have a delay between the time the software notifies my sound card to do the click, and the time that I can hear it, but I also assumed this would be the same for the vst plugin in waveform that produces the piano sound from my midi signal.
Comment: @piiperiReinstateMonica I did such a thing to make get the whole roundtrip latency (from key press to sound). What do you mean exactly? Assess the timing by a recording of the click AND of the sound (in the way you've described), and then compare to the timing information in the DAW, to see what the differences are?
Comment: The time placement of MIDI notes on the recorded MIDI track may contain jitter relative to what you hear, and if what you hear is based on virtual plugin instruments, what you hear may jitter relative to the MIDI input timing. _In addition_ to latency. And MIDI timing from your MIDI keyboard will jitter relative to what physically happens on the keys. Only audio recording of what you actually hear records what you hear. And what you hear differs from what you physically do. How much jitter does your MIDI keyboard's key scanning add? Physical instruments are always a good option for learning.
Answer: It's very subjective to 'the moment' and also dependent on the genre of music in question. What is considered emotive, exciting, phrasing in one piece may be perceived as sloppy or late execution in another.
That said, when I was working in a fancy studio with high budgets and little room for error my secondary specialism (primary being tuning parts with melodyne etc.) was editing drums. Ideally no editing was the objective but if a band or specifically drummer wasn't getting locked in to the degree desired then editing was needed, generally this was with smaller bands on a lower budget but it happens with very experienced bands too.
When required to make a drum edit I would of course take stylistic considerations into the task, but to answer the question on what's percievable I would tend to work toward either a 10ms or 3ms edit, the latter taking quite a lot more time of course. Other engineers at the studio would send me a part to edit while I was on downtime and they would carry on tracking (of course, tracking to a sub optimal part was not ideal but time is money in studios). The 3ms / 10ms choice would essentially be how I would set the 'nudge' settings in protools, the resolution of each edit I made. 10ms is about enough to take a sloppy part and make it sound 'in time'. This is heavily dependant on the tempo of course but I've found percentage deviation of the position of a transient is less important than absolute deviation. Though this relationship MAY be non linear as using varispeed, which is (generally) speeding up a recording in post does tend to make all timing errors sound less egregious, but it doesn't effect the feel a rhythmic player is imparting.
I should be clear this is with rock/pop recordings where tempo is strict and room for performer time manipulation for the purposes of feel very limited.
So within about 10ms, ±5ms about the objective ideal, a part would sound in time and probably not perceptible as 'bad' to most listeners.
If I were doing a 3ms edit then I would spend time on each snare (for example) pushing and pulling it to get the feel required. 3ms deviation is very hard to recognise, even when I was doing this 10 hours a day every day for many (misspent perhaps?) years of my life! But once you learn to hear it it becomes the difference between something sounding amazingly locked in, ala Steely Dan or similar, and just, well, 'in time'. I want to be clear I was not aiming towards aligning with a grid of the tempo, editing in the way I describe is all done by ear and with an understanding that the most 'locked in' a drum part may sound may be with all the snares (for example, rarely the kicks) behind the beat, or ahead of the beat for that matter.
So I would say a good evaluator of performance at the highest professional level would work at around the 3ms level being the smallest possible margin of error thats perceivable and editable. That sounds small but I think most accomplished and experienced musicians would hear the difference between a drum loop where snares were all exactly on beat compared to where they were all dragging by 3ms. This is probably a skill that only has need to develop  in the western popular music field, where metronomic accuracy, and deviation from it to create pushy or pully feels, is an important but relatively unique concept.
That said, a further understanding of perception of delay can be investigated when recording and dealing with the latency of native audio systems (that is, the audio processing is happening on a general purpose computer, not a specialised low latency device). A critical drawback of earlier audio interfaces was inherent latency in the system, so much so that insanely expensive hardware solutions were the norm (think early Protools). A latency of 5ms - 10ms in recording is definitely noticeable, though not perhaps to the average listener. A performer used to perceiving audio feedback to the tune of the speed of sound (which is probably the limiting factor given the nature of the audio reproduction and perception systems we use) is used to near instant response when sitting next to their drum or guitar amp, and a processing delay of even 5ms was considered 'not that good' for audio interfaces, suggesting some kind of perceived difference down to this level.
Anecdotally, I perceive when playing electric bass (not talking of editing now) about 2ms of latency as feeling odd, and this will potentially impact a recording I make, though it's very subjective at this speed and I would hope I could do a good job at much higher latency. This is specifically on headphones and electric bass, where you get used to the instant feel of finger>string>pickup>electric-signal. It's more a situation of being very slightly off-put by not hearing what your fingertips are feeling, rather than anything being 'late'.
That said, you could be way less accurate than this and still be considered playing perfectly 'in time', especially if you are not playing a style that values these quite silly levels of precision (which is most music, to be honest, although these silly levels of precision have their place for a certain sound!).
EDIT - More Detail on the editing process
It seems like lots of people have read this post so I thought I should add a bit more context in case anyone reads and feels it sounds like an impossibly large task to edit in the way I mention, it's not really that bad, ProTools makes this sort of editing very easy.
Using tab-to-transient + shift you can select all of one drum/note, up to the start of the next, then you can split with b then nudge around with m and n to your nudge resolution. Once you reached the end of the section you want to edit you can use an inbuilt feature called 'fill gaps and crossfade' that deals with any gaps caused by delaying rushy notes.
3ms was the resolution, not the accuracy for every beat. 3ms is almost imperceptible, JUST imperceptible. Thats a good threshold for a tool, now a double tap of 'n' is clearly noticeable, a triple tap and you can fix a flaw. If you had time you could spend a bit more time with single taps to get the feel just right. If the part was bad enough to be pressing nudge 6 times I'd scrap 3ms and go to 10.
With practice you could do it touch-type speed (almost) so its a mad process to hear happening.
Also, the aim here is NOT to hear a decent eg. drum track then try to doctor its feel. Generally you would be editing a section where the groove is sloppy or mistakes were made, once mistakes are fixed it can all sound a bit plastic, with the luxury of time you may then be able to go in with the 3ms tool and push it around until it sounds great. Occasionally the snare might be generally too lazy in eg. the chorus, but click, tab, shift+tab, b, n, n per snare and say 32 - 64 snares in a chorus is not that bad all in all.
For this sort of editing you need 100% isolation, or you will get an impossibly phasey mess. You also almost always edit all the drum tracks grouped together as one for the same reasons, your cymbals start sounding very very odd if you leave the overheads out of the drum group by mistake...
And lastly, the need to edit is not always because someone did a bad job. A common problem is bands that get signed often work really hard and are very good musicians but they haven't studied music deeply for years, or had training and a sound practice ethic to be become world class performers, besides that would probably knock all the edge off your average band anyway! But pressure is high, budgets strained and sometimes aspirations of label/producer/band not matching reality ('We want this one to sound really like Dreamtheater!.....').
Comment: Regarding the 2ms of latency, do you always wear headphones when playing? Sound travels 60cm in that time, so you would have to have the amp right next to your ears to get below that.
Comment: Yeah I'm specifically talking about tracking bass with headphones on, that's about the only way you'd be able to notice I'm sure! But with headphones on and flipping around the latency settings you can spot that slightly detached feel. It's more that your fingertip feel doesn't quite line up to what you are aurally perceiving, rather than actually hearing anything being late! I think there is some perception of even very small latencies on speaker setups, we're subconsciously aware of what we should be hearing considering our distance from speakers, but I agree 2ms is a bit of a stretch there!
Comment: I can't tell any difference until I pull it far enough to cause an echo. You had a full isolation both? Or is there a trick to do this to a live multi track recording? How often was it actually sloppy drums and not just latency? I couldn't even get 'em to toss in a snare hit I'd missed, let alone pull them *all* 3ms one way or another. This place was like 5k an hour?
Comment: @Mazura Well, nice studios, Rockfield and Leeders Farm (now gone), big records recorded there but not 5k an hour! That experience of yours is odd, it only takes a minute to drop in a snare enough to cover a little mistake so not sure what they were worried about...  It was never latency, we were on protools hd usually or tape, the computer just controls the relays (former) and the system is latency free. I added an edit above to your other Q's! Regarding feeling it, you can only really hear 3ms in context (just), so I think it was more my fingertips it may not be as sensitive with sticks..
Answer: Answering the question as asked, a trained human (i.e. drummer or bassist) can  hear variations of a few milliseconds in a steady rhythm with clear attack like metronome or hi-hat.
I have seen some analysis where musicians have been found to play at that level of precision. On the other hand, it is common to play slightly off beat (usually behind) in a controlled way. How much behind, depends on many factors but the common range is between 20 to 70 milliseconds. Other common variations are playing straight rhythms with slight swing, and swing with ratio that is not exactly triplets or dotted notes, etc.
Answer: I came across a video about overly precise tempo notation a few moths ago, and I'm quite sure I found the link under a question right here at Music SE.
It's called This BPM is trash, and here's why, by Adam Neely.
The most relevant part here is the second segment, called Part 1: Time perception.
According to the author, and some research work he's quoting, the reliably (consciously) perceptible difference in timing is on average about 30 ms - though I'm not personally convinced it should be expressed in ms, rather than percentage of the beat in the given tempo (which is not given and I don't trust myself to measure it). Another study he quotes right afterwards said that people notice a change of tempo by about 3 bpm at "moderate" tempos (he's talking about 69 vs. 72 bpm, but isn't that more of a slow tempo than moderate?), or more generally a change of tempo by about 6 to 8 percent.
I don't have any independent source to compare his claims against.
Comment: I agree with the percentage of the tempo being relevant here, for most humans it's much easier to detect a given lack of precision when the notes are played fast than when it's very slow. Also Adam Neely being a bass player, he may have talked about being (with consistance) ahead / on / behind the beat as a fundamental aspect of the bass groove (IIRC he talks about it in some videos)
Answer: (I'd rather have posted this as a comment, but I'm prevented from doing so due to my rep points.  If this answer proves to be a faux pas, I'll gladly delete if prompted by any feedback)
What I want to add is rather small and really only relevant in the context of the answer given by OwenM.  Regarding what he says about noticing a 2ms latency when playing bass with headphones on: I've had a related experience when playing electric guitar.  On one occasion I used a 1/4" patch cord coupler to add another 25 foot patch cord between me and the amp just to see what it might be like to play on a big stage.  I was really surprised to find how pronounced the difference was.  What OwenM said about his perception coming from tactile feedback vs purely from the sound picked up by hearing seems to make sense to me.  I imagine that watching a video of myself playing wouldn't be much different if I watched it for a second time but 25 feet further from the speaker.
I hope this was worth mentioning.
(EDIT:  I had to remove my calculation of the time needed to travel the extra 25 feet due to the extra patch cord.  I kept getting different numbers.  I'll leave the calculations to the reader in the event that anyone is curious.)
Comment: @Stu Smith What I meant was that the extra patch cord let me stand 25 feet further from the amp and resulted in the sound taking longer to reach me.  My original number is probably wrong.  It looks like I should have given a time of about 22ms as the additional time needed for the sound to reach me.  For simplicity, my approximation assumed the travel time of the electrical signal to be zero seconds since I couldn't possibly detect such a tiny amount of time like 0.000003 ms.  I will try to edit in something with better wording to reflect what I have just said here. Thanks for the heads up.
Comment: The speed of sound is about 1125 feet/second, 1.125 feet per millisecond. So that would be... how many milliseconds for 25 feet? My math gets 22 ms, not 4.
Comment: If I understand you correctly, you're saying that the electrical signal took 4ms to travel the 25 feet of patch cord.  If my math is correct, electricity will travel that distance in about 0.00003 ms.  See *https://en.wikipedia.org/wiki/Speed_of_electricity*
Comment: Sound does travel about 60 cm in 2 ms in air, one foot is about 30 cm. 25 ft is about 750 cm, or 7.5 m. I assume you were talking about air distance and you walked 25 ft away from the amp.
Comment: @piiperi Reinstate Monica   I shouldn't have relied on one of the comments which said that sound travels 60cm in 2ms to make my calculations.  I will edit in the correction you suggest.  (I guess I should do my own research for verification first, right?)
Comment: @piiperi Reinstate Monica  Okay now I'm really embarrassed!  I didn't think my math was so awful.  To expose my error, I took 60 cm and divided by 2.54 which is about 23.62 inches for 2 ms and then divided by 2 to get inches traveled in 1ms and then divided by 12 to get feet traveled in 1ms.  I then divide the length of 25 feet by the first result to see how many ms it took to travel 25 feet.  I did this two times an got the same result of about 4ms.  Just now I got about 25ms.  I have now proved my incompetence as a poster here.  I should have listened to my gut. Penalty: HUMILIATION!
Comment: 25 feet of cable between guitar and amp will also change the tone if you don't have active preamp or buffering pedals at the guitar end. 22ms is definitely noticeable though.
Answer: Sound travels about 1ft in 1 millisecond. A typical small ensemble will span something like 10 feet, a large ensemble like an orchestra can span several tens of feet.  People don't notice the members of these groups being "out of time" with each other, so in terms of the relative timing of components of an ensemble consistent relative delays of a few to maybe a few tens of milliseconds do not seem un-musical.
Comment: I have heard from other sources that full size orchestra needs a conductor, because they hear each other out of time. The question was also about jitter between individual notes, not constant delay.
Comment: Latency, delay, is not a big problem because it can be compensated, adjusted to. A lot of that adjusting happens automatically, and can be worked around with organization. Jitter is a problem because it cannot be compensated, it is timing error. Jitter cannot be corrected with any sort of organizing, only by playing better.


Functional equation $f(x+y)=\frac{f(x)+f(y)}{1-4f(x)f(y)}$ with $f'(1)=1/2$

Question: Try to find the solution of  the functional equation $$f(x+y)=\frac{f(x)+f(y)}{1-4f(x)f(y)}$$ with $f'(1)=1/2$.
Comment: What is your question? What have you tried so far?
Comment: I would say $f(x)=(\tan x)/2$ if it were $f'(0)=1/2$
Comment: Apparently you assume that $f$ is differentiable.
Answer: HINT: Notice that the equation is equivalent to
$$2 f(x+y) = \frac{2 f(x) + 2 f(y)} { 1 - (2 f(x)) \cdot (2 f(y))}$$
while we know that
$$\tan(a(x+y)) = \frac{\tan(ax) + \tan(ay)}{1 - \tan(a x) \cdot \tan(a y)}$$
so we might just have $2 f(x) \equiv \tan a x$ for some convenient $a$.
Answer: I am solving it with the assumption that $f'(0)=1/2$ and $f$ is continuously differentiable in an open neighbourhood containing $0$.
First, we observe that $f(0)=0$.
Then,
$$
\frac{f(x+h)-f(x)}{h}=\frac{\frac{f(x)+f(h)}{1-4f(x)f(h)}-f(x)}{h}
=\frac{f(x)+f(h)-f(x)+4f(h)f^2(x)}{h\big(1-4f(h)f(x)\big)}
=\frac{f(h)\big((1+4f^2(x)\big)}{h\big(1-4f(h)f(x)\big)}\to f'(0)\big(1+4f^2(x)\big)=\frac{1}{2}\big(1+4f^2(x)\big).
$$
Hence, $f$ satisfies the IVP
$$
f'(x)=\frac{1}{2}\big(1+4f^2(x)\big), \quad f(0)=0.
$$
Hence
$$
\frac{d}{dx}\arctan\big(2f(x)\big)=\frac{2f'(x)}{1+4f^2(x)}=1.
$$
Thus $\arctan\big(2f(x)\big)=x+c$, for some $c\in\mathbb R$, and thus
$f(x)=\frac{1}{2}\tan (x+c)$. The initial data $f(0)=0$ forces $c=0$.
Thus $f(x)=\frac{1}{2}\tan x$.
Note. If $f'(1)=1/2$ is not a mistype in the OP, then we obtain instead that
$$
f(x)=\frac{1}{2}\tan\big(2f'(0)x\big),
$$
and
$$
\frac{1}{2}=f'(1)=\frac{1}{2}\cdot 2f'(0)\cdot\frac{1}{\cos^2\big(2f'(0)\big)},
$$
and thus $2f'(0)= \cos^2\big(2f'(0)\big)$. Now the equation $x=\cos^2(x)$, has a unique solution $s_0\approx 0.6417$ and thus
$$
f(x)=\frac{1}{2}\tan\big(s_0x\big).
$$
Comment: Thanks! But, could you solve the original problem with  the condition $f'(1)=1/2$?
Comment: @Pippo That's the answer!
Answer: Define $g(x)=\arctan (2f(x))$ so $f(x)=\frac{1}{2}\tan g(x)$ and $$\tan g(x+y)=\frac{\tan g(x) + \tan g(y)}{1-\tan g(x) tan g(y)}=\tan (g(x)+g(y)).\quad(*)$$ Thus $$g(x+y)=g(x)+g(y)+\Pi(x,\,y),\quad(**)$$ where $\Pi(x,\,y)$ is a function with range $\subseteq \pi\mathbb{Z}$. Thus $g'(x+y)=g'(x)+g'(y)+\frac{\partial \Pi}{\partial x}$, where for the last term to be well-defined requires $\Pi$ to be constant for fixed $y$. A similar argument with $y$-derivatives shows $\Pi$ is constant; taking $x=y=0$ in $(**)$ gives $g(0)=-\Pi$. Since $\Pi$ is constant, define $G:=g-\Pi$ so $G$ satisfies the Cauchy relation $G(x+y)=G(x)+G(y)$. The only solution is $G(x)=Kx$ with $K$ constant. (The axiom of choice implies other solutions to the Cauchy relation exist, but they preclude f(0) being well-defined.) Thus $0=G(0)=g(0)-\Pi=-2\Pi$ and $\Pi=0$. Then $1=2f'(0)=g'(0) \sec^2 g(0)=K$ and $f(x)=\frac{1}{2}\tan x$.


How many sequences of length 5 can be formed using the digits 0, 1, 2, . . . , 9 with restriction

Question: How many sequences of length 5 can be formed using the digits 0, 1, 2, . . . , 9 with the property that exactly two of the 10 digits appear, e.g., 05550?
My thoughts are to determine the number of the first number first , then determine how many of the first number it can have .
$ {5 \choose 1} * 2^4$
is that correct?
Answer: First, let us choose 2 digits out of 10 that will appear in the sequence, there are $$\binom{10}{2}$$ possibilities. Now let's form a sequence with these 2 digits. For each element of the sequence we have two choices. So there are $$2^5$$ possible combinations. However, remember that sequence such as $00000$ or $55555$ are illegal. So our final answer will be $$\binom{10}{2}\cdot(2^5-2)=1350$$


CG-modules: what does this notation mean?

Question: I am trying to solve a question, but I do not know what the notation used means. If anyone could help me out that'd be great! I don't need help doing the proof, just what the notation means would be brilliant.
I have a finite group $G$ and $V, W$ finite-dimensional  $\mathbb{C}G$-modules. Let $L:V\rightarrow W$ be a linear map and define $p:V\rightarrow W$ by $p(x)=\sum_{g\in G}g^{-1}Lg(x)$.
Prove that $p$ is a homomorphism of $\mathbb{C}G$-modules.
I don't understand the what the definition of $p$ means.
Thanks,
Andy.
Answer: The notation here is a little awkward, so I'm going to adjust it a bit. You have that $V$ is a $\mathbb{C}G$-module, so for each $v\in V$ and $g\in G$, you should know what $g.v$ means (and similarly for $g.w$, $w\in W$).
Now,  starting from a linear transformation $L:V\to W$ and $x\in V$ we can compute $g^{-1}.L(g.x)$ (that is, act on $x$ with $g$, map it over to $W$ using $L$, and then act on that by $g^{-1}$). This produces a vector in $W$.
We can do this for each $g\in G$, and since $W$ is a vector space, it makes sense to add all these elements together. In the end, we get a map
$$p(x)=\sum_{g\in G} g^{-1}.L(g.x).$$
It is claimed that this is a $G$-map, so you need to prove that $p(h.x)=h.p(x)$ for every $h\in G$.

Remark about notation: Saying that $V$ and $W$ are $\mathbb{C}G$-modules is the same as having homomorphisms
$$\pi_V:\mathbb{C}G\to \mathrm{End}(V)\;\;\mbox{ and }\;\;\pi_W:\mathbb{C}G\to \mathrm{End}(W)$$
Therefore, $\pi_V(g):V\to V$ is a linear map, as is $\pi_W(g):W\to W$. If we abuse notation and identify $g$ with its image under these maps, then we can interpret $g^{-1}Lg(x)$ as the composition of maps
$$
V\longrightarrow^{\pi_V(g)}V\longrightarrow^LW\longrightarrow^{\pi_W(g^{-1})}W
$$
Answer: For $g\in G$ and $x\in V$, $g^{-1}Lg(x)$ is the element of $W$ obtained by first multiplying $x$ by $g$ using the $\mathbb{C}G$-module structure of $V$, then applying the linear map $L$ to get an element of $W$, and then multiplying that element of $W$ by $g^{-1}$ using the $\mathbb{C}G$-module structure of $W$.  Then $p(x)$ is just the sum of the elements of $W$ you obtain in this way using each element $g\in G$.
Comment: Thank you Eric. I have managed to prove that p is a linear map also, but have hit a bit of a stumbling block on the second condition for a CG-homomorphism. When trying p(hx). (For some h in G) I get the following (omitting the sum sign) g^-1 Lg(hx). Which can be written as g^-1 L (gh)x. But I can't seem to get any further than that. Any ideas? Thanks, Andy.
Comment: The key thing you want to use is that the set $\{gx:g\in G\}$ is equal to the set $\{ghx:g\in G\}$, since $\{gh:g\in G\}=G$.
Comment: Thanks again Eric, that makes sense - I think I was just overthinking it. I'm trying some questions from my uni days and I'm not quite as switched on as I used to be!


Effects of Education Spending on Crime, multi-level mixed-model structure

Question: I'm looking at the effect of education_expenditure per school district on crime rate within the cities and towns those school districts serve over a fifteen year period. (The DV has 1,676,191 observations of city/town crime data over those fifteen years).
Cities are technically crossed with school district, in that one city might attend multiple school districts. This means that one city could have multiple values for expenditure per student. School districts, however, overlap with counties.
Cities are nested within county, but given that each city/town has a distinct PLACE_ID, my understanding is that this could be represented as (1|PLACE_ID) + (1|COUNTY_ID) or (1|PLACE_ID/COUNTY_ID).
I'm pretty familiar with mixed-effect models, and I've looked through clear and informative posts such as this one: https://stats.stackexchange.com/questions/228800/crossed-vs-nested-random-effects-how-do-they-differ-and-how-are-they-specified ; however, I'm still a little stuck with whether I can create a crossed effect model like the one below:
glmer.total <- glmer(CRIME_TOTAL ~ cent.log.pop + cent.log.pop.dens + year + cent.log.unemployment_rate + cent.schooldist.prop5.17.pov + cent.log.per.cap + diff.dem + cent.log.enforcement +cent.EXP_STUDENT + (year|PLACE_ID/COUNTY_ID) + (year|full_district_id) + (1|STATE), family = "poisson", control = glmerControl(optimizer = "nloptwrap", calc.derivs = FALSE), REML = FALSE, total.years, na.action = "na.omit")

Variables are centered and logged: pop per city, pop.dens per city, year, unemployment rate per county, proportion children living in poverty per school district, per capita income per county, difference in those who voted democrat in presidential elections per county, log enforcement per city/town, centered expenditure per student/ 1000 (per school district). PLACE_ID corresponds to cities and towns, COUNTY_ID to counties, full_district_id to school districts, and state.

Would I need to choose to average expenditure per city by year, or is
the above code legitimate? I'm receiving the following error:

extra argument(s) 'REML' disregardedError in pwrssUpdate(pp, resp, tol = tolPwrss, GQmat = GHrule(0L), compDev = compDev,  :
  (maxstephalfit) PIRLS step-halvings failed to reduce deviance in pwrssUpdate

I've provided a brief snippet of the data below (but selected only a number of variables to make it much smaller, as a dput.
Thanks!
structure(list(STATE = c("alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama", "alabama", "alabama",
"alabama", "alabama", "alabama", "alabama"), state = c("AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL", "AL",
"AL"), full_district_id = c("0100510", "0100510", "0100510",
"0100510", "0100090", "0100540", "0101860", "0102635", "0102760",
"0100090", "0100540", "0101860", "0102635", "0102760", "0100090",
"0100540", "0101860", "0102635", "0102760", "0100090", "0100540",
"0101860", "0102635", "0102760", "0100090", "0100540", "0101860",
"0102635", "0102760", "0100090", "0100540", "0101860", "0102635",
"0102760", "0100090", "0100540", "0101860", "0102635", "0102760",
"0100090", "0100540", "0101860", "0102635", "0102760", "0100090",
"0100540", "0101860", "0102635", "0102760", "0100090", "0100540",
"0101860", "0102635", "0102760", "0100090", "0100540", "0101860",
"0102635", "0102760", "0100090", "0100540", "0101860", "0102635",
"0102760", "0100090", "0100540", "0101860", "0102635"), SCHOOL_DISTRICT.x = c("butler county school district",
"butler county school district", "butler county school district",
"butler county school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district", "piedmont city school district",
"anniston city school district", "calhoun county school district",
"jacksonville city school district", "oxford city school district",
"piedmont city school district", "anniston city school district",
"calhoun county school district", "jacksonville city school district",
"oxford city school district"), COUNTY = c("butler ", "butler ",
"butler ", "butler ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ",
"calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun ", "calhoun "
), year = c(2006, 2007, 2008, 2009, 2006, 2006, 2006, 2006, 2006,
2007, 2007, 2007, 2007, 2007, 2008, 2008, 2008, 2008, 2008, 2009,
2009, 2009, 2009, 2009, 2010, 2010, 2010, 2010, 2010, 2011, 2011,
2011, 2011, 2011, 2012, 2012, 2012, 2012, 2012, 2013, 2013, 2013,
2013, 2013, 2014, 2014, 2014, 2014, 2014, 2015, 2015, 2015, 2015,
2015, 2016, 2016, 2016, 2016, 2016, 2017, 2017, 2017, 2017, 2017,
2007, 2007, 2007, 2007), COUNTY_ID = c("01013", "01013", "01013",
"01013", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015", "01015", "01015", "01015", "01015", "01015",
"01015", "01015"), SCHOOL_DISTRICT.y = c("BUTLER CO SCH DIST",
"BUTLER CO SCH DIST", "BUTLER COUNTY SCHOOL DISTRICT", "BUTLER COUNTY SCHOOL DISTRICT",
"ANNISTON CTY SCH DST", "CALHOUN CO SCH DIST", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCH DIST", "PIEDMONT CTY SCH DIST", "ANNISTON CTY SCH DST",
"CALHOUN CO SCH DIST", "JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCH DIST",
"PIEDMONT CTY SCH DIST", "ANNISTON CTY SCH DST", "CALHOUN COUNTY SCHOOL DISTRICT",
"JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCHOOL DISTRICT",
"PIEDMONT CITY SCHOOL DISTRICT", "ANNISTON CITY SCHOOL DISTRICT",
"CALHOUN COUNTY SCHOOL DISTRICT", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCHOOL DISTRICT", "PIEDMONT CITY SCHOOL DISTRICT",
"ANNISTON CITY SCHOOL DISTRICT", "CALHOUN COUNTY SCHOOL DISTRICT",
"JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCHOOL DISTRICT",
"PIEDMONT CITY SCHOOL DISTRICT", "ANNISTON CITY SCHOOL DISTRICT",
"CALHOUN COUNTY SCHOOL DISTRICT", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCHOOL DISTRICT", "PIEDMONT CITY SCHOOL DISTRICT",
"ANNISTON CITY SCHOOL DISTRICT", "CALHOUN COUNTY SCHOOL DISTRICT",
"JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCHOOL DISTRICT",
"PIEDMONT CITY SCHOOL DISTRICT", "ANNISTON CITY SCHOOL DISTRICT",
"CALHOUN COUNTY SCHOOL DISTRICT", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCHOOL DISTRICT", "PIEDMONT CITY SCHOOL DISTRICT",
"ANNISTON CITY SCHOOL DISTRICT", "CALHOUN COUNTY SCHOOL DISTRICT",
"JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCHOOL DISTRICT",
"PIEDMONT CITY SCHOOL DISTRICT", "ANNISTON CITY SCHOOL DISTRICT",
"CALHOUN COUNTY SCHOOL DISTRICT", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCHOOL DISTRICT", "PIEDMONT CITY SCHOOL DISTRICT",
"ANNISTON CITY SCHOOL DISTRICT", "CALHOUN COUNTY SCHOOL DISTRICT",
"JACKSONVILLE CITY SCHOOL DISTRICT", "OXFORD CITY SCHOOL DISTRICT",
"PIEDMONT CITY SCHOOL DISTRICT", "ANNISTON CITY SCHOOL DISTRICT",
"CALHOUN COUNTY SCHOOL DISTRICT", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCHOOL DISTRICT", "PIEDMONT CITY SCHOOL DISTRICT",
"ANNISTON CTY SCH DST", "CALHOUN CO SCH DIST", "JACKSONVILLE CITY SCHOOL DISTRICT",
"OXFORD CITY SCH DIST"), EXPENDITURE_PER_STUDENT = c(7593, 8334,
9281, 9147, 8240, 7216, 6822, 7554, 7155, 8758, 8186, 7323, 8380,
7710, 9070, 8707, 8070, 8853, 8054, 9364, 8212, 7787, 8560, 7760,
10006, 8264, 7964, 8771, 8442, 10154, 8209, 7955, 8967, 7911,
10661, 8157, 8096, 9097, 7660, 11480, 8415, 9351, 8829, 8102,
12057, 8479, 8708, 8965, 8443, 10988, 8930, 8932, 9118, 8706,
11277, 9134, 9223, 9347, 8524, 11277, 9134, 9223, 9347, 8524,
8758, 8186, 7323, 8380), PLACE_ID = c("0101345496", "0101345496",
"0101345496", "0101345496", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101501852", "0101501852", "0101501852",
"0101501852", "0101501852", "0101529992", "0101529992", "0101529992",
"0101529992"), CITY = c("mckenzie", "mckenzie", "mckenzie", "mckenzie",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"anniston", "anniston", "anniston", "anniston", "anniston", "anniston",
"glencoe", "glencoe", "glencoe", "glencoe"), POPULATION.EST = c(548,
542, 536, 526, 23470, 23470, 23470, 23470, 23470, 23360, 23360,
23360, 23360, 23360, 23313, 23313, 23313, 23313, 23313, 23262,
23262, 23262, 23262, 23262, 23106, 23106, 23106, 23106, 23106,
22849, 22849, 22849, 22849, 22849, 22644, 22644, 22644, 22644,
22644, 22457, 22457, 22457, 22457, 22457, 22280, 22280, 22280,
22280, 22280, 22107, 22107, 22107, 22107, 22107, 21926, 21926,
21926, 21926, 21926, 21770, 21770, 21770, 21770, 21770, 32, 32,
32, 32), CRIME_VIOLENT = c(1, 0, 2, 1, 521, 521, 521, 521, 521,
541, 541, 541, 541, 541, 572, 572, 572, 572, 572, 584, 584, 584,
584, 584, 0, 0, 0, 0, 0, 411, 411, 411, 411, 411, 504, 504, 504,
504, 504, 461, 461, 461, 461, 461, 536, 536, 536, 536, 536, 607,
607, 607, 607, 607, 735, 735, 735, 735, 735, 754, 754, 754, 754,
754, 9, 9, 9, 9), CRIME_PROPERTY = c(6, 9, 9, 7, 3044, 3044,
3044, 3044, 3044, 2912, 2912, 2912, 2912, 2912, 2429, 2429, 2429,
2429, 2429, 2379, 2379, 2379, 2379, 2379, 2038, 2038, 2038, 2038,
2038, 2323, 2323, 2323, 2323, 2323, 2484, 2484, 2484, 2484, 2484,
1988, 1988, 1988, 1988, 1988, 1711, 1711, 1711, 1711, 1711, 1645,
1645, 1645, 1645, 1645, 1712, 1712, 1712, 1712, 1712, 1352, 1352,
1352, 1352, 1352, 106, 106, 106, 106), CRIME_TOTAL = c(7, 9,
11, 8, 3565, 3565, 3565, 3565, 3565, 3453, 3453, 3453, 3453,
3453, 3001, 3001, 3001, 3001, 3001, 2963, 2963, 2963, 2963, 2963,
2038, 2038, 2038, 2038, 2038, 2734, 2734, 2734, 2734, 2734, 2988,
2988, 2988, 2988, 2988, 2449, 2449, 2449, 2449, 2449, 2247, 2247,
2247, 2247, 2247, 2252, 2252, 2252, 2252, 2252, 2447, 2447, 2447,
2447, 2447, 2106, 2106, 2106, 2106, 2106, 115, 115, 115, 115),
    prop.5.17.pov = c(30.2, 33.1, 28.1, 33.4, 25.6, 25.6, 25.6,
    25.6, 25.6, 24, 24, 24, 24, 24, 20.1, 20.1, 20.1, 20.1, 20.1,
    23.7, 23.7, 23.7, 23.7, 23.7, 30.8, 30.8, 30.8, 30.8, 30.8,
    31.1, 31.1, 31.1, 31.1, 31.1, 30.5, 30.5, 30.5, 30.5, 30.5,
    27, 27, 27, 27, 27, 25.4, 25.4, 25.4, 25.4, 25.4, 29.8, 29.8,
    29.8, 29.8, 29.8, 24.2, 24.2, 24.2, 24.2, 24.2, NA, NA, NA,
    NA, NA, 24, 24, 24, 24)), row.names = c(NA, -68L), class = c("tbl_df",
"tbl", "data.frame"))

Comment: This doesn't seem like a specific programming question that's appropriate for Stack Overflow. If you have general questions about the appropriate use of various statistical methods, then you should ask such questions over at [stats.se] instead. You are more likely to get better answers there.
Comment: It deals with the specific data structure of R, and then the coding of a mixed-effect model in R. Lme4 is a package specific to R, and the nesting and crossed random effects structure specific to lme4 in a complicated (at least for me) dataset is the essence of my question.
Answer: A very partial answer:

(year|PLACE_ID/COUNTY_ID) looks backwards (the nesting syntax is larger/smaller), but given your description of the coding, either  (year|PLACE_ID) + (year|COUNTY_ID) or (year|COUNTY_ID/PLACE_ID) would work. (Or (year|STATE/COUNTY_ID/PLACE_ID) ?)
crossing districts and states/counties seems reasonable, although I can imagine things getting a bit weird if some PLACE_IDs constitute a single school district ...
it should be fine to give expenditure per district

It's hard to say what's going wrong with the full model.  How small a subsample can you take of the data and replicate the problem?  Does the model work as a linear mixed model? If so, does the fitted model give you any hints about what might be weird with the data/model?
Comment: Thanks, Ben! Sorry--the COUNTY_ID/PLACE_ID was a writing mistake.
Comment: That's helpful to know that I'm at least structuring the model correctly. The problem I have with (year|STATE/COUNTY_ID/PLACE_ID) is that it takes forever to run. I used UCSDs supercomputer and I received a kill error; apparently it went over 128gb, and that's winnowing the dataset to a single year (2016). I also tried the "bobyqa" optimizer (optCtrl=list(maxfun=2e5)). Still no luck. When I run the model with some of the covariates on R, I either get the error listed in my post, or it takes forever. Not sure why it consumed so much memory on the supercomputer.
Comment: One possible issue is that year should be **numeric**, not a factor.  That would make a huge difference ...  Running with a single year seems weird (because the random effects will work badly in any case).  1.6M observations is a *lot* for a mixed model.  How do you do with, say, 10,000 or 100,000 observations sampled at random?
Comment: PS it might be worth trying `firstname.lastname@example.com` with this one, to get a wider range of opinions from a group of people who are more specifically interested in this topic ...


Active Directory Multi Site. Choose nearest DCs into Linux/Not-Microsoft applications

Question: i'm having a problem using ldap based on Active Directory in a multi-site environment.
Basically I have several different environments(sites) and each of them have its own dedicated couple of domain controller servers.
Each couple of domain controllers talks to the couples in other sites in order to keep all informations synchronized.
In each environment I have also several different linux servers(webservers,applications servers etc.) and, for authentication and authorization purposes, they have to contact the correct domain controllers which are dedicated to their environment.
My problem is that i can't find a way to specify into the configuration of those servers how to contact their "nearest" domain controller.
Until now i used the DNS A record domain.local which returns EVERY domain controller into the domain; the problem is that it returns also domain controllers not in the correct site and so unreachable.
The another way i thought is to create an CNAME record for each site which refer to the correct DCs.

DC-Site1 CNAME to DC1 and DC2
DC-Site2 CNAME to DC3 and DC4
DC-Site3 CNAME to DC5 and DC6
....

So using the record DC-SiteX.domain.local i'm able to contact the correct couple of domain controllers in the site.
The problem of this solution is that i have to hardly code into the configuration server which site it belongs. I don't like because i may move the servers to different site and i have to remember to update this configuration.
Usually how do you handle this situation?
Do you have an elegant solution for this problem?
Answer: You are correct, using domain.local name will return list of all addresses of all domain controllers in the domain, but if you will try to ping domain.local (I mean actual name of your domain), DNS must return IP address of local domain controller.
I don't know if your domain controllers are also acting as DNS servers, but if so, check DNS Server Setting -> Advanced Tab -> Enable netmask ordering checkbox must be checked in.
Regarding CNAME, you can create several CNAMEs for one server, but not one CNAME for several servers, so you have to create two A records with the same name poining to different IPs of local domain controllers. But for this case you also need to make sure you have anabled roud robin in advanced DNS Server settings, so queries will be balanced beetween servers by DNS server.
Best regards, Sergey
Comment: Both enable netmask ordering and Enable round robin are enabled on all DNS (all DC have the DNS role). However it happens that using the "domain.local" A record sometimes the applications try to use domain controllers in different sites.
Comment: Have you added all clients networks to appropriate sites in AD Sites and Services? Or you mean that clients from same network that is defined in AD Sites and Services and assigned to correct site sometimes picking up "incorrect" domain controller?
Comment: yes, i confirm that into the active directory sites and service there are all mappings between all subnets and all sites
Comment: An example of the problem is the following: a apache webserver belongs to site A which have DC1 and DC2 as domain controller. Site B uses DC3 and DC4. Sometimes i found that the apache is trying to contact DC3 or DC4.. of course the apache IP belongs to a subnet which is mapped to site A (into AD sites and services)
Comment: Take a look on this article - https://support.microsoft.com/en-us/kb/247811 Basically check DNS server, do you have ldap _srv records for "remote" domain controllers like DC3 and DC4 in _msdsc.domain.local -> dc -> _sites -> SiteA -> _tcp. Also make sense to check all DNS server structure for SVR records to make sure you do not have foreign SVR record in local site location
Answer: Do the unix servers have knowledge of the site they are in?
Does the AD controller has correctly updated the SRV DNS records (it does by default)?
If the unix server is aware to be, for example, in SITE1.EXAMPLE.ORG, it can ask AD controller which LDAP server is enabled on SITE1 with the following:
ldapsearch -v -H "ldap:///dc%3Dsite1%2Cdc%3Dexample%2Cdc%3Dorg" [...]

basically it is a ldapsearch to whatever server has a LDAP SRV registration on SITE1.EXAMPLE.ORG:
host -t SRV _ldap._tcp.site1.example.org.

Comment: @Daniel: It IS the responsibility of the client to: use DNS to determine which site it is in, determine the domain controllers for that site (or domain if no domain controllers/not in a site), and perform DNS/LDAP tests on the domain controllers to determine which domain controller is most suitable.  The domain controller location process has always worked this way. Any application or computer/operating system that needs to integrate with Active Directory MUST also do this if there is any expectation that it will perform correctly and reliably.
Comment: While this is old, I had to add a note about OpenLdap's support of SRV records. It works fine. Active Directory does not "own" SRVs and it supports the basic LDAP standard. So the above answer is accurate. Other than the fact that sites are typically contained inside the _sites subzone in AD DNS: `ldapsearch -v -H "ldap:///dc%3DSite1%2E%5Fsites%2Cdc%3Dexample%2Cdc%3Dcom"`
Answer: This is the responsibility of the application.  Any application that needs to integrate with Active Directory must conform to how it works.  This includes prioritizing the domain controllers to use by site/location preference, selecting an appropriate domain controller that is functional, and failing over to another domain controller if an error occurs.
In other words, the application needs to function just as a Windows client functions.
Answer: This gives me the correct SRV records for a specific site's domain controllers.
There was a firewall involved so relying on the "client's IP" was ineffective.
windows command line tool
nslookup -type=SRV _ldap._tcp.$sitename$._sites.dc._msdcs.$domain.name$

Answer: This is a somewhat old post, but I wanted to offer the following information to anyone that comes across this.  It is possible to control what A records and SRV records your domain controllers dynamically register and in what sites and configure the SRV record weight and priority.  See the following Microsoft article:
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-adts/76bc11f0-8bc2-4d6c-a643-328f479aae1b


How to display 'Terms and conditions' before starting a download, possibly using Rules?

Question: We have a page displaying various download options. When the user clicks one of the options we would like to display a terms and condition page and then take the user further to the actual download page.
There is only one terms and conditions but multiple download pages.
I looked at the Legal module. But this seems to be for users who register and users don't need to register for download.
I also looked at Rules module. But this is for recurring events. But it is not a recurring event but an on click event.
Has anybody got an idea which strategy would be helpful here?
Note: terms and conditions are the same for every option.
Comment: try using `hook_file_download`.
Answer: It should be possible to implement a procedure that fits your requirement, using a combination of the Rules and Flag modules. Here is how you could do so:

Create a Flag (using the Flag module), related to your "terms and conditions" page (=TOC). The actual perpose of your flag is to have the user "flag" the TOC-page to confirm something like "I agree to the terms and conditions".
For any of the relevant download pages, try to have a path for each of them that starts with something like downloads/. It can be any path you like, just make sure it's something that uniquely indicates "this is one of the download pages for which we want to make sure the user agreed to the TOC as on the TOC-page".
When the user then requests the download page to be viewed (opened), add a rule that is a variation of the rule I included (in export format) in my answer to the question about How to restrict access to a node via node/12 and allow access via a path like content/sometitle?
These are the changes to that rule to make it work for this case also:

Change node to downloads (as in my assumptions above in "2.").
Add a Rules 'Condition' like "If this user did NOT flag the TOC-page yet".
Change the "Message" (in the Rules 'Action'- to something to indicate that the user must first "accept the TOC" (by flagging the TOC-page, as in "1.".
Add an extra Rules 'Action' to issue a redirect from the selected download page to ... you guessed it ... the TOC-page!

Haven't experimented/tried creating this Rule (= variation of the rule in the answer I mentioned above), but pretty sure it'll work. Hopefully you'll be able to complete the rest of the puzzle in my answer on your own (if not let me know where you get stuck and I might enhance my answer for it also).


change search item url and routing in codeigniter

Question: I want to change the URL with Codeigniter routing:
here is my url:
home/search?location=BD

home/search?location=BD&category[]=123

home/search?location=BD&category[]=123&category[]=124&category[]=125

like above url but I want to routing this url with
home/BD

home/BD/123

home/BD/123+124+125

or
home/BD/123/124/125

My route.php:
$route['home/(:any)/(:any)'] = 'home/search/$1';

What is my problem in route.php page?
Answer: Try to use a (.+) pattern on your route.php, the $1 will contain the location value (BD) and the $2 will contain every parameters past the home/BD/ url :
$route['home/(:any)/(.+)'] = 'home/search/$1/$2';

The (.+) pattern is useful if you don't know how many parameters are being passed, it will allow you to capture all of them. And maybe you should use & in place of the + sign on your url since the + sign is probably disallowed by default :
home/BD/123&124&125

Then you could explode the categories on the controller :
public function search($location = '', $categories = '')
{
    if (!empty($categories)) {
        $categories = explode('&',$categories);
    }
    ...
}


Debug C++ dll from C# works in VS2013, but not VS2010

Question: Enabled "Enable native code debugging" setting in 2010/2013 and verified debug symbols are loaded.  Cannot step into the C++ source.  After running the same project in VS2013 (without changing any settings), code steps into the C++ source.  Why?
Note: This is being run from a c# unit test project.
Answer: Having the PDB file next to the DLL is not enough to debug the DLL.
You will need to have access to the DLL code so that you can step into it from your debugger.
Have a look at that thread:
http://social.msdn.microsoft.com/Forums/en-US/69e84750-6636-4656-bbd4-8d3586290af3/how-to-debug-native-c-dll-from-c-application?forum=netfxcompact


A successor to regex?

Question: Looking at some of the regex questions commonly asked on SO, it seems to me there's a number of areas where the traditional regex syntax is falling short of the kind of tasks people are looking for it to do nowadays. For instance:

I want to match a number between 1 and 31, how do I do that ?

The usual answer is don't use regex for this, use normal conditional comparisons. That's fine if you've got just the number by itself, but not so great when you want to match the number as part of a longer string. Why can't we write something like \d{1~31}, and either modify the regex to do some form of counting or have the regex engine internally translate it into [1-9]|[12]\d|3[01] ?

How do I match an even/odd number of occurrences of a specific string ?

This results in a very messy regex, it would be great to be able to just do (mytext){Odd}.

How do I parse XML with regex ?

We all know that's a bad idea, but this and similar tasks would be easier if the [^ ] operator wasn't limited to just a single character. It'd be nice to be able to do <name>(.*)[^(</name>)]

How do I validate an email with regex ?

Very commonly done and yet very complex to do correctly with regex. It'd save everyone having to re-invent the wheel if a syntax like {IsEmail} could be used instead.

I'm sure there are others that would be useful too. I don't know too much about regex internals to know how easy these would be too implement, or if it would even be possible. Implementing some form of counting (to solve the first two problems) may mean it's not technically a 'regular expression' anymore, but it sure would be useful.
Is a 'regex 2.0' syntax desirable, technically possible, and is there anyone working on anything like this ?
Comment: Regexes are the **lowest common denominator** of text matching, and they're good at that. Anything more specific would be better handled by DSLs than the general purpose tool that is Regexes. Yes, it'd be nice to add a kitchen sink to Regexes, but they're already complex enough. Tempted to vote for *subjective and argumentative*.
Comment: It's called a lexer and parser. There are plenty generators on the net. They have been around probably just as long as Regex.
Comment: The e-mail one is something which ought to be built into the standard networking API of every new language which has one. It's the kind of wheel which people shouldn't need to think about inventing.
Comment: @leppie, @deceze looking at this from the perspective of a user of regexes io looking at the technical difficulties involved adding the functionality, it would be handy (and probably much (ab)used). Using a lexer and parser often still requires you to use regexes. Now you have to know two things.
Comment: @Leiven: To build a house, you need to learn to use multiple specialized tools... There is nothing wrong with that. A specialized tool with always do the job better then a generic tool.
Comment: great question, miket2e. on of the "yeah, exactly, whats up with that?" questions. unfortunately i have no answer ready but i'm curious for your results
Comment: When all you have is a hammer, everything looks like a nail. This isn't really a question.
Comment: Maybe you should try it on http://programmers.stackexchange.com -- more suited for this kind of question.
Comment: @Andrew - following that logic we would still have a seperate scanner, copier and fax. Instead we have multifunctionals. The question is where to draw the line in making things easier.
Comment: @Sohnee - The problem is that the nail in this example is a string. When people stop using strings, developers will stop trying to hammer them.
Comment: @CurtainDog - RegEx isn't *the* tool for strings. It is *a* tool for strings.
Comment: @Lieven: The multifunction of string parsing is a lexer-generator and parser-generator that were built to go together.
Answer: I believe Larry Wall covered this with Perl 6 regexes. The basic idea is to replace simple regular expressions with more-useful grammar rules. They're easier to read and it's easier to put code in for things like making sure that you have an number of matches. Plus, you can name rules like IsEmail. I can't possibly list all the details here, but suffice it to say, it sounds like what you're suggesting.
Here are some examples from http://dev.perl.org/perl6/doc/design/exe/E05.html:
Matching IP address:
token quad {  (\d**1..3) <?{ $1 < 256 }>  }
$str ~~ m/ <quad> <dot> <quad> <dot> <quad> <dot> <quad> /;

Matching nested parentheses:
$str =~ m/ \(  [ <-[()]> + : | <self> ]*  \) /;

Annotated:
    $str =~ m/ <'('>                # Match a literal '('
               [                    # Start a non-capturing group
                    <-[()]> +       #    Match a non-paren (repeatedly)
                    :               #    ...and never backtrack that match
               |                    # Or
                    <self>          #    Recursively match entire pattern
               ]*                   # Close group and match repeatedly
               <')'>                # Match a literal ')'
             /;

Comment: @Andrew: Did you read the question? He's not trying to find a tool to write every possible parser. He just wants to extend a very popular tool to do tasks that it can *nearly* do. For example, a standard drill is useless as a screwdriver because it only goes forward and at high speed. Once you make it reversible and variable speed, it's a great screwdriver (and nutdriver, etc.). The OP just wants to add a few features to a great tool to make it even better -- or just create a new tool that can do the same.
Comment: Was just writing about the same :-)
Comment: If you want grammars, well, Prolog had DCG's (definite clause grammars) built-in all the time, and they are strictly more powerful than context-sensitive languages (they should be something like indexed grammars, afaik)! Just that a language-specific solution is not a general solution.
Comment: Felix: No, DCGs are overkill for what the OP is looking for.
Comment: @Gabe: What the OP is looking for is a Swiss Knife™ for all his string needs. What's he forgetting however is that a Swiss Knife™ is compromised of multiple specialized tools. For XML, HTML and EMAILs, parsers exists. For integers, well, language constructs.
Comment: @Gabe: "No"? I was just saying that (1) this is language specific and therefore not general, (2) there are more powerful and still easy-to-use parsing methods, even built-in, in some languages. I might also have mentioned SNOWBALL and SNOBOL. Also, i disagree with the notion of "Overkill" for DCG's: they are about as easily written and more readable (IMHO) as your annotated Perl 6 regex.
Comment: Looks a bit like BNF meets Regex. Interesting approach.
Comment: Although this is language-specific, many languages current regex implementation is based on Perl's. Who knows, the same might happen with these new Perl 6 rules.
Answer: Don't blame the tool, blame the user.
Regular Expressions were built for matching patterns in strings. That's it.
It was not made for:

Integer validation
Markup language parsing
Very complex validation (ie.: RFC 2822)
Exact string comparison
Spelling correction
Vector computation
Genetic decoding
Miracle making
Baby saving
Finance administering
Sub-atomic partitioning
Flux capacitor activating
Warp core engaging
Time traveling
Headache inducing
Never-mind that last one. It seems that regular expressions are very well adapted to doing that last task when they are being used where they shouldn't.

Should we redesign the screwdriver because it can't nail? NO, use a hammer.
Simply use the proper tool for the task. Stop using regular expressions for tasks which they don't qualify for.

I want to match a number between 1 and 31, how do I do that?
Use your language constructs to try to convert the string to an integer and do the appropriate comparisons.
How do I match an even/odd number of occurrences of a specific string?
Regular expressions are not a string parser. You can however extract the relevant part with a regular expression if you only need to parse a sub-section of the original string.
How do I parse XML with regex?
You don't. Use a XML or a HTML parser depending on your need. Also, an XML parser can't do the job of an HTML parser (unless you have a perfectly formed XHTML document) and the reverse is also true.
How do I validate an email with regex?
You either use this large abomination or you do it properly with a parser.
Comment: Don't take this the wrong way, but this answer seems very emotional, and I think most of it is a rant. Part of our work is building better tools: the OP is inquiring about such tools, and wants to discuss their possible usefulness. In fact he is correct: many regex flavors do offer extensions that enable stronger abilities, making them a fitting tool for more tasks (not to mention code callouts, which are strictly cheating `:)`).
Comment: @Gabe: Ultimately, the logic of your validation really shouldn't be stored in an Apache config or in your database. If you do have a system with variable input, store parameters that your can parse for which your code will send to the proper validator after. For example `"int|required|min:13"` for your age validation and `"email|required"` for your email. If you do have something that has a specific format and can be easily validated using a regular expression, then by all means: `"regex|pattern:/^REF[0-9]{5,8}$/i"` but beyond that, don't.
Comment: The point is that we often find ourselves in situations where we need to do things like validate input ("Is this a valid email?", "Is this age at least 13?") and you can't stick a YACC grammar into a line in an Apache config or a `VARCHAR(100)` field in a database. It doesn't seem unreasonable to make a better tool for this kind of task.
Comment: I think Kobi's right. The problem is that many of us are often given a single tool (of which validation frameworks and Apache modules are merely 2 examples of such situations) and are forced to perform all of our tasks with that one tool. If we could choose a more versatile tool to be our one, we could get a lot more common things done without having to hire a professional (e.g. LEX & YACC, ANTLR, etc.).
Comment: I see your point, but I'm not proposing they be completely redesigned. It just seems like there's scope to improve them in a few ways, to match the things people are trying to do with them nowadays.
Comment: I think this is the right answer. Extensions such as those suggested by the OP will only encourage people to keep using the wrong tool.
Comment: So you're suggesting that when I need to create a URL rewriting rule (which must reside in my httpd.conf) that is too complex for a regex, my only options should be to change my requirements or write my own Apache module? I don't see why using a more powerful matching language shouldn't be a valid option.
Comment: @miket2e: The problem is that it shouldn't be used to do those things in the first place. It never was the right tool, and there are plenty of tools which are built exactly to do those tasks. Regular expressions are right now extremely good at what they are supposed to do. **A fork is a fork. A spoon is a spoon. You may thing you want a spork but it ain't as useful as it seems.** I definitively don't want regexes to become a spork.
Comment: He's not saying "turn regexes into a spork"; he's saying "let's build a new tool to replace regexes, which can do things regexes aren't good for".
Comment: @Gabe: I understand that... But they are already tools out there to do that kind of work. Regex is not to blame for not solving problems it wasn't meant to solve in the first place. It doesn't need replacing. People need to stop using them as an end-to-all-means solution for string data extraction. That's not why they exists. **Use specialized parsers and lexers; they were made for that kind of work.**
Comment: @Gabe: You were talking about validation, now you are talking about URL rewriting... Two different things. URL rewriting uses patterns to redirect/rewrite the request. What do you know, regular expressions were made for pattern matching, so please store away in your Apache Config. *"Is this a valid email?"*, keep elsewhere. If the rewrite depends on the validation, grab the common denominator, validate in your controller.
Comment: I think this post is kind of offencive. are you calling senior team leaders and architects users? They believe they are almost gods and are never wrong. I'm shocked you did not get down-voted to hell, but maybe they don't use SO. Most of the time people are using regex, because they have to not because they want to.
Answer: All of those are reasonably possible in Perl.
To match a 1..31 with a regex pattern:
/( [0-9]+ ) (?(?{ $^N < 1 && $^N > 31 })(*FAIL)) /x

To generate something like [1-9]|[12]\d|3[01]:
use Regexp::Assemble qw( );
my $ra = Regexp::Assemble->new();
$ra->add($_) for (1..31);
my $re = $ra->re;                 # qr/(?:[456789]|3[01]?|1\d?|2\d?)/

Perl 5.10+ uses tries to optimise alternations, so the following should be sufficient:
my $re = join '|', 1..31;
$re = qr/$re/;

To match an even number of occurrences:
/ (?: pat{2} )* /x

To match an odd number of occurrences:
/ pat (?: pat{2} )* /x

Pattern negative match:
/<name> (.*?) </name>/x  # Non-greedy matching

/<name> ( (?: (?!</name>). )* ) </name>/x

To get a pattern matching email addresses:
use Regexp::Common qw( Email::Address );
/$RE{Email}{Address}/

Answer: Probably it is already there and from a long time ago. It's called "grammars". Ever heard of yacc and lex ? Now there is a need for something simple. As strange it may appear, the big strength of regex is that they are very simple to write on the spot.
I believe in some (but large) specialized areas there is already what is needed. I'm thinking of XPath syntax.
Is there a larger (not limited to XML but still simple) alternative around that could cover all cases ? Maybe you should take a look at perl 6 grammars.
Answer: No. We should leave regular expressions as is. They are already far too complicated. When was the last time you thought you had nailed it, i.e., got the whole extended regex syntax (choose your flavour) loaded in your squashy memory?
The theory behind regexes is nice and simple. But then we wanted this and that to go with it. The tool is useful, but falls short on non-regular matching. That is ok!
What most people miss, is that context-free grammars and little specialized interpreters are really easy to write.
Instead of making regexes more difficult, we should be rooting for parser support in standard libraries for our languages of choice!

A successor to regex?


Exchange 2007 - Restore mailbox's calendar and contacts

Question: I know it is possible to do a restore of a user's mailbox. But does anyone have a really simple way of doing this? Like an Exchange console command or something?
Also, if I understood this correctly, the restore is put in a restore container of some sort? How easy is it to restore that content into the actual mailbox I'm trying to recover?
Answer: It almost sounds like you're asking two different things here.
Restoring a mailbox requires a backup, and the directions differ wildly based on what backup product you use.
Individual deleted items can be recovered as long as you have Deleted Items Retention set.  They can be recovered through Outlook.  Anything past the Deleted Items Retention will require a backup to restore from.
Looking at it again, calendar and contacts are treated a little differently and my experience has been that those need to be recovered from a backup as well.


How to get IpAddress and UserAgent in ASP.NET Web API get methods

Question: I am using ASP.NET Web Api to expose a few GET methods.
But before I return the data I need to log a couple of details to the db, of which few of them are as listed below :

Caller's Ip
Caller's User Agent
Caller's Used Url

Now in the controller when I used to do this I used to use the following code,
var ipAddress = Request.ServerVariables["REMOTE_ADDR"];
var userAgent = Request.UserAgent;

But here in Web API I am unable to use this.
Can anyone please help me out with this.
Answer: I figured it out,
public static LogModel GetApiLogDetails()
{
    var logModel = new LogModel();
    logModel.TimeStamp   = DateTime.Now;
    logModel.CallerIp    = HttpContext.Current.Request.UserHostAddress;
    logModel.CallerAgent = HttpContext.Current.Request.UserAgent;
    logModel.CalledUrl   = HttpContext.Current.Request.Url.OriginalString;
    return logModel;
}

with a little help from
Get Web Api consumer IP Address and HostName in ASP.NET Web API
&
Get the IP address of the remote host
Comment: I would try and avoid HttpContext as it only works in WebHost.
Comment: @DarrelMiller: Since this question has come up many times, do you think Web API should have an extension on HttpRequestMessage to get the client IP address? Like the System.Web.Http.WebHost and System.Web.Http.Owin.Selfhost can create request message extensions which users could use.
Comment: @KiranChalla  It would be a fairly low impact change that would definitely help a lot of people.
Comment: @DarrelMiller: Sure, created an issue here: https://aspnetwebstack.codeplex.com/workitem/1116
Comment: How-to add System.Web to NET 6 Minimal API ? good pattern ?
Answer: You should use HttpRequestMessage class, that conteins all data you need.
Read more:

How to get IP adress
Extension methods for HttpRequestMessage
MSDN
Comment: How-to use HttpRequestMessage in Minimal API NET 6?


Como Alterar Idioma Padrão de Mensagem de Validação de Uma Aplicação ASP. NET MVC

Question: Tenho um modelo assim:
[Display(Name = "Protocolos Abertos")]
public int ProtocolosId { get; set; }

que utilizo em um SelectList como o value, quando vou fazer o submit de um formulário a mensagem de validação é retornada em idiomas diferentes para localhost e para o servidor de homologação.
- Servidor de Teste:

- Localhost:

a web.config está configurada:
<globalization culture="pt-BR" uiCulture="pt-BR" />

Existe alguma outra configuração na aplicação a ser realizada para efetuar essa alteração de inglês para português?
OBS:
 Sei que é possível criar mensagens de erro customizadas para os campos, porém a mensagem padrão é bem útil.
Comment: Essa validação é a do backend ou do frontend?
Comment: Acredito que seja no frontend, pois nem chega no método para tratar o post.
Comment: Utilizo `@Html.DropDownListFor(model => model.ProtocolId, Model.ProtocolsList,"Selecione um protocolo", new { @id = "ProtocolId" })` quando está em "Selecione um protocolo" não há nenhum valor em `ProtocolId`, o que ocasiona essa mensagem, pois o mesmo é do tipo `int`
Answer: A solução que encontrei para o problema foi instalar o language package correspondente ao idioma desejado direto no servidor.
Após isso as mensagens de validação começaram a aparecer no idioma correto.


How to identify a case of overfitting using stratified k fold cross validation?

Question:
I am working on a problem with a very small amount of data of 211 examples. The problem is a binary classification problem with 2 sets of classes. The data is highly imbalanced with 84% being the majority class and 16% being the minority class.
I am using logistic regression for classification and using f1 score as an evaluation metric. I did stratified k fold cross validation with 5 folds. I divided my data into train and holdout. I am using train in cv
and holdout to see how good my model is generalising on unseen data.
f1_score_list = []
f1_score_holdout = []
f1_score_train = []
folds = []
model = LogisticRegression(random_state=RANDOM_STATE)
skf = StratifiedKFold(n_splits=5, random_state=RANDOM_STATE, shuffle=True)
for i, (train_index, test_index) in enumerate(skf.split(X, y)):
    X_train, X_valid = X[train_index], X[test_index]
    y_train, y_valid = y[train_index], y[test_index]
    model.fit(X_train, y_train)
    train_pred = model.predict(X_train)
    print('Logistic Regression, training set, fold ', i, ': ', f1_score(y_train, train_pred))

    pred = model.predict(X_valid)
    #Measure of the fit of your model.
    print('Logistic Regressaion, validation set, fold ', i, ': ', f1_score(y_valid, pred))

    # DATA WHICH MODEL HAS NOT SEEN
    pred_holdout = model.predict(X_holdout)
    print('Logsitic Regression, holdout set, fold ', i, ': ', f1_score(y_holdout, pred_holdout))
    print('Prediction length on validation set, Logistic Regression, fold ', i, ': ', len(pred))

    folds.append(i)
    f1_score_list.append(f1_score(y_valid, pred))
    f1_score_holdout.append(f1_score(y_holdout, pred_holdout))
    f1_score_train.append(f1_score(y_train, train_pred))

print ('train f1_score', np.mean(f1_score_train))
print ('cross-val f1_score', np.mean(f1_score_list))
print ('hold out score', np.mean(f1_score_holdout))
plt.plot(folds, f1_score_list, label = 'validation score')
plt.plot(folds, f1_score_holdout, label='holdout score')
plt.plot(folds, f1_score_train, label='training score')
plt.legend()
plt.show()

The output is listed below.
Logistic Regression, training set, fold  0 :  1.0
Logistic Regression, validation set, fold  0 :  0.7692307692307693
Logsitic Regression, holdout set, fold  0 :  0.9166666666666666
Prediction length on validation set, Logistic Regression, fold  0 :  30
Logistic Regression, training set, fold  1 :  1.0
Logistic Regression, validation set, fold  1 :  0.8333333333333333
Logsitic Regression, holdout set, fold  1 :  0.9166666666666666
Prediction length on validation set, Logistic Regression, fold  1 :  30
Logistic Regression, training set, fold  2 :  1.0
Logistic Regression, validation set, fold  2 :  0.9090909090909091
Logsitic Regression, holdout set, fold  2 :  0.9565217391304348
Prediction length on validation set, Logistic Regression, fold  2 :  30
Logistic Regression, training set, fold  3 :  1.0
Logistic Regression, validation set, fold  3 :  1.0
Logsitic Regression, holdout set, fold  3 :  1.0
Prediction length on validation set, Logistic Regression, fold  3 :  29
Logistic Regression, training set, fold  4 :  1.0
Logistic Regression, validation set, fold  4 :  0.888888888888889
Logsitic Regression, holdout set, fold  4 :  0.9166666666666666
Prediction length on validation set, Logistic Regression, fold  4 :  28
train f1_score 1.0
cross-val f1_score 0.8801087801087801
hold out score 0.941304347826087

Based on these results how can I identify whether model is overfitting or not. Any suggestions in this regard would be helpful.
Answer: You can check how your model is performing when predicting the training samples on each step of the k-fold cross validation. If the metric is higher on the training samples than on validation or holdout samples, it could be a symptom that the model is overfitting.
Comment: from the training samples, you mean I should make predictions on the same data I am fitting to the model and then calculating training score ?
Comment: yes. measure f1. it should be lower than the metrics for vallidation and holdout
Comment: I have edit my question to add more information of training f1 score.
Comment: Your graphic shows that the model is slightly overfitting (But f1 score in holdout score is close to 100%, so its not a problem)
Comment: yes I tested on new data and most of the predictions are correct, I will try with different parameters to see it generalises well. thanks for the suggestion :)


How many combinations can I make?

Question: let $n \gt 1$ be an integer, and consider $n$ people; $P_1, P_2,..., P_n$ let $A_n$ be the number of ways these $n$ people can be divided into groups, such that each group have either one or two people
determine $A_1, A_2, A_3$
So I have
$A_1 = 1$ way
$A_2 = 4$ way
$A_3 = 12$ way
But I am not sure if this is right... Can anyone confirm?
edit:
My way of getting this is
$A_1 = 1 $ because {P1} only has 1 way to sort
$A_2 = 4$ because {P1,P2} = {P1,P2},{P2,P1},{P1P2},{P2P1}
etc.
Comment: The groups $\{P_1,P_2\}$ and $\{P_2,P_1\}$ are the same group: the order in which you list its members doesn't change the identity of the group. There are actually just two ways to do it: you can put both people in one group, getting the group $\{P_1,P_2\}$, or you can put each person into a separate group, getting the two groups $\{P_1\}$ and $\{P_2\}$.
Comment: What are the $4$ ways that you found to divide $P_1$ and $P_2$ into groups?
Comment: Here $A_n$ actually represent the number of ways the n people can be divided into groups. So $A_1$ represents division of people such that a group only include one person but not only P1 and similarly $A_2$ represents division of people such that a group only includes two person but not only $P_1$ and $P_2$ and .......
Comment: For a more general discussion, [please see this.](http://math.stackexchange.com/questions/964367/counting-the-number-of-different-ways-in-which-groups-of-one-or-two-can-be-forme)
Answer: The only case that hasn't been handled in the comments is $n=3$. If each group must have either one or two people, and you have three people altogether, there are two basic types of groupings: you can put each person into his own group, or you can have one group of two people and one singleton. The first possibility gives you the groups $\{P_1\},\{P_2\}$, and $\{P_3\}$. There are several ways to get a group of two and a singleton; can you see what they are?

How many combinations can I make?


Adding Office 365 apps to an existing Azure AD Free

Question: I have an Azure AD Free subscription that has users. I want these users to have Office 365 (Essentially Outlook).
I want to manage this from my existing Azure if possible and I don't have an Office 365 account at this stage as I want to avoid costly merge problems if there is a standard way.
All the resources online are assuming you already have an Office 365 subscription!
What do I need to do if I already have an Azure account and no Office 365?
Answer: Basically, you need to buy Office 365 licenses and assign them to your Azure AD users.
If you want actual services, such as Exchange Online, SharePoint Online and/or Teams, you'll also need to set them up.
Depending on your environment, synchronization between Azure AD and on-premises Active Directory may or may not be required.
Comment: I don't have anything on-premise and I am only interested in Office 365, how do I buy the licences and assign them?
Comment: You can buy office 365 licences by going to the licences tab in Azure AD and clicking add
Comment: @SamCogan There is no reference what so ever to Office in this location.
Comment: There is a link on the licences page to the "purchase services catalog", but as you can't see it, the direct link is https://admin.microsoft.com/AdminPortal/Home?ref=/catalog


Proof verification [Function with finitely many discontinuities is integrable]

Question: Today at the end of class, our professor wrote this theorem on the blackboard. Could someone verify if my proof is correct?

Let $f:[a,b]\rightarrow\mathbf{R}$ be bounded with finitely many discontinuities $\{x_1,\ldots, x_n\}$. Prove that $f$ is integrable.

Let $I_1=[a,x_1-\delta]$, $I_2=[x_1-\delta,x_1+\delta]$, $\ldots$, $I_{k-1}=[x_n-\delta,x_n+\delta]$, $I_k=[x_n+\delta,b]$ and let $\delta=\frac{\varepsilon}{4Mk}$.
Let $\varepsilon>0$ be given. This gives rise to partitions $P_i$ for the intervals in which $f$ is continuous. In these intervals, we have $$U(f,P_i)-L(f,P_i)<\frac{\varepsilon}{2(k+1)}.$$ (In the remaining intervals, we introduce another partition.)
Let us consider the partition $P:=\bigcup_{i=1}^k P_i$. Since $P$ is a refinement of $P_i$, we have $U(f,P)\leqslant U(f,P_i)$ for all $i$; therefore $kU(f,P)\leqslant \sum_{i}U(f,P_i)$. Repeating the same procedure, we also have $I(f,P)\geqslant L(f,P_i)$ for all $i$; so $-kL(f,p)\leqslant \sum_{i}L(f,P_i)$. Therefore
$$U(f,P)-L(f,P)\leqslant \frac{1}{k}\sum_{i=1}^k U(f,P_i)-L(f,P_i)$$
We wish to show that $U(f,P)-L(f,P)<\varepsilon$. For all intervals which don't contain a point of discontinuity, we already have $U(f,P_i)-L(f,P_i)<\frac{\varepsilon}{2(k+1)}$. In the remaining intervals, we perform a trick.
Since $f$ is bounded, we have $|f(x)|\leqslant M$ for some $M>0$.
Consider a partition $P_i$ of an interval which contains a point of discontinuity. We have
$$U(f,P_i)-L(f,P_i)=\sum_{p=1}^q (\sup f(x)-\inf f(x))\cdot (t_p-t_{p-1})\leqslant 2M\sum_{p=1}^q (t_p-t_{p-1})=2M\cdot 2\delta=4M\delta$$
Combining these (by an induction argument there are exactly $\frac{k-1}{2}$ intervals in which $f$ is discontinuous, and $\frac{k+1}{2}$ intervals in which $f$ is continuous), we get:
$$U(f,P)-L(f,P)< \frac{1}{k}\left(\frac{k+1}{2}\frac{\epsilon}{2(k+1)}+\frac{k-1}{2}\cdot 4M\delta \right)<\frac{\varepsilon}{2}+2M\delta=\frac{\varepsilon}{2}+\frac{\varepsilon}{2}=\varepsilon.$$
$\square$
Answer: Your claim that $U(f,P) \leq U(f,P_i)$ does not hold since $P$ is not a refinement of $P_i$. They are actually partitions on different sets: $P_i$ is a partition on one single interval and $P$ on the union of these intervals.