Being limited to the natural minor/major, relative minor/major and parallel key

Question: I'm not entirely sure if this question would be to broad. But say there is a chord progression in A minor. The key I have to play in would be A aeolian wich resolves to the following modes:

i A Aeolian (minor)
ii B Locrian (minor)
III C Ionian (major)
iv D Dorian (minor)
v E Phrygian (minor)
VI F Lydian (major)
VII G Mixolydian (major)

Now I know I can break the rules and play the relative major. Understandable because these are the same notes. From research I also saw I can play in the parallel key, which resolves to A major.
Am I limited to the notes in this scale or can I use more scales? I hear some people talking about taking notes from the Chromatic scale, but these contains all the notes from one note to another in the next octave. This doesn't make sense to me.
For an example I found a song by Pantera; Cowboys from Hell. This is tuned in Standard E. The notes, therefor chords of the intro riff are E, G, A, A#, B, D.
This fits in the D major key, as this site says when checking for the key.
However, when looking at this A# note. This is not in the D minor key. But is seen as the augmented 5th as the intervals. Can I still use this note in this key? How does this work? This note is however in the D minor key.
Also, when looking at the solo build up in the tablature below, there are a few notes that don't fit in the key or parralel key: A, D# and G#. Where do these notes come from?

Can I use more scales belonging to D major? When can I use the harmonic minor scale as an example?
When checking for the scales in my guitar pro software. It says that the riff uses the Blues Minor Hexatonic scale. When can I use this? There are so much. Can't I just lookup scales which belong to this scale and play that?
Comment: Your software is silly and is over-complicating it: that's just an E blues scale. Also ditch the software and learn how to build / identify scales yourself! It is a crutch! Last, you can do whatever you want; that said, you should impose limits on yourself for each piece in order to focus and foster creativity.
Comment: I agree with jjmusicnotes. There is so much 'helpful' stuff out there that if we all relied on it, we'd expect it to wipe our noses too. Get into the real world, listen to what's happening, and react accordingly. DO NOT rely on Micky Mouse sites which are doing more harm than good: your question proves this! No-one's even been arrested for using a different set of notes for a tune than the 'theory' says. 'When can I use this?' Whenever you want. If people only went by the 'rules' we'd still be playing nursery rhymes on Penny whistles. And I know she does - I've heard her!
Comment: Your first para. uses false premises. The key of A Aeolian *does not* resolve to any of the other modes/keys. The purpose of a key is having a home centre to which it gravitates. There cannot be any gravitation from A Aeolian to, say, D Dorian - their roots are different, even though they both use the exact same notes.
Comment: You don't HAVE to gravitate from A Aeolian to D Dorian.  You're there already!
Comment: @jjmusicnotes Well you're right, I should ditch it. But how would you identify that as that the blues scale? Isn't there any place I can start so I can start learning all of this? Or do I need to read lots and lots of music?
Comment: @Bas - You look at the way their built; blues scales have the following: 1, b3, 4, #4/b5, 5, b7. You see a scale that looks like that and you know it's blues. Learn how to build your majors / minors / modes / ethnic scales / jazz scales / etc. Learn the framework for all so that when you're working you can just fill in the blanks with the necessary notes
Comment: But, like I said, this may be looked at as a much simpler scale, with a chromatic auxilary note introducing each fragment.
Answer: Keys and modes are frameworks, not restrictions.  You may use diatonic notes and chords (ones that use the notes of the basic scale of the key), or you may use chromatic notes and chords (everything else).   You may freely move into other keys, and much music doesn't really analyse as being in any particular key at all!
Also, don't worry about all those different modes and scales.  They're all the same scale, just starting on different notes. No need to waste time learning them all seperately.
There you are - your entire concept of 'theory' thrown away in a couple of paragraphs!   Read and play lots of music.  Use theory to make sense of what music DOES do, not to tell you what it MAY do.  There are no rules, but there are lots of descriptions of things that work.  Theory Describes, It Does Not Command.
Looking at your music example, we have a sequence of minor 3rd intervals - E-G, A-C, D-F, G-Bb (rather confusingly spelled as A#)  with a semitone below leading in to each of them.   I suggest you consider only the second and third notes of each group as harmonic, the first as a chromatic decoration.  One scale that contains all the harmonic notes would be F major.  There are other possibilities. If you want a chord sequence it could be Em, Am, Dm, Gm.   Or C, F, Bb, Eb.  Or Am7, Dm7, Gm7, Cm7.   Or something less symmetrical.
Comment: Oh, and forget about all the mode names and other fancy words. You can come back to them in say 10 or 20 years time, but never forget that music is meant for the **ears**, not for the eyes to read complicated descriptions of it.
Comment: But I came here to make sense out of that piece of music. There must be any logic behind that right? What if created a similar riff on another place on the fretboard and wanted to see which key it was in?
Comment: It sounds abit to filosofic to me..
Comment: I've expanded my answer above.   You're right - don't get too philosophical about it!  How does labelling it with a key help you play this music?
Comment: I got confused even more. Can songs be in more then one key? When labelling a key to a song it helps me choose chords so I can play along or find my places for riffing! I also updated my answer.
Comment: Aren't there just a set of scales belonging to a minor or major scale that I can look up and use? Such as this crazy blues scale?
Comment: The sort of music I think we're talking about will tend to have a key centre, though it might change during the song!   But even when there IS a firm key centre, non-diatonic chords are common.  Take the simple example of a song in C major which contains both G7 and Bb major chords (the Beatles loved doing that).   Don't get hung up over there being both Bb and B natural notes.  No, they're not both in the same scale.  No, we haven't modulated.  We're allowed to walk on the cracks of the paving stones!
Comment: Do you mean "Just feel it and do whatever sounds cool"? What about my question about choosing this Blues Minor Hexatonic scale. WIll this always fit over a minor scale?
Comment: Most of the stuff that 'sounds cool' HAS been catalogued and recognized by 'theory'.  But there's so MUCH that sounds cool :-)    Don't work outwards from theory, but inwards from existing music.  Read, play, listen.  When you hear something cool, incorporate it into YOUR 'theory' book so you can do it again later.  I'm going to bed now.  Enjoy!
Comment: "Do you mean "Just feel it and do whatever sounds cool"?" - Yes, yes, yes, yes!!!! Many beginners try to find the rabbit-hole that "contains all the rules". It doesn't exist. Think about it - if it **did** exist, nobody could ever write a "new" piece of music, because everything would already have been included in the existing "rules".
Comment: "I hear some people talking about taking notes from the Chromatic scale, but these contains all the notes from one note to another in the next octave. This doesn't make sense to me." Well, it only "doesn't make sense" because you are trying to find a **theory** where it doesn't make sense. If the theory doesn't fit the music, and the music sounds OK, then the music is right and the theory is wrong! Get hold of the idea that **every** key and scale contains **all twelve notes** - it's just that in any particular scale, some of the notes are more common, and/or more important, than others.
Comment: That's a good filosofy, I'm going to have to listen more. But say, what if I wanted to transpose this song. How can I get this same _sound_ of the `A#` note but in a different key? I still don't know where it comes from. Or did it come from the fact that it sounded cool?
Comment: @alephzero - that rabbit hole is often a pigeon hole.
Answer: That set of notes - E, G, A, Bb, B and D constitute what we call minor blues. Play them in ascending, descending order, and you play the Em blues scale. The A#/Bb is a tritone from the root of E, and gives that bitter-sweet sound. In blues, it's become known as flat five, thus Bb here, whereas in jazz the same interval/note is known as sharp four.
It's a set of notes with its own 'theory' - note, not rules, and must feature on many thousands of songs from the '50s on.
To an extent, it produces its own set of chords, using its own notes, obviously. However, although it's essentially minor, it has been found to work over the parallel major key chords, mainly I, IV and V.One really effective chord used occasionally is E7b9 - aka the Hendrix chord, although many others used it after, and before. The b9 sounds like the m3, so it appears to contain both major3 and 'minor3'.
You say you found the set of notes fit into D/Dm. But not the A#. That's cos Dm uses Bb instead of A#. Some of these 'helpful' sites aren't that helpful, and are short on actual facts - or worse - give misinformation. It ain't necessarily so, just cos it's on www!
PLEASE don't try to compartmentalise everything. It just doesn't work in music. The oft quoted 'if it sounds good, it probably is' is a far safer way of regarding something in music. So many questions here revolve round so-called 'rules', which then trip the followers up. A quick analogy - did you have to learn about and understand how the gearbox works, before you could successfully drive the car? Same principle in music. I hope you answered 'not really'...
EDIT: that site stating it's in D or Bm is stating rubbish! Listening to the song puts E very clearly as 'home'. I can to a degree understand why it states D, as the notes from D are included. That's not even half of the picture though. An interesting exercise I use with students is to get them to use, say, D major scale notes, over me playing in E. Because D uses G instead of G# from key E, and D instead of D# in key E, the overall effect is playing a sort of blues. Try it.
Comment: I'm feeling like I'm going to need some music lessons or a good way to start my theory lessons. I'm so overwhelmed by all this. You can't just _feel_ something when it sounds bad when going outside the scale. Where do I start? Because I assume I need to start with some rules before I can do all of this? Do you mean you can just play in D major and play notes from E major over the D major scale?
Comment: I always hear that there is a 'corresponding chord' to every note in a scale. What would be that chord then if you would take the first note of that solo build up? In this case that's the `D#`
Comment: No,I mean using D scale notes while a rhythm is being played in E, as in using E, A , B as the accompanying chords.There may be a 'corresponding chord' to each scale note, but that's certainly not how we find the chords to a song.Take the note E, in key/scale E.Chords that will fit under that E are E, A, F#m7, C#m - as they all contain the note E.Obviously the  E chord is the most convincing, but you can't say that because there's an E note in a tune, that the best fit chord will be E.The D# in the solo is a passing note, used chromatically, not of much consequence, key-wise, but sounds good.


Como se define as vulnerabilidades de uma linguagem de programação?

Question: Essa pergunta pode parecer um pouco genérica e abrangente, mas acho necessária. Li em um artigo sobre as muitas vulnerabilidades da linguagem C, porém não entendi o que torna a linguagem em si vulnerável.

São as estruturas de dados que ela suporta ou não suporta?
Essas vulnerabilidades estão relacionadas com o tipo de aplicação?
Existe alguma métrica para isso?
Answer: É a existência de certas características. Ela é muito liberal em várias coisas e exige que o programador faça tudo certo. Não é que ela seja vulnerável, ela só exige que a pessoa saiba bem o que está fazendo, ela não te protege muito. E as linguagens são todas suscetíveis a algumas falhas. Não caia no marketing de alguma linguagem ser uma maravilha.
C faz você controlar toda memória na mão, e isso não é fácil, se você falhar em algum caso, dará problema.
C permite acessar a memória indiscriminadamente, e até onde não devia, é o programador que deve cuidar para que isso aconteça só do jeito certo.
C tem muito comportamento indefinido e o programador tem que saber bem disso e cuidar para que aconteça o que ele espera em cada plataforma.
C tem algumas funções da biblioteca padrão que são consideradas inseguras, então deve não usá-las ou usar com mais cuidado.
Tudo isso dá vantagens para C que outras linguagens não costumam ter.
Comment: Muito obrigado pela resposta, mas ainda assim tenho dúvidas. Por exemplo, quando você diz de características que tornam a linguagem insegura, quais seriam elas ? E quais falhas são mais suscetíveis  em cada linguagem ? Sei que isso parece meio abrangente e talvez não tenha uma única resposta para ela, mas o que eu mais gostaria de saber qual é o fator determinante pra escolher uma linguagem de programação em determinado projeto.
Comment: A respeito das bibliotecas inseguras, imagino que você esteja falando das bibliotecas de alocação dinâmica de memória, correto ? Em todo lugar que eu vejo sempre tem algum aviso sobre isso, parece até "magia negra".


host configuration is enabled by unknown instead of site administrator

Question: I changed an Apache reverse proxy configuration, to add a new hostname.
I don't usually manage webservers / proxies, though I basically replicated the existing structure:

created a separate mysite.com.conf file with the relevant VirtualHost directives inside the sites-available folder
created a symlink named mysite.com.conf in the sites-enabled folder pointing toward the new file
restarted Apache with systemctl restart apache2

Everything works as expected, the website is reachable and such. There's only a small thing that puzzles me: if I do a2query -s I can see the new host I added in the command output.
But, while all the others are listed like:

www.example1.com (enabled by site administrator)
www.example2.com (enabled by site administrator)
...

Mine is instead:

mysite.com (enabled by unknown)

From where does that unknown come from and how do I fix it?
Answer: On Debian (and so, on Ubuntu as well), the preferred way to create sites is to create them in the /etc/apache2/sites-available directory (named something.conf), then run the a2ensite something command, where something is the name of the config file without the extension.
Similarly, enabling config snippets (these are in the /etc/apache2/conf-available directory) and modules (from the mods-available directory) should be done by using the a2enconf and a2enmod commands.
These commands have a corresponding "disable" pair, named a2dissite, a2disconf and a2dismod, and they write a status to the /var/lib/apache2 directory, from which the a2query script reads the info. If there is a config file which doesn't have an entry in /var/lib/apache, then it is there by "unknown".


How to set up a "public" DNS server on linux server?

Question: I need to set up a DNS server to be used with OpenVPN. I can not use conventional public DNS IPs (Like Google public DNS ones) because all of them are blocked in the ISPs where the OpenVPN clients are going to be used,  hence need to define my own.
I appreciate your help.
P.S. I know how to define an authoritative DNS server using nsd3, but nsd3 is authoritative-only and I guess here what I need is a recursive DNS server. I chosen nsd3 after having enormous complexity issues with BIND9, so I hope you can suggest some solution that does not need BIND9.
P.S.2 I just found a nice comparison of DNS servers here on wiki. So I basically know what DNS servers CAN do the job. What I need is HOW to do so.
Answer: DNSMasq can do this: http://thekelleys.org.uk/dnsmasq/doc.html
For RHEL based distros, you can do something like this:
<code>yum install -y dnsmasq
chkconfig dnsmasq on
service dnsmasq start
</code>
Then you should be able to point your clients at the IP of the server you just installed it on, and have DNS work nicely.  Make sure the server has valid nameservers in /etc/resolv.conf though.
Comment: Does DNSMasq have a .deb package too? How can I make valid nameservers? Thanks
Comment: It probably does, try searching for it with aptitude (or apt-cache).  You need to include nameserver lines in your /etc/resolv.conf.  This is typically done automatically (if you can access the internet from the machine you are going to run this on, your DNS servers are already set correctly)
Comment: Yup:

    apt-cache search dnsmasq
    dnsmasq-base - A small caching DNS proxy and DHCP/TFTP server
    dnsmasq - A small caching DNS proxy and DHCP/TFTP server
    openresolv - management framework for resolv.conf


How to include HTML badges as labels in column chart?

Question:

var xValue = ['1','2','3','4','5','6','7','8','9','10','11','12','13'];
var yValue = [10, 6, 8,9,1,17,3,7,13,2,12,4,13];
var wicket1 = ['W','','W','W','','','','W','','','','','W'];

var trace1 = {
  x: xValue,
  y: yValue,
  type: 'bar',
  text: wicket1,
  textposition: 'outside',
};

var data1 = [trace1];

Plotly.newPlot('myDiv', data1);

Comment: what do you mean with *badges*?
Comment: I need to replace 'W' with W
Comment: It depends on the specific chart library that you are using. You should probably be asking this on their Github page.
Answer: I hope this is what you are looking for, I am not able to make it work using class, but if I add the styles inline it works great, hope that is acceptable.
Only some styles seem to work, so place the styles of "badge" as inline styles and see if your required output is produced.

var xValue = ['1','2','3','4','5','6','7','8','9','10','11','12','13'];
var yValue = [10, 6, 8,9,1,17,3,7,13,2,12,4,13];
var wicket1 = ['W','','W','W','','','','W','','','','','W'].map(function(el) {
return ''+el+''
});;

var trace1 = {
  x: xValue,
  y: yValue,
  type: 'bar',
  text: wicket1,
  textposition: 'outside',
};

var data1 = [trace1];

Plotly.newPlot('myDiv', data1);

Comment: @FaLIL Does this answer help you
Answer: Yes it is possible, the script creates a SVG file. Inside these g-tags you can place text and images. But you need to do it in SVG. (text=<text x="0" y="15" fill="red">Your text here</text>)
Comment: may i get a sample code to generate a simple column chart using SVG??
Comment: You can't just make a column chart using SVG, but you can add text to them using js
Comment: cant make a column chart using SVG?? @North-Wind
Comment: You need JS to make a SVG (scalable vector graphics). Plotly makes a SVG, and you can change it using JS.
Answer: I hope this will help you a little further... sorry if not.

var xValue = ['1','2','3','4','5','6','7','8','9','10','11','12','13'];
var yValue = [10, 6, 8,9,1,17,3,7,13,2,12,4,13];
var wicket1 = ['W','','W','W','','','','W','','','','','W'];

var trace1 = {
  x: xValue,
  y: yValue,
  type: 'bar',
  text: wicket1,
  textposition: 'outside',
};

var data1 = [trace1];

Plotly.newPlot('myDiv', data1);

var container = document.getElementsByClassName("bartext")[0];
container.textContent = "Label";


No volume and unconnected wires

Question: I have a LTD ESP Snakebyte and the bridge volume knob is overturning and I think it took one of the wires out when I turned it too much. I checked the wires that go to the input and both of them are in the right spot. However theres a red wire that splits off into three different sections and it's not connected to anything.
Comment: Do you know anything about electronics? Do you know how to solder and do you have a soldering iron? If the answer to any of these is "no", your best option is to take it to a guitar tech or luthier to have it fixed.
Comment: Hi, and welcome! It would help if you could include some more information about your issue, and maybe a picture showing the problem. It's difficult to understand exactly what's wrong at the moment.
Comment: I agree wholeheartedly with @ToddWilcox. I would suggest that you learn to solder and do it yourself, except the Snakebyte is an expensive model and damage is a risk. Good luck!
Comment: Given the limited information, Todd's comment is as good as an answer.
Answer: This is a common problem I see come through the repair shop. The nut holding the pot loosens, allowing the pot to rotate out of position from where it was mounted which can break off the wires at the solder joint.
As Todd answered in the comments, unless you know how to solder and follow a wiring diagram, the best thing to do is to take the guitar to a technician and have the pot reconnected and tightened.
Answer: Some pickups have wires that aren't used, except for certain applications. If the guitar was working prior to the Pot being banjo'd then the problem lies with the pot. change it out. they are inexpensive and easy to swap out.

No volume and unconnected wires


If a skew-symmetric real matrix has all eigenvalues zero, must it be the zero matrix?

Question: This can be easily verified for $2\times2$ and $3\times3$ matrices, but can the result be generalised?
Comment: The answer is yes. One way to see this is to consider the trace of $A^2$ where $A$ is the skew-symmetric matrix in question.
Answer: If the matrix has real entries: Any matrix with all eigenvalues zero is nilpotent, the only nilpotent skew-symmetric matrix is the zero matrix.
Comment: Begs the question, why the only nilpotent skew-symmetric matrix is the zero matrix.
Comment: So now the question is why the only nilpotent symmetric matrix is the zero matrix. But of course symmetric matrices are diagonalizable.
Comment: https://math.stackexchange.com/questions/478470/is-the-zero-matrix-the-only-symmetric-nilpotent-matrix-with-real-values
Comment: $A^2$ is symmetric


Magento: qual a melhor versão?

Question: Boa tarde, estou iniciando os estudos no e-commerce Magento e gostaria de saber de quem já tem alguma experiência com esta plataforma qual a melhor versão para ser utilizada. No caso, esta ultima versão 2.0.7 seria uma boa alternativa ou devo utilizar alguma versão anterior?
Obrigada.
Comment: A versão dois do Magento está muito boa - gostei bastante. Mas, comercialmente não consegui utilizá-la. Um ponto que deve considerar é a disponibilidade de módulos. Tivemos dificuldades em encontrar módulos de pagamento e frete para os gateways brasileiros. Comercialmente ainda recomendo a 1.9.2.x.
Comment: não entendi o porque da negativação da pergunta, fiz apenas porque não consegui encontrar publicações recentes sobre a última versão.
Comment: Quem o fez, deve pensar que as respostas aqui podem ser baseadas em opinião e não em fatos.
Comment: a mais instavel sempre ou a que voce tem mais feicão
Comment: @cbonomini obrigada pela resposta, estou em dúvida justamente por essa disponibilidade de módulos de pagamento, depois que instalei a última versão fiz uma pesquisa rápida e parece que está inviável comercialmente.
Answer: Evite iniciar um negócio novo sob uma versão antiga de uma plataforma pois normalmente são defasadas, sem garantias e suporte. Terá ainda um custo de tempo e dinheiro relativamente grande com upgrades, sem contar os problemas oriundos de bugs que normalmente são corrigidos nas versões mais recentes.
Note que isso não vale somente para o Magento. Isso é válido para quaisquer plataformas.
Pode existir exceções onde será forçado a usar uma versão antiga, mas isso é muito raro e pode ser resolvido de formas mais inteligentes para que evite ao máximo o uso de algo defasado.
Adicionalmente, evite também "abandonar" a plataforma. Sempre que surgir um upgrade, faça-o num ambiente de testes e então atualize no ambiente de produção.
Portanto, uma vez instalado, precisa de manutenção constante.
Essas plataformas populares estão sempre lançando novos updates numa média de 1 a cada 2 meses. Cada update desses traz mudanças diversas e normalmente correções contra bugs e também podem causar novos bugs que devem ser corrigidos o mais rápido possível.
Não deixe para fazer um upgrade depois de 1, 2, 3 anos, por exemplo, porque normalmente torna-se um pesadelo devido ao acúmulo de bugs inesperados.
Answer: Pelo que presenciei de clientes e até cheguei a mecher com o Magento, salvo engano até fevereiro de 2016 o Magendo 1.9.1 era usado por grandes empresas(Walmart,Nike,ford). Porém as empresas de pagamento também tem que se readaptar para usar o novo 2.x. Então eu recomendo a 2.x pois logo as grandes empresas que utilizam devem migrar também, pois os plugins podem não ser mais criados pelos desenvolvedores e pela versão community, acarretando um custo maior para continuação da plataforma.
Validando também que existe integração do 1.9 para a 2.x para quem ainda é usuário da 1.9.

Magento: qual a melhor versão?


Predicate logic, Every boy who loves a girl is also loved by a girl

Question: I'm doing some selfstudying and I'm lacking the anwsers to check if I'm correct. So this is why I'm on here so frequently, as I really want to understand the matter.
So here is the sentence I'm trying to convert to predicate logic:
Every boy who loves a girl is also loved by some girl.
DoD = Humans
Here's what I got:
$\forall x(Bx \wedge \exists y(Gy\wedge Lxy))\rightarrow \exists z(Gz \wedge Lzx)$
With B being a boy, G being a Girl, and L x Loving y.
Comment: In short you have $A\rightarrow E\left(x\right)$. In $A$ variable
$x$ is bounded and in $E\left(x\right)$ it is not.
Comment: It should be: $\forall x\left[\left[Bx\wedge\exists y\left[Gy\wedge Lxy\right]\right]\Rightarrow\exists y\left[Gy\wedge Lyx\right]\right]$
Comment: Your translation is correct, apart from a parenthesis problem.
Comment: Ok so by adding the parenthesis I would include x into the last part of the formula?
Comment: Yes, adding parentheses solves it.
Comment: I've never filled a tank with blankets, so obviously this is not a thing that is done.
Answer: You are saying here:
If every boy loves a girl then there is a girl that loves some particular boy yet to be specified.
See also my comment.
Comment: Symbol $x$ is not a constant, so it's not "a specific boy $x$", more like "some particular boy yet to be specified". Or perhaps "If every boy loves a girl, then there is a girl that loves ."
Comment: @dtldarek Thanks, repaired.


How can i resolve all external addresses to internal address?

Question: I am currently setting up a Linux server for a WIFI access-point. When ever someone who is connected to the hotspot/access-point? tries to reload a page they get forced onto the one page.
Note: this wont have internet access!
ie: user tries accessing www.google.com => it returns 192.168.1.200 or example.domain
I've read that "dnsmasq" can be used to redirect any external addresses to an internal address. but haven't had any luck. Anyone have an example of a config for "dnsmasq"?
I have also read that this can be done through a proxy?
Comment: I can't help with specifics but you're attempting to create a "Captive Portal" as used in many hotels, wi-fi hotspots etc. This may help you Google for relevant information. It's not just a case of redirecting DNS, because that would mean typing in an IP would simply bypass it. Or, indeed, using a different DNS server.
Comment: Because.......?
Comment: In this instance i think redirecting dns would work because... 1. it doesn't matter if someone types an IP address directly 2. and the DNS server is supplied with DHCP this would be just like a hotel WIFI but it will be 100% closed

i have also tried using "address=/#/192.168.2.1" in the dnsmasq config
Answer: When you say "this wont have internet access!" I am reminded of a deployment I did a few years ago.
I did this for a kiosk deployment once. The kiosks were connected to LAN that had no Internet connectivity. The kisos were used to access a single webapp, hosted by a beefier "server" kiosk machine. Any attempts to access other web sites needed to be redirected to a single "error" page.
I deployed a DNS server with a wildcard root record (in this case, bind, but I'm sure other DNS servers would support wildcard roots). That  wildcard root record resolved to the IP address of Apache server which used mod_rewrite to rewrite all incoming requests to the URL of a single "You're not connected to the Internet..." page.
If your statement "this wont have internet access!" isn't true, and there is Internet access involved, then you're moving more into a captive portal scenario, and would probably be better served by any one of a number of open source and commercial products that could do this. (Personally, I like the Coova Chilli product, but that's just me...)


What does it mean having 1 as best k parameter in K-NN?

Question: I'm working with a large dataset (761 rows and about 57k-60k features) and after doing a feature selection to select the best 10 features I'm using different ML algorithms to classify some cases. In the case of the K-NN I performed a hyperparameter tunning using GridSearchCV from sklearn and after many tests this returns 1 as the best value for k.
I tried with the RandomizedSearch too and increasing n_iter parameter makes it to eventually return 1 aswell as the best value for k.
Does this mean something? It seems strange to me having this value as the best for k. Could it be overfitting maybe?
Thanks!
Comment: If the test set isn't used for model or feature selection, then I suspect you are O.K. and it is just a relatively easy classification problem.  I tend to repeat the whole experiment several times with different initial test-training splits in order to get some idea of the variation in the performance estimate due to that partitioning.  It is possible to have bad  or good luck with that, especially for very small datasets.
Comment: @Julien I mean patterns where the attribute values are the same as the attribute values of another pattern.  In that case 1-nn performs well if there are a lot of duplicate patterns.  How does SelectKBest work?  If you are applying it to the whole dataset and then using cross-validation to tune the hyper-parameters, you have a dataleak as the test data for hyper-parameter selection has already been used to select the features.  I've worked on microarray data, LASSO/L1 regularisation is likely to be a better approach.
Comment: Try doing the feature selection independently in each fold of the cross-validation procedure used to tune the hyper-parameters.  With cross-validation it is best viewed as a means of estimating the performance of a *method* for generating a model, rather than the model itself, so you need to cross-validate *all* parts of the method, including standardisation and feature selection.
Comment: @DikranMarsupial I checked the duplicates after the feature selection using `X_train_scaled.duplicated(subset=None, keep=False)` and it seems that there are not duplicates
Comment: Does your dataset contain exact duplicate patterns (after feature selection)? Also how is the feature selection performed?
Comment: @DikranMarsupial What do you exactly mean with duplicate patterns? My dataset contains 761 cases of both breast cancer and ovarian cancer patient cases and each column represents the gene expession level of a concrete gene. First I standardise the data, then remove 0-variance features and then perform a feature selection with SelectKBest from sklearn selecting the best 10 features. After this I use a LogisticRegressor model to get the best 5 out of the best 10 (using the beta coefficients).
Comment: @DikranMarsupial First I split the data into training and test, then fit the scaler and the feature selector only with the training data and then transform both training and test data. Seems like I do have some duplicates, but the thing is that when using different models like naive bayes, decision tree or svm i get almost the same results. All around 97-100%
Comment: @DikranMarsupial Fine, I'll try anyways splitting the data differently to see how it behaves. Thanks!!
Answer: There is nothing "bad" about $k=1$. It's a hyperparameter to tune, so different values would work for different problems. If you did your hyperparameter tuning correctly, i.e. there are no data leaks in cross-validation, then it's not overfitting because the parameter was validated on external data. If you worry about overfitting, use the held-out test set to verify the final solution, if it is overfitted then the model would perform poorly on the test set.
If $k=1$ is the best, this may simply mean that your data can be easily explained by a fixed set of "typical cases" that you just need to memorize for making predictions. With higher values of $k$ you would average over multiple examples to reduce noise, maybe here it unnecessarily "blurs" the predictions and noise is not an issue. Of course, those are just guesses, you would need to conduct an in-depth exploratory analysis of the predictions to answer the "why" question.
Comment: "i.e. there are no data leaks in cross-validation, then it's not overfitting because the parameter was validated on external data." this is not strictly speaking true because it is possible to over-fit the cross-validation statistic in tuning the hyper-parameters.  This isn't usually an issue if there are few hyper-parameters, but if other hyper-parameters were being tuned (which is where we would normally use RandomizedSearch) is may be a consideration.
Comment: @DikranMarsupial ok, you can overfit to test set if you try hard enough.
Comment: @DikranMarsupial I don't see it like this, but mentioned in my answer that this needs to be double checked by OP before concluding that the results are valid.
Comment: Right, overfitting would lead to poor performance on test set but I got 100% accuracy on both training and test sets, is this common?
Comment: @Julen maybe your problem is just simple to solve? Make sure there is no data leak, bugs in the code, etc if that is not the case, yes it is possible to have a perfect fit in some cases. If still in doubt, gather more "realistic" examples for testing and see if the performance still holds.
Comment: @Tim Fine, I'll try gathering more data and seeing how it behaves with it. Thanks!
Comment: @Tim, in this case I think the issue is a data leak caused by performing feature selection first and then tuning $k$ (if cross-validation had already been used for that, then it may have already been overfitted).


(hierarchical) cluster analysis with non-standard distance

Question: My question is triggered by a question that was asked on stackoverflow: https://stackoverflow.com/questions/12198115/using-different-metric-for-hclust-linkage.
The thing is this:
I can formulate an algorithm for hierarchical clustering with some kind of distance between original objects, and I'll also need a distance between objects and clusters of objects (linkage).
Let's say the linkage attaches a "handle" to the cluster which is a (hypothetic) object that can be used with the distance function to calculate the distance from another object (or cluster, if it has such a "handle").
For consistency, I'd use the same distance function for the initial calculation of distance between single objects and later for distance between objects or "cluster handles".
So far, so good.
What I'm wondering about is: what happens if the cluster algorithm is fed with a distance matrix instead of the data matrix (as it is e.g. the case with R's hclust).

How do I know which distance function on the distance matrix agrees with what distance function on the data matrix? Examples?
The more basic maths/statistics question underlying this is: please explain to me (chemist, no mathematician/statistician) what the meaning of e.g. Euclidean distance between rows of a distance matrix is?
Answer: Many of the linkage distances are computed on a set of pairwise distances. Then this works just fine with a distance matrix. So most of the time, while the intuition is to have a "handle", what is being done is more a statistical approach.
E.g. single-linkage is the minimum distance, and average linkage is the mean distance taken from the submatrix connecting the two clusters.
Note that for efficiency, you will often want to avoid computing the distance matrix (which needs $O(n^2)$ memory) in the first place. I guess the main reason why this is frequently been done is because it allows you to easily plug in other distances, hierarchical clustering is sensible for tiny data sets only in the first place (the naive algorithm has $O(n^3)$ run time!) and because the operation of computing pairwise distances is available as a highly optimized native C operation, which is significantly faster than anything done in the R interpreter.
As for your question on computing a distance on the rows/cols in the distance matrix: I don't think this is a very sensible measure. As each entry is a distance, it corresponds to a RMSE of distance similarity of two objects. I.e. "the object is as far to all other objects as the other". But there are some related methods used for high dimensional data, that choose a number of reference objects (but not the whole database!) and judge object similarity by how similar the distances to these reference points are. So sometimes, this seems to work quite well; but I wouldn't choose the whole data set as reference objects; after all you want to keep dimensionality low (and one reference object ~ one dimension!) and get below $O(n^2)$ runtime. With this approach, you need the full distance matrix, which is a lot of (possibly redundant) computations and memory.
Answer: A somewhat complicated answer to your first question about distances goes like this.
Rencher's Multivariate Analysis cites Lance and Williams (1967) who proposed a general (flexible beta) method underlying most of the existing hierarchical cluster analysis: for three objects A, B and C, be that clusters or individual points considered to be clusters of size 1, if clusters A and B are joined into AB, then the new distance from AB to C can be expressed by
$$
d(AB,C) = \alpha_A d(C,A) + \alpha_B d(C,B) + \beta d(A,B) + \gamma| d(C,A) - d(C,B) |,
$$
They suggested $\alpha_A + \alpha_B + \beta=1$, $\alpha_A = \alpha_B$, $\gamma=0$  and $\beta < 1$, although you could try other choices. E.g., single linkage is $\alpha_A = \alpha_B = 1/2$, $\beta=0$, $\gamma=-1/2$ (contracting the space), complete linkage is the same except for $\gamma=1/2$ (diluting the space), and Ward's method is $\alpha_A = (n_A + n_C)/(n_A + n_B + n_C)$, $\alpha_B = (n_B + n_C)/(n_A + n_B + n_C)$, $\beta= - n_C/(n_A + n_B + n_C)$, and $\gamma=0$ (somewhat space-contracting). The book discusses the properties of the algorithms, such as monotonicity and space contraction/dilution as functions of these parameters.
So an algorithm can be build just based on the distances, and does not need the data matrix. However, the distance matrix for typical problems (sample size $n$ > dimension of the data $p$) takes more memory than the data matrix, so this may not be a very efficient way of approaching the problem, unless of course the data matrix is all you have.
Comment: Thanks. I'll have to think a bit about the choice of those αs, β, and γ. And, no, I don't want to build an algorithm based on the distance matrix. I just wondered how hierarchical clustering can possibly work if the distance function is unknown to the dendrogram building algorithm. And I notice that unsuitable choice of linkage method and distance can lead to dissimilarities that cannot be interpreted as the original distance (e.g. using 1/2 - cor (x, y)/2 as distance + Ward's method often gives fusion distances > 1).


Strange solution for differential equation $dy/dx = \cos^2{x}\cos{y}$

Question: This is easily integrable however, to make $y$ the subject of the equation gets a strange result.
$$\sec{y} +\tan{y} = Ae^{1/2x+1/4 \sin{(2x)}}$$
The first way is by Weirstrauss substitution to combine the $\sec{y} + \tan{y}$
The second way is to form a quadratic in $\sin{y}$, which gives $$y=\arcsin\left(\frac{-2\pm\sqrt{4-4\left(1+A^{2}e^{\left(\frac{1}{2}\sin2x+x\right)}\right)\left(1-A^{2}e^{\left(\frac{1}{2}\sin2x+x\right)}\right)}}{2\left(1+A^{2}e^{\left(\frac{1}{2}\sin2x+x\right)}\right)}\right)$$
Note the plus minus, HOWEVER, when put into desmos (https://www.desmos.com/calculator/aez6zkd92u) I see that this solution is only valid if the numerator is infact $$\pm(2-\sqrt{...})$$ specifically plus for $A<0$ and minus for $A>0$. Perhaps I have formed my quadratic equation incorrectly but I don't think so. The solution graph is in red but I don't know how this solution was attained. (Wierstrauss gives $2\arctan{(...)}$ due to $\tan{(\frac{y}{2})}$)
Why is this happening to my quadratic? Can you simply "factorize" out the $\pm$?
Comment: But naturally, $-(\pm) = \mp$.
Comment: I dont understand.
Comment: Oh wait, I made a mistake - I thought for a short moment that arcsin was even. I'm not quite sure either now.
Comment: How can I "bump" this question without reposting it?
Answer: Note that
$$\tan(y)+\sec(y)=\frac{2}{e^{iy}-i}-i=f(x)\implies e^{iy}=\frac{2}{f(x)+i}+i=-i\frac{i-f(x)}{i+f(x)}$$
$$\implies y=-i\log\Bigg(e^{-i\pi/2}\frac{i-f(x)}{i+f(x)}\Bigg)$$
This function is absolutely real, however it has discontinuous jumps as function of $f(x)$. Up to jumps
$$y=-2\arctan(f(x))-\pi/2+2\pi n, \quad n\in\mathbb{Z}.$$
Comment: Hi, I don't really get how this answers my question. Your last equation looks different to my and the answer's solution (see graphs on desmos)


Runner at first goes, but nobody else

Question: Yesterday at our training we got to a situation that got discussed. Bases where loaded (lets say Adam was at 1st, Bert on 2nd and Ceasar on 3rd). On a ball Adam starts to run, while Bert and Ceasar stays.
In this situation, our coach said if ball was thrown to 1st, he thought it would be a force on Adam. I feel like first baseman still need to tag Adam if i turns back.
But what if Adam gets to 2nd, can the catcher step on home plate for a force on Ceasar, since Bert should need to run for 3rd?
Comment: What caused Adam to start running (and nobody else)?
Comment: Since the ball is not in play, a force out is not an option. The runner needs to be tagged out. It is, of course, made easier if the other runners don't go anywhere since getting to second for Adam doesn't do anything. During the run down to tag Adam, though, Bert and Ceasar are free to take off as well.
Comment: As I understand it, he missed that he had a runner in front of him and wanted to steal second.
Comment: This is just a pickoff. Whether Adam takes a few steps off first or continues towards second (regardless of how *fast* he goes), he still needs to be tagged out.
Comment: Does stepping on second change nothing?
Answer:
In this situation, our coach said if ball was thrown to 1st, he
  thought it would be a force on Adam. I feel like first baseman still
  need to tag Adam if i turns back.

The MLB rulebook has this definition for a force play:

A FORCE PLAY is a play in which a runner legally loses his right to
  occupy a base by reason of the batter becoming a runner.

The batter received a ball and remained at home plate.  Since the batter has not become a runner, there is no force play.  Ball is not dead and runners may move to another base at their own risk.  Runners need to be tagged out.

But what if Adam gets to 2nd, can the catcher step on home plate for a
  force on Ceasar, since Bert should need to run for 3rd?

No.  Bert is still entitled to 2nd base (there is no force).  So there are no outs to be made on home plate or third (or on first).  They need to throw to second.  5.06(a)(2) tells us that the following runner (Adam in your example) would be out if tagged, if both Adam and Bert are on second.

5.06(a)(2)  Two runners may not occupy a base, but if, while the ball is alive, two runners are touching a base, the following runner shall
  be out when tagged and the preceding runner is entitled to the base,
  unless Rule
  5.06(b)(2) applies.

5.06(b)(2) is for force situations and changes which runner would be out when tagged if two occupy a base.

Runner at first goes, but nobody else


How to find derivative of $[1-F(v)]^{n}$ and $1-[1-F(v)]^{n}$?

Question: How can I find the derivative with respect to $v$ for the following two equations:
$$[1-F(v)]^{n} \tag{1}$$
and
$$1-[1-F(v)]^{n} \tag{2}$$
where
$$F'(v)=f(v)$$
Comment: Note: An "equation" must have an "equal" sign in it (hence the name). What you give are not "equations", they are expressions.
Comment: Have you studied the chain rule?
Comment: Well, you have to show that by induction.
Answer: By using the chain rule:
$$\Big((1-F(v))^n\Big)'=(1-F(v))'n(1-F(v))^{n-1}=-f(v)n(1-F(v))^{n-1}$$
Similarly:
$$\Big(1-(1-F(v))^n\Big)'=0-(-f(v)n(1-F(v))^{n-1})=f(v)n(1-F(v))^{n-1}$$
Comment: got it, thank you so much!


how do i read from firebase into app inventor into individual spots?

Question: totally new to this site and app development in general
i have been teaching myself for a couple weeks now
been looking around for a few days now to a solution to my issue
overall plan is to create an app for my work that will allow me to scan items to make an order
rigt now ive been trying to get the databasing stuff down
i figured out how to read and write to the Firebase database
but the read part isnt quite working that well
* i want to retrieve data from the database and insert or assign it into individual textboxes and or labels *
attached are screenshots of the designer view, codeblocks and firebase page. also i have included the .aia file as well
any help would be greatly appreciated!
lemme kno if more inf is needed
thank you so much in advance!
codeblocks
designview
firebasedb
dropbox link for AIA file
Comment: What is that, is that scratch programming language? And what is the problem with reading?
Comment: @YunusKulyyev this is App Inventor, see also https://stackoverflow.com/tags/app-inventor/info
Comment: you might want to ask your question in the MIT App Inventor community https://community.appinventor.mit.edu/
Answer: I think I understand your question:
You can write and read data per tag.

Make sure your ProjectBucket is correct.


Scenario is passed even dynamic values present in response

Question: My script gets succesful(in VIew results tree all scenarios are in green color )eventhough dynamic values present in response in Jmeter , and if i check the application manually nothing is working,Could Please any one assist
Comment: Please add more details in the query.
Comment: I am trying to create a order id, and order id is creating when i run the script for first time but not the rest of times and script gets passed with 200 response code, could you please assist
Comment: So as @Dmitri suggested you can add a Response Assertion to check for the Text response should contain the orderID and Response Code should be 200 then only allow it to pass. More references on multiple condition with [Response Assertion](https://stackoverflow.com/questions/36275886/how-to-add-or-condition-in-assertion)
Answer: JMeter automatically treats HTTP Status codes below 400 as successful therefore JMeter will not automatically fail the HTTP Request sampler if response code is in the "successful" range.
If you want to add some extra checks for presence/absence of the specific content, request duration, etc. - add JMeter Assertions to your test, for instance Response Assertion should be sufficient to cover the majority of use cases


create subdomain in centos 6

Question: I'm trying to create a subdomain in my virtual server (it's a centOS). I have edited the httpd.conf file like this:
NameVirtualHost *:80
<VirtualHost *:80>
  DocumentRoot /var/www/html/zend/
  <Directory "/var/www/html/zend">
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
  ServerName zend.mydomain.es
</VirtualHost>

When I try www.mydomain.es it's work fine, but when I try zend.mydomain.es I get Server Not Found error.
Are there any other files i must configure for this to work?
Comment: Did you setup your subdomain in your dns settings? E.g. DNS manager on GoDaddy?
Comment: Also make sure you restart your httpd service after you make your change. E.g. `service httpd restart`
Answer: You should also setup dns for the your subdomain and make sure subdomain is pointing to the correct ip
Also check ping if it is replying:

ping zend.mydomain.es

If the IP is correct check both Apache logs (error.log and access.log).
Answer:
I would recommend verifying your DNS settings for the subdomain.
E.g. If you are hosting with GoDaddy, I would recommend making sure
you have an entry for your subdomain there and that the IP points to
the same box.
Next, if you haven't yet, after you make your changes to httpd.conf
you need to restart the httpd service for the changes to take
affect. So try service httpd restart

Hopefully one of those helps solve your issue.

create subdomain in centos 6


Show that $f(z)$ is constant

Question:
If $f(z)=u(x,y) + iv(x,y)$ is an entire function such that $u\cdot v$ is constant then $f(z)$ is constant.

I know that I need to use the Cauchy-Riemann equations, but I don't know how to start. Should I differentiate $u\cdot v$ with respect to $x$ or $y$ then substitute with Cauchy-Riemann?
Comment: Have you tried this approach? How far did you get?
Comment: What happens when you differentiate the product first wrt $x$ and then wrt $y$?
Answer: I think this might work:
Consider the entire function $(f(z))^2$; with
$f(z) = u(x, y) + iv(x, y) \tag{1}$
we have
$(f(z))^2 = (u(x, y) + iv(x, y))^2 = (u(x, y))^2 - (v(x, y))^2 + 2i u(x, y) v(x, y).  \tag{2}$
A very simple application of the Cauchy-Riemann equations to (2), using the fact that $u(x, y) v(x, y)$ is constant, shows that $u(x, y))^2 - (v(x, y))^2$ is constant is well; i. e., we have, by hypothesis,
$(u(x, y) v(x, y))_x = (u(x, y) v(x, y))_y = 0,  \tag{3}$
or
$\nabla(u(x, y) v(x, y)) = 0; \tag{4}$
thus CR yields
$\nabla((u(x, y))^2 - (v(x, y))^2) = 0; \tag{5}$
$(u(x, y))^2 - (v(x, y))^2$ is also constant.  We see then that $(f(z))^2$ is itself constant, and so $f(z)$ must be . . . you guessed it, constant.
Those feeling the need for yet more verbiage can note that
$f(z)f'(z) = \dfrac{1}{2}((f(z))^2)' = 0; \tag{6}$
if $f(z) = 0$, then it is . . .  well, constant; if $f(z) \ne 0$, then $f'(z) = 0$, and again, $f(z)$ is . . . constant!  QED!
Note added Sunday 31 May 2015 10:10 AM PST: This in response to Samar Hayek's comment below:  for general holomorphic $g(z) = r(x, y) + is(x, y)$, Cauchy-Riemann asserts that $r_x = s_y$ and $r_y = - s_x$; if now $\nabla s = 0$, $s_x = s_y = 0$, so $r_x = r_y = 0$, i.e. $\nabla r = 0$.  Applying this to the above shows (4) implies (5), since $u^2 - v^2$ and $2uv$ and $u^2 - v^2$ are the real and imaginary parts of $(u + iv)^2$.  End of Note.
Comment: how does 4 imply 5 ?
Comment: @SamarHayek:  see the note I added to my answer.  Cheers!
Answer: More easiest way:
Consider the function $$g(z)=e^{if^2(z)}.$$
Given , $uv=\text{ constant }=c$(say).
Now $|g(z)|=e^{-2c}$
Then $g(z)$ is bounded entire function and hence it is constant and consequently $f(z)$ is constant.
Answer: The condition given implies that the range of $f(z)$ is a subset of a hyperbola $xy = c$. By the open mapping theorem this is only possible if $f(z)$ is constant.
If you wish to use the Cauchy-Riemann equations, it would be better to use them on $g(z) = (f(z))^2$, since Im$(g(z)) = 2uv$ is constant and therefore the Cauchy-Riemann equations immediately give that all partials of $g(z)$ are zero. This implies $g(z)$ is constant, and it's not hard to go from there to show that $f(z)$ is constant.
Answer: Hint: Given u.v=constant.....(1)
Differentiate partially (1) w.r.t. x to obtain,

$uv_x+vu_x=0$ ......(2)

and  partial differentiation of (1) w.r.t. y gives,

$uv_y+vu_y=0 $

Use C-R equations i.e. v_y=u_x and v_x=-u_y we get,

$uu_x-vv_x=0$  ...(3)

Solve (2) and (3) to get u=constant. Similarly, obtain v=constant. Hence proved.

Show that $f(z)$ is constant


Como fazer menu funcionar para link externo

Question: Estou utilizando o seguinte script para fazer um efeito de deslizamento entre as seções do meu site (links com ancoras. P.E. #contato). No entando no meu menu tem um link externo, como faço para ele funcionar?
$(function() {

var navH = $('body > .nav').outerHeight(true);

$('body').css({
"margin-top": navH + "px"
});

$('body > nav .container-fluid .navbar-collapse .nav ').on("click", 'a', function(e) {
e.preventDefault();

var target = $(this).attr("href"),
  elementOffset = $(target).offset().top,
  distance = (elementOffset - navH);

$("html, body").stop(true, true).animate({
  scrollTop: distance
}, '500', 'swing', function() {
  $('body > .nav .active').removeClass("active");
  $('a[href="' + target + '"]').parent().addClass('active');
});
});
});

Answer: Resolvido. Inclui a classe "desliza" na linha:
$('body > nav .container-fluid .navbar-collapse .nav .desliza ').on("click", 'a', function(e) {

e nas tags
 antes dos link.


Is Ionic 3 incompatible w/ OneSignal?

Question: My app is created with Ionic 3 and my OneSignal push notifications stopped working. I've got a team working on it saying that it's incompatible since it's supposedly "outdated" language and doesn't work well with OneSignal.
Answer: Onesignal still support ionic 3 but you have to install the v4 @ionic/native and the latest onesignal cordova plugin
Install the Cordova and Ionic Native plugins:
  ionic cordova plugin add onesignal-cordova-plugin --save
  npm install --save @ionic-native/onesignal@4

To read more about the plugin you can see Ionic official website for ionic3/Onesignal from here and the Onesignal documentation home page which have a tab for Ionic CLI v3 that still support from here

Is Ionic 3 incompatible w/ OneSignal?


Do numbers $a, b, c$ exist such that there is always a polynomial with integer roots?

Question: Is it possible for some numbers $a, b, c \in \mathbb{R}$, $abc \neq 0$ that for every $n > 3$, there exists a polynomial $P_n(x) = x^n + a_{n-1}x + ... + ax^2 + bx
+ c$ such that all its roots are integers?
I have concluded with Vieta's formula that all coefficients of every polynomial $P_n$ must be integers.
Comment: I'd start with $c=p$, a prime.  So the only candidates are $(x+1)^r(x-1)^s(x\pm p)$.  See how restrictive that is as to the possible $a,b$.
Comment: Where do you find these problems ? they are amazing
Comment: Any integer root would have to be a divisor of $c$, so the factored form of the polynomials is pretty limited.
Comment: Yeah,  I just realized that.
Comment: @lulu Yes, up to a point. There can be multiple 1s and -1s as roots of P and I don't know how to use this info.
Comment: @MohammedM.Zerrak From some old book, that is, unfortunately not available online nor in English.
Answer: Nope.
Let's say the polynomial has $N$ integer roots $\lambda_1,\ldots,\lambda_N \in \{ -1, 1 \}$ and $M$ integer roots $\lambda_{N+1},\ldots,\lambda_{N+M} \notin \{ -1, 1 \}$. Upto a sign, we have
$$\prod_{k=1}^M \lambda_{k+N} = \pm \prod_{k=1}^{N+M} \lambda_k = \pm c$$
So $M$ is bounded from above by  $\Omega(c)$, the number of prime factors in $n$ (see wiki entry of prime omega function for definition/detail).
By Vieta's formulas, $N$ is also bounded.
$$N = \sum_{k=1}^N \frac{1}{\lambda_k^2} \le \sum_{k=1}^{N+M}\frac{1}{\lambda_k^2} = \frac{b^2 - 2ac}{c^2}$$
In order for such a polynomial to have all integer roots, its degree need to satisfy
$$n = N + M \le \frac{b^2-2ac}{c^2} + \Omega(c)$$


Online learning in practice

Question: Is online learning / optimization used in practice? I saw a lot of papers, and they mention document ranking, stock prediction, online ad placements as examples applications of online learning. Is it actually used / evaluated in industry? Is there any white paper either for or against using online learning?
Answer: There is no reason to prefer online learning to batch learning when you can use both methods.
But in some cases (millions of features, millions of observations), you have to use online learning because everything else will fail (or will not terminate).
Some implementations of online learning methods even have a constant memory footprint (hashing the features and allowing collisions).
Click-through rate (CTR) prediction, per example, is generally tackled using online algorithms. Ad Click Prediction: a View from the Trenches is a nice overview.
Answer: There are two main reasons to use online learning in practice:

When not all data fits in memory;
When data arrives in real-time and the amount of processing power per arriving datum is low.

RUser4512 gave a good example of scenario 1 in his answer.
Scenario 2 is very frequently encountered in industrial applications. For instance, stochastic gradient descent and adaptive filtering techniques have been used for decades in electronics and control.
Here's a white paper on an implementation of an efficient recursive regression algorithm Implementation of CORDIC-Based QRD-RLS
Algorithm on Altera Stratix FPGA with Embedded
Nios Soft Processor Technology.
Answer: Online learning is also useful in case of nonstationarities. You might want to track changes in a signal or system.

Online learning in practice


Post to anonymous user's Facebook Wall with rules

Question: I wonder what is the best way to implement post to User's wall for anonymous users. All solutions I came across seem to require creation of a Drupal User Account (using FB Connect) first.
I need a rules action that automatically asks for permission to post and then posts immediately, without creating a Drupal User Account, fetching any additional information and especially without requiring the user to connect manually in advance.
Thank you for suggestions!
Answer: You didn't say what have you tried, so how can we know if we're suggesting some new solution? Anyway you could try Facebook Autopost https://www.drupal.org/project/fb_autopost if it isn't on you personal "All solutions I came across" list
Comment: Thank you, this is exactly what I needed!

I apologize for not giving details. It does not seem to be possbile with FBOauth + Facebook Rules as post to wall action does not fire if user is not connected manually. Drupal for Facebook also misses this feature.

When using FB Autopost make sure to set publishing permissions in the module settings and create correct entity with rules, then it works perfectly!
Comment: You're welcome! I also had lots of troubles with the "post to Facebook modules". Some time ago I tried "Drupal for Facebook", which I think it's the most complete Facebook module for Drupal, but it's also very complex and so I ended using fb_autopost :)


file conversion- .mwb to .sql file

Question: I have a database in .mwb format. I want to use it in eclipse so i need to convert it into .sql format. If anyone know how to do that by using any tool. I think My sql workbench can do that, but don't know how to do that.
Answer: In the menu click "Database" and then "Forward engineering" (shortcut CTRL+G).
UPDATE:
Your .mwb file is a proprietary format, which also holds information of the graphical representation in Workbench. Your generated .sql file is a script, which your MySQL server needs to interpret. To create the database you designed via command line write something like this in the console of your choice:
<code>mysql -u yourUser -p yourDatabase < yourFile.sql
</code>
You can omit "yourDatabase" when you create it with your script.
Comment: i tried that and then clicked next and next after that i saved the file but now i am not able to open that file. How can i open that file
Comment: @VikasGupta Edited my answer. Ask if you have more questions.
Comment: thanks for replying, but i am very week in using command line. So isn't there any other war?
Comment: @VikasGupta You can of course use a GUI like "Toad for MySQL", "HeidiSQL" or "Navicat" and open your .sql file in there and run it.
Comment: Also, phpMyAdmin. Good god, how did the OP survive this far?!
Comment: File -> Open Model (select .mwb file) then Database -> Forward Engineering
Answer: Forward Engineering on my Debian installation of workbench did not work.
This worked for me to convert .mwb file to .sql file.
https://github.com/tomoemon/mwb2sql

file conversion- .mwb to .sql file


arrow hover adding a circle div behind

Question: I have two arrow images (previous / next) with the code here like this:
.fp-controlArrow {
    position: absolute;
    width: 32px; /* This can be added here so you don't have to set a width and height 2 times for each arrow, this will create one width for both arrows */
    height: 32px; /* This does the same as above */
    margin-top:-15px; /* This value must always be half of the height - This does the same as above */
    z-index: 30;
    top: 50%;
    cursor: pointer;
}

.fp-controlArrow.fp-prev {
    left:0;
    background: url(https://uploads-ssl.webflow.com/602dbecd2160ce28b5bc428b/602e923133f8dc0bf994fc49_left-arrow.svg) no-repeat;
}
.fp-controlArrow.fp-next {
    right:0;
    background: url(https://uploads-ssl.webflow.com/602dbecd2160ce28b5bc428b/602e9231ec03b6c9682b540c_right-arrow.svg) no-repeat;

I would like when I hover over an arrow it a round transparent circle appeared behind the arrow like on this example:
enter image description here
how can I achieve this?
thank you!
Answer: padding: 1em;
border-radius: 50%;
background-color: rgba(200,200,200,0.5)

Add those lines to the .fp-controlArrow class That should do it.
Edit
Sorry, I didn't realize it was svg. You will have to inline the svg.

  <svg width="32" height="32" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg" data-reactroot="">
    <path stroke-linejoin="round" stroke-linecap="round" stroke-width="1" stroke="#221b38" d="M16 20L8 12L16 4"></path>
  </svg>

  <svg width="32" height="32" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg" data-reactroot="">
    <path stroke-linejoin="round" stroke-linecap="round" stroke-width="1" stroke="#221b38" d="M8 20L16 12L8 4"></path>
  </svg>

.fp-controlArrow {
    position: absolute;
    width: 32px; /* This can be added here so you don't have to set a width and height 2 times for each arrow, this will create one width for both arrows */
    height: 32px; /* This does the same as above */
    margin-top:-15px; /* This value must always be half of the height - This does the same as above */
    z-index: 30;
    top: 50%;
    cursor: pointer;
    padding: 1em;
    border-radius: 50%;

}

.fp-controlArrow:hover {
  background: green;
  background-color: rgba(200,200,200,0.5);
}

.fp-controlArrow.fp-prev {
    left:0;
}
.fp-controlArrow.fp-next {
    right:0;

Here is a codepen that demonstrates it.
Comment: Hi, have you tried this? I couldn't get it to work, I think because the images are SVGs, but maybe I did something wrong.
Answer: Sorry, I didn't saw the space between : and hover.
.fp-controlArrow.fp-prev {
left:0;
background: url(https://uploads-ssl.webflow.com/602dbecd2160ce28b5bc428b/602e923133f8dc0bf994fc49_left-arrow.svg) no-repeat;
border-radius: 50px;
}
.fp-controlArrow.fp-prev:hover{
background-color: #999999;
}

Comment: I don't think this works - the background images are svgs.
Comment: Apologies, I'd tried you code and it didn't work, but now I see there's a space between the : and the hover in your code which I hadn't noticed. If that is removed it's OK. Perhaps you could edit it out?
Comment: I actually tried your code and I don't find any problems with the svg file. Also you already put "no repeat" to the svg so there's no need to worry.

arrow hover adding a circle div behind


O que são antipadrões de projeto de software?

Question: Existe um catálogo deles para consulta, assim como existe para os padrões de projeto? Antipadrões são o mesmo que más práticas?
Answer: É uma maneira de resolver um problema que foi considerada ineficiente, ineficaz, inefetivo ou improdutiva. Existe em diversos campos.
Curiosamente alguns padrões de projeto conhecidos na computação são considerados antipadrões por alguns.
Ele tem o mesmo problema dos padrões, as pessoas vão atrás deles como se aquilo fosse uma receita para adotar em todos os casos causando mais problema do que solução. Então já podemos começar citando o antipattern que é seguir padrões e antipadrões cegamente, bem como boas práticas ou más práticas.
A questão é como e onde é usado. O contexto define de fato se aquilo é bom ou ruim. Inclusive há controvérsia do que é realmente um antipattern estabelecido ou não, Em geral isso ocorre porque não há contexto.
E de fato os antipatterns normalmente definidos existem de forma muito informal e não passa pelo mesmo crivo que os patterns mais conhecidos.
No caso dos antipadrões eles se parecem muito com más práticas, já que eles não costumam definir fórmulas prontas de solução do problema e sim apenas recomendações. Em caso de código eles são code smells.
Assim como é uma bobagem seguir padrões conhecidos como se fossem fórmulas mágicas, juntar um monte de antipadrões por si só não vale muito. Na verdade, pode ser pior, porque a pessoa pode achar que se não está ali não tem problema fazer algo. As pessoas são muito criativas para fazer coisas erradas.
E é preciso entender que tanto o pattern, quanto o antipattern é só uma regra de como fazer algo que pessoas consideraram como boas ou ruins. É uma boa dica, mas é preciso aprofundar-se e entender sua motivação para usar corretamente.
Eu costumo dizer que pattern é tudo o que é executado sempre do mesmo jeito para resolver o mesmo problema. Ao contrário do que as pessoas pensam não é só um conjunto com algumas receitas de bolo que as pessoas ouviram falar por aí.
Um exemplo que cito muito é a variável, que é um pattern tão usado que está pronto para uso sem que você se dê conta que ele é um padrão de projeto.
E como existem padrões, as linguagens ou outras tecnologias acabam criando antíteses aos padrões de projeto. Ou elas decidem não adotar algo porque consideram um antipattern. Por exemplo, Java não queria criar um mecanismo de delegados, relutaram por anos dizendo que era um antipattern, até que perceberam que era só um pattern, e dos mais importantes, e botaram na linguagem. Percebe que muitas vezes o antipattern está carregado de desinformação e preconceito? E não estou falando de gente boba, falei de um grupo de pessoas dos mais inteligentes do mundo. Preconceito existe em todo mundo, inclusive nas pessoas que são contra preconceitos.
E existe a recursividade que é um antipattern em vários contextos, mas muita gente usa porque acha engraçadinho.
Outro exemplo: as pessoas dizem que usar goto é um anti-padrão, mas as atrocidades que acabam fazendo em alguns casos onde ele é útil só para dizer que não usaram um antipattern, é claramente um antipattern.
Fazer códigos piores para atender uma regra estabelecida por alguém que sequer entendia do que estava falando ou que o que ela falou virou "telefone sem fio" é o maior antipadrão que existe.
Existe um site que tem uma lista de alguns.
A Wikipedia traz uma lista que me parece mais interessante.
Tem uma resposta no SO com uma lista específica bem útil.
Foram publicados alguns livros sobre o assunto.
Podem existir outros mais específicos, inclusive para linguagens e paradigmas, talvez mostrando o que não é idiomático, como o Pitfalls of Object Oriented Programming.
Coloquei no GitHub para referência futura.
Comment: Antipadrões Javeiros: http://antipadroesjava.blogspot.com/


Onde fechar EntityManager

Question: Estou desenvolvendo uma aplicação Java, e estou com dúvida aonde fechar o Entity Manager.
Tenho uma classe Abstrata AbstractService<T> que é extendida por todas as entidades do sistema, e essa classe possui os métodos de persistência.
Em uma servlet qualquer, eu instancio uma ou mais classes de serviços (cada classe de entidade possui uma classe de serviço, como por exemplo a classe Category_DTO e sua classe de serviço CategoryServiceImpl), que contém queries de busca, e ao mesmo tempo estende AbstractService<T>
Minha dúvida é o melhor lugar para fechar o entityManager, em.close().
Pensei em fechar na própria servlet, no final da mesma, utilizando a instância de DocumentServiceImpl com dsi.closeTX(), mas achei meio gambiarrado (lembrando que uma servlet pode conter várias classes de serviço, e eu teria que fechar o entitymanager de cada uma que estendeu AbstractService e criou o entitymanager).
Vou mandar um JSP, com um código de exemplo que é utilizado em servlets, onde retornarei todas as categorias. No final do JSP estou fechando o entitymanager, mas acredito que o correto seria fechar na classe de persistência e não na servlet.
Segue o código:
Entidade:
    package freedom.technology.domain.document;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.NamedQueries;
import javax.persistence.NamedQuery;
import javax.persistence.Table;

import freedom.technology.enus.CategoryType;

@Entity
@Table(name = "FRE_CL_GED_CATEGORY")
@NamedQueries({
    @NamedQuery(name="Category_DTO.findAllCategories", query="SELECT a FROM Category_DTO a WHERE :search is null OR (a.id LIKE :search OR a.description LIKE :search) ORDER BY a.description ASC"),
    @NamedQuery(name="Category_DTO.findAllCategoriesDESC", query="SELECT a FROM Category_DTO a WHERE :search is null OR (a.id LIKE :search OR a.description LIKE :search) ORDER BY a.description DESC"),
    @NamedQuery(name="Category_DTO.findAllCategoriesByType", query="SELECT a FROM Category_DTO a WHERE (:search is null OR (a.id LIKE :search OR a.description LIKE :search)) AND a.type = :type ORDER BY a.description ASC"),
    @NamedQuery(name="Category_DTO.findAllCategoriesByTypeDESC", query="SELECT a FROM Category_DTO a WHERE (:search is null OR (a.id LIKE :search OR a.description LIKE :search)) AND a.type = :type ORDER BY a.description DESC"),
    @NamedQuery(name="Category_DTO.getNumberOfCategories", query="SELECT COUNT(*) FROM Category_DTO"),
    @NamedQuery(name="Category_DTO.getNumberOfCategoriesBySearchCondition", query="SELECT COUNT(*) FROM Category_DTO a WHERE :search is null OR (a.id LIKE :search OR a.description LIKE :search) ORDER BY a.id DESC"),
})

public class Category_DTO {

    @Id
    @GeneratedValue(strategy=GenerationType.AUTO)
    @Column(name="id")
    private Integer id;

    @Column(name="description")
    private String description;

    @Column(name="type")
    private CategoryType type;

    public Integer getId() {
        return id;
    }

    public String getDescription() {
        return description;
    }

    public CategoryType getType() {
        return type;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public void setDescription(String description) {
        this.description = description;
    }

    public void setType(CategoryType type) {
        this.type = type;
    }
}

Classe que retorna registros com base em uma pesquisa específica:
package freedom.technology.service.document.impl;

import java.util.List;

import javax.persistence.TypedQuery;

import freedom.technology.domain.document.Category_DTO;
import freedom.technology.domain.document.Subject_DTO;
import freedom.technology.enus.CategoryType;
import freedom.technology.exception.ExceptionType;
import freedom.technology.exception.FREBDCoreException;
import freedom.technology.service.AbstractService;
import freedom.technology.service.document.api.CategoryService;

public class CategoryServiceImpl extends AbstractService<Category_DTO> implements CategoryService {

    @Override
    protected Class<Category_DTO> getEntityClass() {
        return Category_DTO.class;
    }

    @Override
    public List<Category_DTO> findAllCategories(Integer pageSize, Integer pageNumber, String searchCondition, String orderBy)
            throws FREBDCoreException {

        List<Category_DTO> result = null;

        if (pageSize == null || pageNumber == null){
            throw new FREBDCoreException(ExceptionType.NULLPOINTER, "Parametro null: pageSize/pageNumber");
        }
        try {
            TypedQuery<Category_DTO> query = em.createNamedQuery("Category_DTO.findAllCategories", Category_DTO.class);
            if(orderBy.equals("DESC")){
                query = em.createNamedQuery("Category_DTO.findAllCategoriesDESC", Category_DTO.class);
            }

            if(searchCondition != null) {
                query.setParameter("search", "%"+searchCondition+"%");
            } else {
                query.setParameter("search", null);
            }

            query.setFirstResult(pageSize*pageNumber);
            query.setMaxResults(pageSize);
            result =  query.getResultList();
        } catch(Exception e) {
            e.printStackTrace();
            throw new FREBDCoreException(ExceptionType.NORESULT, "findAllCategories");
        }
        return result;
    }

    @Override
    public List<Category_DTO> findAllCategoriesByType(Integer pageSize, Integer pageNumber, CategoryType type, String searchCondition, String orderBy)
            throws FREBDCoreException {

        List<Category_DTO> result = null;

        if (pageSize == null || pageNumber == null){
            throw new FREBDCoreException(ExceptionType.NULLPOINTER, "Parametro null: pageSize/pageNumber");
        }
        try {
            TypedQuery<Category_DTO> query = em.createNamedQuery("Category_DTO.findAllCategoriesByType", Category_DTO.class);
            if(orderBy.equals("DESC")){
                query = em.createNamedQuery("Category_DTO.findAllCategoriesByTypeDESC", Category_DTO.class);
            }

            if(searchCondition != null) {
                query.setParameter("search", "%"+searchCondition+"%");
            } else {
                query.setParameter("search", null);
            }

            if(type != null) {
                query.setParameter("type", type);
            } else {
                query.setParameter("type", null);
            }

            query.setFirstResult(pageSize*pageNumber);
            query.setMaxResults(pageSize);
            result =  query.getResultList();
        } catch(Exception e) {
            e.printStackTrace();
            throw new FREBDCoreException(ExceptionType.NORESULT, "findAllCategoriesByType");
        }
        return result;
    }

    @Override
    public int getNumberOfCategories() throws FREBDCoreException {
        int result;

        try {
            TypedQuery<Long> query = em.createNamedQuery("Category_DTO.getNumberOfCategories", Long.class);
            result = query.getSingleResult().intValue();
        } catch(Exception e) {
            result = 0;
        }

        return result;
    }

    @Override
    public int getNumberOfCategories(String searchCondition)
            throws FREBDCoreException {

        int result;

        try {
            TypedQuery<Long> query = em.createNamedQuery("Category_DTO.getNumberOfCategoriesBySearchCondition", Long.class);

            if(searchCondition != null) {
                query.setParameter("search", "%"+searchCondition+"%");
            } else {
                query.setParameter("search", null);
            }

            result = query.getSingleResult().intValue();
        } catch(Exception e) {
            result = 0;
        }

        return result;
    }

}

Classe de persistência:
package freedom.technology.service;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import freedom.technology.exception.ExceptionType;
import freedom.technology.exception.FREBDCoreException;
import freedom.technology.utils.PersisteUtils;

public abstract class AbstractService<T> implements GenericService<T> {

    private static EntityManagerFactory factory = PersisteUtils.getEntityManagerFactory();
    protected EntityManager em;
    protected abstract Class<T> getEntityClass();

    public AbstractService() {
        em = factory.createEntityManager();
    }

    public void beginTX(){
        em.getTransaction().begin();
    }

    public void commitTX(){
        em.getTransaction().commit();
    }

    public void closeTX(){
        em.close();
    }

    public void rollback(){
        em.getTransaction().rollback();
    }

    public T findById(Integer id){
        T result = null;

        try {
            em.getTransaction().begin();
            result = em.find(this.getEntityClass(), id);
            em.getTransaction().commit();
        } catch(Exception e) {
            e.printStackTrace();
        }

        return result;
    }

    public void persist(T entity) {
        try {
            em.getTransaction().begin();
            em.persist(entity);
            em.getTransaction().commit();
        } catch (Exception ex) {
            ex.printStackTrace();
            em.getTransaction().rollback();
        }
    }

    public void merge(T entity) {
        try {
            em.getTransaction().begin();
            em.merge(entity);
            em.getTransaction().commit();
        } catch (Exception ex) {
            ex.printStackTrace();
            em.getTransaction().rollback();
        }
    }

    public void removeObject(T entity) {
        try {
            em.getTransaction().begin();
            em.remove(entity);
            em.getTransaction().commit();
        } catch (Exception ex) {
            ex.printStackTrace();
            em.getTransaction().rollback();
        }
    }
}

Jsp exemplo, fechando o entityManager no final:
<%@page import="freedom.technology.domain.document.Category_DTO"%>
<%@page import="freedom.technology.service.document.api.CategoryService"%>
<%@page import="freedom.technology.service.document.impl.CategoryServiceImpl"%>

<%@page import="java.util.List"%>

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Insert title here</title>

<%

    CategoryService ds = new CategoryServiceImpl();
    List<Category_DTO> listCategory = ds.findAllCategories(100, 0, null, null);

    for(Category_DTO category : listCategory){

        out.print("Categorias:");
        out.print("

");
        out.print("Categoria: " + category);
        out.print("
");
    }

    ds.closeTX();
%>

Comment: Dica relacionada, sugiro repensar essa classe `AbstractService`, da forma que esta essa hierarquia vai crescer e se expandir por todo o seu sistema, num futuro proximo isso sera um verdadeiro pesadelo.
Answer: Separação de responsabilidades
A maioria dos sistemas pode ser dividida em algumas camadas conceituais:

Interface ou apresentação: telas/formulários/páginas ou fachada de serviço para consumo externo.

Aplicação: recebe requisições do usuário ou dos sistemas externos e invoca os componentes de negócio para processar as requisições.

Negócio: processamento das requisições.

Persistência: a base de dados onde tudo é persistido.

Se você conseguir manter cada código no seu lugar, fica mais fácil perceber onde abrir e fechar conexão com o banco de dados (ou "sessão", ou "contexto"... dependendo da cultura/plataforma/framework) e onde abrir e fechar a transação.
Gerência de sessão e transação de banco de dados
Uma transação de banco de dados serve para garantir a atomicidade de uma operação. Ou seja, se a operação vai alterar dois registros: ou ambos serão alterados com sucesso ou nenhum será alterado - se der erro na alteração do segundo, desfaz-se a alteração do primeiro.
Pensando em atomicidade de operação, não faz sentido o gerenciamento da transação estar dentro da classe responsável pela persistência de uma única entidde!
Então uma operação (ou "processo de negócio") sabe se ela exige uma transação ou não: se ela vai alterar mais de um registro, ela vai querer atomicidade, então vai exigir transação.
public void processo_de_negocio() {

    // Não ligo sobre transação. Uma transação com commit implícito já serve pra mim.
    // Neste caso aqui, se a transação estiver
    // na classe de persistência deste único registro,
    // parece ok.
    salvarRegistro();
}

public void processo_de_negocio2() {

    // Quero garantir que o salvamento dos dois registros seja atômico!
    // Então a transação não pode ser controlada pela classe
    // de persistência de cada um dos registros.
    beginTransaction();

    salvarRegistro();
    salvarOutroRegistro();

    commitTransaction();
}

Muito bem. Mas o processo de negócio 2 não sabe se ele fará parte de um outro processo maior e se precisará participar da mesma transação. Então ele também não deveria cuidar da transação.
Então quem deveria cuidar da transação? A camada de aplicativo deveria cuidar da transação:
public void post() {

    beginTransaction();

    processo_de_negocio();
    processo_de_negocio2();

    commitTransaction();
}

E a conexão/sessão/entitymanager? A gestão da conexão também é responsabilidade da camada de aplicativo:
public void post() {

    em = factory.createEntityManager();
    em.getTransaction().begin();

    processo_de_negocio(em);
    processo_de_negocio2(em);

    em.getTransaction().commit();
    em.close();
}

É claro que os frameworks facilitam a vida e eventualmente não precisamos ficar passando conexão/sessão/entitymanager por parâmetro, e usando estes frameworks também não precisamos gerir a transação na mão.
Mas conceitualmente a idéia é esta e é assim que os frameworks trabalham.
O seu código
Isto posto, vamos dar uma olhada no seu código.
A classe AbstractService
Ora, se a gestão da transação é função da camada de aplicativo, quase todo o trabalho realizado hoje por sua classe AbstractService está no lugar errado.
Se você passar a gestão da transação para o lugar certo, a maioria dos métodos que estão hoje em AbstractService deixam de ser necessários e os métodos que sobrarem você pode passar para a CategoryServiceImpl, pois eles terão uma única linha cada (exemplos: em.persist(entity);, return em.find(this.getEntityClass(), id);).
E já que você nunca vai precisar representar a CategoryServiceImpl na forma de AbstractService, e nem há muito código a reaproveitar, esta herança é completamente desnecessária.
A página JSP
O código aí também não está respeitando a definição de camadas, pois ele mistura apresentação e lógica de aplicativo.
Se você quiser mais facilidade de manutenção e de reutilização, você pode criar uma classe Servlet cujo método doGet entregaria esta lista de entidades para a página JSP. E, no caso de ser necessário um processamento de negócio no submit da página, o método doPost trataria de obter a sessão e gerenciar a transação, como no exemplo que mostrei acima.
O tratamento de exceções
Você está sumindo com os possíveis erros. Quando eles acontecerem, o seu aplicativo vai se comportar de uma maneira quase imprevisível e de difícil diagnóstico.
Não adianta imprimir a exceção no log, o usuário precisa saber que a operação dele não ocorreu com sucesso.
Também não adianta, no catch, retornar zero ao tentar ler "quantos registros". Ali só vai dar erro se houver algo muito errado, como por exemplo o nome da tabela ter mudado; encobrir este erro retornando um confuso Zero não vai ajudar no diagnóstico do problema.
Algumas vezes faz sentido um tratamento de exceção nas camadas inferiores mas, de maneira geral, o tratamento de exceções também pertence à camada de aplicativo.
Então você deveria retirar a maior parte do tratamento de exceções que tem hoje, e mover para a camada de aplicativo.
O servlet (pertencente à camada de aplicativo), ficaria afinal mais ou menos assim:
public void post() {
    try {
        em = factory.createEntityManager();
        em.getTransaction().begin();

        processo_de_negocio(em);
        processo_de_negocio2(em);

        em.getTransaction().commit();
        em.close();

    } catch (Exception e) {
        // aqui, avisa o usuário de maneira amigável,
        // e mostra ou registra a exceção no log
    }
}

Um finally também seria útil ali para garantir o rollback e fechamento da conexão (ou sua devolução para o pool) imediatamente.
Nota final: depois de entender estes conceitos, considere usar frameworks para reduzir o tanto de código que você precisa escrever e preocupações que você precisa ter.
Answer: Boa tarde,
Obrigado pela resposta.
Com base em estudos que realizei e visando atender as necessidades do software, encontrei a seguinte solução:
Realizei o gerenciamento das instâncias do EntityManager e transactions através de uma classe filtro da javax.servlet, que filtrará as requisições, instanciará o entityManager e abrirá transaction caso necessário: (SecurityFilter implements Filter).
 -Antes de iniciar a servlet eu crio a instância do em e abro a transação;
 -Após finalizar a servlet, o filtro commita a transação, e finaliza o entityManager;
 -Em caso de erro, rollback em tudo, garantindo a atomicidade.
package freedom.technology.session;

import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map.Entry;
import java.util.Set;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

import freedom.technology.service.PersistEngine;

@WebFilter("/*")
public class SecurityFilter implements Filter {

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {

        HttpServletResponse httpResponse = (HttpServletResponse) response;
        HttpServletRequest httpRequest = (HttpServletRequest) request;
        HttpSession session = httpRequest.getSession();
        UserSession userSession = (UserSession) (session.getAttribute("UserSession"));

        String url = httpRequest.getRequestURI();
        StringBuilder urlToRedirect = new StringBuilder("?");

        if ((userSession == null) && (url.startsWith("/FRE_BPM/page-wrapper"))) {

            //redirecionamento de url
            //documentos e tarefas enviadas por email, etc
            HashMap<String, String[]> params = (HashMap<String, String[]>) request.getParameterMap();
            Set<Entry<String, String[]>> setParams = params.entrySet();

            for (Iterator<Entry<String, String[]>> it = setParams.iterator(); it.hasNext(); ) {
                 Entry<String, String[]> entry = it.next();

                 urlToRedirect.append(entry.getKey()).append("=").append(request.getParameter(entry.getKey())).append("&");
            }

            httpResponse.sendRedirect("/FRE_BPM/login.jsp".concat(urlToRedirect.toString()));
        } else {

            boolean transactional = true;

            if(url.contains(".") && !url.endsWith(".jsp")){
                transactional = false;
            }

            if(transactional){

                System.out.println("Iniciando transação para a url: " + url);
                try {
                    PersistEngine.beginTransaction();
                    chain.doFilter(request, response);
                    PersistEngine.commit();
                } catch (Exception e) {
                    e.printStackTrace();

                    if ( PersistEngine.getEntityManager() != null && PersistEngine.getEntityManager().isOpen()) {
                        PersistEngine.rollback();
                    }

                } finally {
                    PersistEngine.closeEntityManager();
                }
            } else {
                chain.doFilter(request, response);
            }
        }
    }

    @Override
    public void destroy() {
    }

    @Override
    public void init(FilterConfig arg0) throws ServletException {
    }

}

Migrei todo a lógica de persistência para uma classe que criei (PersitEngine). Esta classe criará as instâncias do entityManager, controlará as transações, e realizará as ações de persist, merge e remove.
package freedom.technology.service;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;

import freedom.technology.utils.PersisteUtils;

public class PersistEngine {

    private static EntityManagerFactory factory = PersisteUtils.getEntityManagerFactory();
    private static final ThreadLocal<EntityManager> threadLocal = new ThreadLocal<EntityManager>();

    public static EntityManager getEntityManager() {
        EntityManager em = threadLocal.get();

        if (em == null) {
            em = factory.createEntityManager();
            threadLocal.set(em);
        }
        return em;
    }

    public static void closeEntityManager() {
        EntityManager em = threadLocal.get();
        if (em != null) {
            em.close();
            threadLocal.set(null);
        }
    }

    public static void closeEntityManagerFactory() {
        factory.close();
    }

    public static void beginTransaction() {
        getEntityManager().getTransaction().begin();
    }

    public static void rollback() {
        getEntityManager().getTransaction().rollback();
    }

    public static void commit() {
        getEntityManager().getTransaction().commit();
    }

    public static <T> void persist(T entity){

        getEntityManager().persist(entity);
    }

    public static <T> void remove(T entity){

        getEntityManager().remove(entity);
    }

    public static <T> T merge(T entity){

        return getEntityManager().merge(entity);
    }

    public static <T> Object findById(Class<T> clazz, Integer id){

        T result = getEntityManager().find(clazz, id);

        return result;
    }
}

As classes AbstractService e GenericService serão excluídas, pois não necessitarei mais das mesmas.
Comment: Bacana, você está caminhando para a idéia da minha resposta: gerenciar sessão/conexão e transação de banco de dados na camada de aplicativo. Nunca fiz isso em um Filter; fiquei meio receoso mas não consegui pensar agora no que poderia dar errado...
Comment: Dica: não adianta muito começar sua reposta dizendo "obrigado pela resposta", uma vez que se aparecerem outras, não se saberá do que vc está falando. E agradecimentos de maneira geral são meio dispensáveis por aqui :-) Uma opção melhor é votar nas respostas que lhe forem úteis.

Onde fechar EntityManager


Labelling Layered Charts in Altair (Python)

Question: I am attempting to create two layered histograms in Altair (and a vertical mean ruler for each). I would like a legend to label each of these four.
I am using the first 'Birth weight I' data that can be found here
My code (real long, apologies) looks something like this:
from altair import datum

# This histogram for baby weights of mothers who dont smoke
dont = alt.Chart(babyData).mark_bar().encode(
    alt.X("bwt-oz:Q", axis=alt.Axis(title='Birth Weight (Ounces)'), bin=True),
    alt.Y('count()', axis=alt.Axis(title='Count'), scale=alt.Scale(domain=[0, 350]))
).properties(
    width=400,
    height=400
).transform_filter(
    datum.smoke == 0,
)

mean = alt.Chart(babyData).mark_rule(color='red').encode(
    x='mean(bwt-oz):Q',
    size=alt.value(4)
).transform_filter(
    datum.smoke == 0
)

dontSmokeChart = dont + mean

# This histogram for baby weights of mothers who smoke
do = alt.Chart(babyData).mark_bar().encode(
    alt.X("bwt-oz:Q", axis=alt.Axis(title='Birth Weight (Ounces)'), bin=True),
    alt.Y('count()', axis=alt.Axis(title='Count'), scale=alt.Scale(domain=[0, 350]))
).transform_filter(
    datum.smoke == 1
).properties(
    width=400,
    height=400
)

mean2 = alt.Chart(babyData).mark_rule(color='red').encode(
    x='mean(bwt-oz):Q',
    size=alt.value(4)
).transform_filter(
    datum.smoke == 1
)

doSmokeChart = do + mean2

# This layers, and puts them all together

layer = alt.layer(
    dont,
    mean,
    do,
    mean2
).properties(
    title="Layered Histogram of Baby Weights of Mothers Who smoke Vs. Who Don't",
).configure_mark(
    opacity=0.5,
    color='blue',
)
layer

The final layered chart looks something like this:

I would simply like a legend specifying which histogram/mean belongs to what.
If I could color them too, and perhaps add a legend that way, that would be nice as well, but I am unsure how to do so.
Thanks for any insight!
Answer: Rather than manually creating layers with filtered data, you should use a color encoding on your full dataset: then a legend will be generated automatically.
For example:
import altair as alt
import pandas as pd

babyData = pd.read_csv('https://www.stat.berkeley.edu/users/statlabs/data/babiesI.data', delim_whitespace=True)

base = alt.Chart(babyData).transform_filter(
    'datum.smoke != 9'
)

hist = base.mark_bar(opacity=0.5).encode(
    alt.X("bwt:Q",title='Birth Weight (Ounces)', bin=True),
    alt.Y('count()', title='Count'),
    color='smoke:N'
).properties(
    width=400,
    height=400
)

mean = base.mark_rule().encode(
    x='mean(bwt):Q',
    size=alt.value(4),
    color='smoke:N'
)

hist + mean

From there you could use standard approaches to Customize the color schemes used for each mark.
Answer: @jakevdp just beat me to it! I was going to say the same thing. Here is a full example for you to work with.
import pandas as pd
import altair as alt

# Link to data source
URL =  'https://www.stat.berkeley.edu/users/statlabs/data/babiesI.data'
# Read data into a pandas dataframe
df = pd.read_table(URL, sep='\s+')

hist = alt.Chart(df).mark_area(
    opacity=0.7,
    interpolate='step'
).encode(
    alt.X("bwt:Q", axis=alt.Axis(title='Birth Weight (Ounces)'), bin=True),
    alt.Y('count()', axis=alt.Axis(title='Count'), stack=None),
    alt.Color('smoke:N')
).properties(
    width=400,
    height=400
).transform_filter(alt.datum.smoke != 9)

rule = alt.Chart(df).mark_rule(color='red').encode(
    alt.Detail('smoke:N'),
    alt.Color('smoke:N'),
    alt.X('mean(bwt):Q'),
    size=alt.value(4),
).transform_filter(alt.datum.smoke != 9)

hist + rule


Ad placement optimization for lead marketplace

Question: I'm looking for suggestions for ad placement optimization problem. Description is below.

There exists a lead marketplace with 4 entities.

Marketplace M
Individuals I
Publishers P
Advertisers A

An Individual I will generate a lead via a Publisher P.There are P
publishers sending leads to the Marketplace M.There are A advertisers
offering their services on marketplace M. Each Advertiser Ai has x Ads
that are active at any one time denoted, Aix.There are n ad slots
available to display ads from choices of Aix to each lead. Each slot
has a different probability of being clicked.Each ad Aix has two
performance variables - probability of being clicked and revenue
generated to marketplace M if the ad is clicked, i.e. some ads pay
more than others and do so more often than others. The goal is to
maximize revenue for marketplace M.

I initially considered bandits, but I don't think that will work because bandits assume a fixed payout with variable payout rate. In our case, we have variable payout rate but also variable payout.
Any suggestions to research papers or specific algorithms would be appreciated.
Answer: Bandits do not assume fixed payouts. Bernoulli bandits assume that the rewards are binary, but there are also bandit algorithms for non-binary rewards, for example, they can be continuous (see e.g. Bonald, 2018). In the generic formulation of the multi-armed bandit problem, you have a set of arms $a \in A$ that generate rewards $r \in \mathbb{R}$ with probabilities $p(r|a)$. We can define expectation
$$
q(a) = E[r|a]
$$
in such a case, the best action is the one that maximizes the expected reward
$$
a^\star = \underset{a \in A}{\operatorname{arg\max}} \; q(a)
$$
where the expected reward per action is
$$
q^\star = \underset{a \in A}{\max} \; q(a)
$$
and we aim to maximize the cumulative reward over time $\sum_t r_t$. For such a problem, you wouldn't use the Bernoulli bandit, but another formulation that allows for non-binary rewards. The common choice is using Gaussian distributions, because of mathematical convenience (easy to update thanks to conjugacy, easy to calculate expectations, etc).
If variable payout rates are an issue for you, you can always play the arm for a fixed amount of time and then look at the total collected rewards in the time window. This would make the results collected for each arm would be comparable.
Answering the comment: If you are not limited to pulling one arm at a time, it's still doable, however, it is not the classical multi-armed anymore, but a budget allocation problem. You can pull multiple arms, doing a Bayesian update of the rewards per each arm. To allocate the resources, you can do it proportionally to the expected rewards or have other strategies depending on the details of the problem. It's a slightly different problem, as you are optimizing the overall expected reward that needs to account for how much do you exploit each of the arms.
Comment: ah interesting. I did not know that. Thank you. What about the situation where we are pulling multiple arms at once - one for each of the ad slots?
Comment: @fcol you can use the same Bayesian updating for each arm, but in such case, it's not the simple multi-arm bandit but a different problem.