Starting to learn about IPtables, why do I need this rule?

Question: I have been reading all day about IP tables, how the rules work, sets of rules for basic webservers, etc. Now I`m trying to configure my webserver with IPtables using this thread as a starting point: Good iptables starting rules for a webserver?
Since I don't want to just copy paste anything that works I`m adding rule by rule and considering every rule as to why I would need it. I came to discover that I basically needed access via port 80, 443, 22, 587 (mailserver).
However, I cannot access my site via the domain after adding these rules. So I compared to the thread I mentioned and I discovered I did not have this rule:
# Allow traffic already established to continue
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

So I added this rule to test things, and all of the sudden I could access my website. Then I thought, maybe that's because I already had been connecting to the website from this IP, but if I connect through other IP`s / networks I still can access my website, dependent on that rule.
Because I want to understand this, I don't really get why access to my site is dependent on that rule. There is already a rule allowing incoming traffic via port 80 (which is the port my webserver runs on) but appearantly just that rule is not enough? PS, this is the rule I have at the moment to open up port 80:
target     prot opt source               destination
 ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
I hope someone can shed some light about that rule and why I can access my site with that rule and not without.
EDIT: I do get why one would want to add this rule, I reckon it would speed up things as it does not have to go through all the other rules anymore but I just want to understand why in my case it is critical to have this rule in order to access my site..
Thanks!
Comment: Rule order matters in `iptables`, so there is no point in showing us one rule and asking why it's not working.  You will need to cut and paste the entire output of `iptables -L -n -v` into your question for us to help.
Comment: Hi, thanks for your comment! I now only have the rules for the ports I described (80, 443, 22, 587) as accept rules, and the last rule is the state established rule. There are no reject rules and no other rules at the moment.
Comment: Don't tell us - *show* us.  Showing us the failure you describe would be a good idea, too.
Comment: Alright, here's a pic: http://i57.tinypic.com/ra6x6g.jpg , please note policy is accept now and with that, it works. If I change policy to drop, it does not work. I already accepted an answer because I'm going for a statefull firewall and it works when I go with that, however I'm still not sure why the rules from the image won't work being a stateless firewall. But I think it has to do with me knowing too little about statefull vs stateless firewalls atm. If you got anything to add having this image as info, please share because I want to learn as much as possible about this!
Comment: PS: not sure about what you mean with  'showing the failure'. Basically what happens is, I cannot access the site (after like 5 - 10 seconds of loading) and Chrome is saying  'Err, connection refused'. I can access the site fine changing the policy to accept or adding the statefull rule for established connections.
PS2: as you can see in the image there are a few new rules now for ping and DNS purposes
Comment: Since you've decided the question has been answered to your satisfaction, I won't say much more save to thank you for accepting an answer - the community hopes that all question authors will do that, but some don't get around to it.  By *showing the failure*, I meant putting into your question an example of what you tried to do, what you expected to see, and what you actually got.  Telling us these things is very helpful, as sometimes the wrong conclusions are drawn from a diagnostic; **showing** us these things is even more helpful, as sometimes people aren't testing what they think they are.
Comment: I see! Thanks mate. In my scenario the failure was pretty simple, opening a site and not seeing anything or opening a site and seeing the website, depending on a firewall rule :) Thanks for your time anyway since I`m going with the statefull firewall I have it up&running now.
Answer: iptables is a firewall statefull, as opposite as stateless which means it identifies connections as streams, associated with states. the connection is first initiated (NEW state), (SYN packet for tcp, or first packet for UDP). Then, a decision is taken (BLOCK, DROP, or ACCEPT). This is what you created before the ESTABLISHED rule. If a packet is accepted on a NEW state, the packets belonging to the same connection won't necessary be (it's not the NEW state anymore. For tcp, it would be a SYN-ACK, ACK, Push, Push-ACK, ... packets). These packets belong to the state ESTABLISHED. Usually, ESTABLISHED packets are accepted to make it work.
If you don't activate this rule, you will have a stateless firewall. To make it work, you would need to accept every packets corresponding to your chosen ports. This can lead to security issues (think at client firewalled trying to reach a webserver; every packets going to webserver port 80 or coming from webserver port 80 would be accepted (for the webserver answer). If someone spoof webserver IP address, then it completely bypass the firewall, as long as it tries connecting with source port 80.
RELATED is not necessary here. Its corresponds to traffic not belonging to the connection, but sometimes usefull:

when a port is closed, a ICMP-port unreachable is emitted, this packets does not belongs to the connection, but may be usefull
for protocols such as FTP whose data port is chosen inside control traffic; RELATED authorizes the data traffic, even if no explicit rule is associated to it.

More on that here
Comment: Wow, thanks so much for this information. It is a lot of new information to take in but it is definitely going to help me understand how it works and why it behaves as it is behaving right now.
Comment: No worries :) glad it can help!
Answer: Confused me for so long, leave for future reference.
Quote from https://www.netfilter.org/documentation/HOWTO/netfilter-hacking-HOWTO-3.html:
NAT:
This table is slightly different from the `filter' table, in that only the first packet of a new connection will traverse the table: the result of this traversal is then applied to all future packets in the same connection.
So this --state ESTABLISHED,RELATED is not in effect in nat table.
And be aware if you use conntrack -D after setting up nat table, this will make iptables think next packet it sees is that "first packet of a new connection"(matchable by --state NEW), causing the middle of one connection to be NAT in reverse direction if you have relative rules which highly probably will break established connection.


Appending arrays to dataframe (python)

Question: So I ran a time series model on a small sales data set, and forecasted sales for next 12 periods. With the following code:
 mod1=ARIMA(df1, order=(2,1,1)).fit(disp=0,transparams=True)
    y_future=mod1.forecast(steps=12)[0]

where df1 contains the sales values with months being the index. Now I'm storing the predicted values in the following manner:
pred.append(y_future)

Now, I need to append the forecasted values to the original dataset df1, preferably with the same index. I'm trying to use the following code:
df1.append(pred, ignore_index=False)

But I'm getting the following error:
TypeError: cannot concatenate a non-NDFrame object

I've tried converting pred variable to list and then appending, but to no avail.
Any help will be appreciated. Thanks.
Comment: The problem here seems to be that your pred is not if the df type but probably of the list type. What you need to do is convert pred to dataframe. Something like this `new_pred = pd.DataFrame(pred)`
Comment: do you want to append it to a new column in the data frame?
Comment: @Pratik Kumar, no I'd like to append it as rows to the existing dataframe.
Comment: @Imre_G, thanks I've tried that but its not getting appended as rows.
Comment: How about `df['predictions'] = pred`?
Comment: @Vivek getting this error: TypeError: Index does not support mutable operations
Answer: One solution could be appending the new array to your dataFrame to the last position using df.loc
df.loc[len(df)] = your_array

But this is not efficient cause if you want to do it several times, it will have to get the length of the DataFrame for each new append.
A better solution would be to create a dictionary of the values that you need to append and append it to the dataFrame.
df = df.append(dict(zip(df.columns, your_array)), ignore_index=True)

Comment: hi@saloua thanks for this, but i wanted to append the forecasted values as rows to the original sales dataframe, the method you've stated would probably hold for columns I believe.
Answer: You can append your results into a dictionary list and then append that dictionary list to data frame.
Let's assume that you want to append your ARIMA forecasted results to the end of actual data frame with two columns "datetime" (YYYY-MM-DD) and "value" respectively.
Steps to follow

First find the max day in datetime column of your actual data frame and convert it to datetime. We want to assign future dates for our forecasting results.
Create an empty dictionary list and inside a loop fill it by incrementing datetime value 1 day and place a forecasted result subsequently.
Append that dictionary list to your dataframe. Don't forget to reassign it to itself as left hand value since append function creates a copy of appended results data frame.
Reindex your data frame.

Code
lastDay = dfActualData[dfActualData['datetime'] == dfActualData['datetime'].max()].values[0][0]
dtLastDay = lastDay.to_pydatetime("%Y-%m-%d")

listdict = []
for i in range(len(results)):
    forecastedDate = dtLastDay + timedelta(days = i + 1)
    listdict.append({'datetime':forecastedDate , 'value':results[i]})

dfActualData= dfActualData.append(listdict, ignore_index=True)
dfActualData.reset_index(drop=True)

Appending arrays to dataframe (python)


More convergent series

Question: This question just reminded me of a conundrum I posed myself in my first year of university. I never did get a satisfactory answer...

Let $a_n$ be a null sequence. Does it follow that $\sum \frac{a_n}{n}$ converges?

Any ideas?
Comment: Just curious: if you couldn't figure this out for yourself, why didn't you ask your instructor?  It's what we're here for, you know.
Comment: I've added a link for definition of a null sequence, since Samuel's answer indicates not everyone is completely familiar with the term. I'd also like the question title to be more descriptive, but @Tom, you should choose one you feel most closely aligns with the intent of your question.
Comment: @Pete, I have no idea... was a long time ago- I just remembered the question out of the blue.
Answer: If by null sequence you mean a sequence that converges to 0, then no. Try $a_n=1/\log n.$ By integral comparison, the series diverges:
$$\sum_2^\infty\dfrac1{n\log n}\geq\int_2^\infty\dfrac{dx}{x\log x}=\int_{\log 2}^\infty\dfrac{du}u=\infty,$$
where I've used the change of variables $u=\log x$.
Comment: I suggest using the condensation test (http://en.wikipedia.org/wiki/Cauchy%27s_condensation_test), which is easier than integrating 1/log(x), IMO
Comment: The integral we get is $\int \dfrac{dx}{x\log x}$, which is transformed to $\int\dfrac{du}u=\infty$ by the change of variables $u=\log x$.
Comment: thanks. I'm sure my younger self could have sworn \sum 1/nlog(n) converged. In fact I'm sure this was the motivating example for me. As it happened, figured another counterexample as soon as I went to bed.

More convergent series


Bizarre behaviour related to iframes

Question: In our project, we dynamically generate pages, and for that we use iframes in iframes. (I know, generally bad idea, but it is a really old project)
The bizarre behaviour is the following:
There is an iframe, which should be hidden, and for that we use display:none;
When a specific button is pressed, it should remove this display:none part, and it does it. However, the iframe does not appear. (for other iframes, this behaviour works perfectly)
The bizarre part comes in, when I (in Chrome) search in the Element selection of the development tools for the specific Id for the iframe, (I kid you not) it suddenly appears on the webpage.
Does anyone has any idea why it doesn't appear, and what kind of update the element search does to it to make it appear?
Thanks
Answer: setting display: none; will stop browser from loading the iframe's content, even if you later make it display block, then the iframe element will be there but it won't fetch data from network.
try this position: absolute; left: -5000px; or use js to refresh iframe after its set to display block

Bizarre behaviour related to iframes


Probability that random variable B is greater than random variable A

Question: I would like to find the answer to the following question. A casino offers games with one of the following 21 p's  [0; 0.05; 0.10 .... 0.90; 0.95; 1] and one of the following 21 v's (values) [0; 5; 10 ... 90, 95, 100]. The person taking part in the game either wins the v or he gets nothing.
In sequence A, 98 games are offered. The total expected value (ev) of the 98 games is: 1785.75 euro. I assumed that the variance (var) could be computed by summing: ev * (v-ev). Thus, a game with a p of 0.5 and a v of 80 has an ev of 40 and a var of 1600. The total var is 31862.94.
In sequence  B, 98 other games with different p's and v's are offered. The total ev is 1276 euro and the total var is 48585.75.
I assume that both binomial probability distributions can be approximated with a normal distribution with means of 1785.75 and 1276 and sd's of 178.5019 and 220.4218 (square root of the variance).
I would like to know the probability based on the normal approximations that sequence B leads to more v than sequence A. The difference between the means is 509,75 euro which is between 2 and 3 sd's. I am not looking for very exact answers. I would just like to know how the probability that V(B) > V (A) can be approximated.
Comment: Sequence A and B are indeed independent from each other. With regard to the normality assumption: The games that make up sequences A and B have been sampled in some way, but after the sampling I take the games as a given. The way I see it, both sequences are 'Poisson binomial distributions'(see: http://en.wikipedia.org/wiki/Poisson_binomial_distribution) because the p's are not all the same. I have assumed that these probability distributions can be can be approximated with a normal distribution. I hope this assumptions is correct.
Answer: Estimating $\theta=P(A<B)$ where $A$ and $B$ are random variables is known as "stress-strength" models. In the normal case, $\theta$ is simply a function of the means and variances of $A$ and $B$:
$\theta=P(A<B) = \Phi\left(\dfrac{\mu_B-\mu_A}{\sqrt{\sigma_A^2+\sigma_B^2}}\right). (*)$
For a more detailed review check this book.
If you have samples from $A$ and $B$, then you can estimate $(\mu_A,\mu_B,\sigma_A,\sigma_B)$ using the sample mean and variances and then plug them into $(*)$. This will be the Maximimum Likelihood Estimator of $\theta$.
Best wishes.
Comment: Thanks a lot for your answer. If I understand you correctly, this leads to the following result.
509.75/ sqroot (31862.94 + 48585.75) = 1.79.
This Z-score has a value of 0.9633. So the probability that sequence B leads to more v than sequence A is 1-0.9633 = 3,67%.
Comment: @FranzBrentano Yes, you are correct. I would also check the normality assumption of the samples. For this purpose, you can use shapiro.test() in R or a graphical tool like qqnorm() and qqline().
Comment: Probably it's understood, but to use that formula the 2 Normal r.v.'s must be independent.


Is learning (theoretical) physics useful/important for a mathematician?

Question: I'm starting to read The Princeton Companion to Mathematics, at the beginning it says:

A proper appreciation of
  pure mathematics requires some knowledge of applied
  mathematics and theoretical physics.

Some of my professors have told me that modern Mathematics require some knowledge about Quantum Mechanics and theoretical Physics.
I attended the second and third day of the José Adem Memorial Lecture Series by Matilde Marcolli at the beginning of this year, they were about Number theory, Quantum statistical Mechanics and Quantum Field theory. I did not understand a single word but somehow the few things I understood had a big impression on me.
Please give me some examples of pure Mathematics that require/use Physics.
Comment: I thought theoretical physics *is* a field of applied mathematics. :-)
Comment: I don't have the book, but I'd guess that the intent of the quote is not that pure mathematics *requires* or *uses* physics or applied considerations, as much as that if you want a "proper appreciation" of pure math, in the sense of where it *comes from*, it helps to understand a little of physics and applied math.  Because so many areas of pure math have their origins in applied problems, even if they are their own thing now.  e.g. anything using the ideas of calculus owes a debt to Newton's mechanics.  You don't *need* to know this, but shouldn't you, in order to appreciate calculus?
Comment: I tough that way too when I was reading it.
Comment: By useful I meant that if being a researcher who knows Physics/Applied Mathematics could help for getting new ideas/concepts.
Comment: @Vicfred: I was wondering if you had any response to my answer below. I see with your bounty that you're still probing for more answers, and maybe I could extend mine if I knew a bit more about what you were looking for.
Comment: I am currently studying a degree in Pure Mathematics which include 4 physics courses (I.- classical mechanics, II.- Thermodynamics and Fluids, III.- Electromagnetism, IV.- Optics) after that I'm free to choose if I take more advanced physics courses and I wanted some opinions to think about that.
Comment: @mixedmath I am looking for examples of Pure Mathematics inspired by Physics/Applied Mathematics.
Answer: As a graduate student of pure mathematics, I question how valid the statement that a proper appreciation of pure math requires some knowledge of applied mathematics and theoretical physics. Perhaps part of this comes from: where is the line that separates applied mathematics and pure mathematics? If we include the whole of calculus within applied mathematics, than this is without a doubt true. But I do a lot of work in analytic number theory, and I have a hard time coming up with theoretical physics that I employ. I also have a hard time separating theoretical physics from pure mathematics - both are largely mystical, loosely formatted, and open. One might argue that theoretical physicists concern themselves more with descriptions of the natural world while mathematicians only concern themselves with what is consistent rather than what is possible... but I don't know how I feel about that either.
I will say, however, that I think both applied math and theoretical physics rely heavily on 'pure math.' I think of fields such as Lie Theory, which I consider a pure math. Lie Groups are one of the fundamental tools used in theoretical quantum physics these days - the existence of many particles and symmetries is often suspected because of mathematics and thoughts birthed within Lie Algebras.
But perhaps it is nice to know the awesome power of pure mathematics sometimes. When learning about Lie Algebra, or any sort of Abstract Algebra, I think it might be very enriching to learn about the solutions to the quantum harmonic oscillator and/or particle in a box situations. While these can be done analytically (or through divine inspiration, as seems to be what my old professor expected of us), Dirac employed a very cool use of algebra to solve these things. This included group-like behavior and the creation of 'ladder operators,' and more can be found at wikipedia. In this sense, I do appreciate pure mathematics more because of this knowledge of applied mathematics.
A large amount of group theory can also be applied to quantum mechanics. The astounding properties of the Pauli spin matrices might seem flukelike, but they can be predicted and analyzed from a group-theoretic context as well. I suspect many find is vastly satisfying to be able to predict experimental results from entirely theoretical pursuits.
As a very pure mathematician, one of the questions I am often asked is 'Why do people care about what you do?' While I might come up with something good to say, I think it is reasonable to say that the majority of the work that pure mathematicians do will never find an application that people would declare 'useful.' Thus whenever a bit of pure mathematics is 'applied,' this re-convinces people that funding the study of pure mathematics is a worthwhile endeavor. I rely on that belief for my own funding, so in that I appreciate applied mathematics a whole lot :P.
I will end with one more example. I again refer to Dirac, because I happen to know a lot about his life and how he went about his research. Dirac championed the use of projective geometry (and what I think we would now classify as differential geometry) to discover physics. Here is a transcript of a talk he gave about this subject at one time. Geometry is an interesting thing, because it's often visually based or well-grounded on intuition. Although I am not a geometer, I nonetheless am very pleased whenever I can use a physical situation (even quantum physical, slightly less intuitive) to better interpret some sort of geometric situation. Similarly, it is nice to be able to apply a geometric intuition to an apparently non-geometric problem from the applied sciences. I really encourage a quick glance through the transcript.
This is what I have for now.
------Post Edit------
I happened across a few links that I think talk about this subject nicely. One is a site that addresses how physics does not come from math alone - I take this as an example of how math is too general (in general) for physics. The second is a paper that talks about three very high-level math things that arose out of physical interpretations. A nice addendum, I hope.
Comment: Don't worry, it was a good read.
Comment: It's a lot longer than I thought it would be.
Comment: @mixedmath: Conformal field theory, motivates and helps to solve many problems in analytic number theory.
Comment: You write that you are a graduate student using a lot of analytic number theory, and you have a hard time finding a connection between your work and theoretical physics.  Random matrix theory, first intensively developed by physicists, plays an important role in motivating numerous conjectures about L-functions in analytic number theory.
Comment: Outstanding answer. Alas, the link to the transcript is broken. Please can you provide us with the proper link? It would be very much appreciated.
Answer: It is certainly possible to study all kinds of topics in pure mathematics without any knowledge of physics, because you will always find literature/researchers who are used to explain the key concepts to fellow mathematicians without any knowledge in physics.
But here are some examples of useful interconnections, in arbitrary order:

The most prominent example of recent history is probably the work of Edward Witten on low dimensional topology that earned him the Fields medal. Wittens work is inspired by thinking about quantum field theory and extensions, using heuristic tools like path integrals. He was very successful, so that other mathematicians have tried to learn tools from theoretical physics, too, which led to seminars which led, e.g., to the two volume book "Quantum Fields and Strings: A Course for Mathematicians". His methods resulted in reducing the length of some proofs of Donaldson from 100+ pages to < 10 pages.
Classical mechanics is the study of symplectic manifolds (this is a simplification, of course). Much of the work of Poincaré on ordinary differential equations was inspired by the study of classical mechanics.
Principle bundles is about the study of both classical and quantum gauge theories, which encompasses electromagnetics, for example. The close ties have led Dale Husemöller to (co-) author "Basic Bundle Theory and K-Cohomology Invariants (Lecture Notes in Physics)", which is an extension of his classic textbook "Fibre bundles".
Quantum mechanics is about linear operators on Hilbert spaces. Most of linear functional analysis has been developed in order to understand quantum mechanics, espacially everything that von Neumann did.
Differential geometry on Lorentzian manifolds is the study of the theory of general relativity = gravitation.
Operator algebras (C* and von Neumann algebras) is about axiomatic quantum statistical mechanics, see Robinson, Bratteli: Operator Algebras and Quantum Statistical Mechanics 1 and two. And it is about axiomatic quantum field theory, see Haag: "Local Quantum Physics". The structure theory of von Neumann algebras finds deep applications here, for example the classification of factors is applied to local algebras, modular theory is connected to representations of the Poincaré group etc. You already mentioned the work of Connes et. alt. about noncommutative geometry and their application to quantum field theory and the standard model. Noncommutative geometry is also applied in order to construct quantum versions of spacetimes (i.e. quantum geometry).
Fluid dynamics is about solutions to the Navier-Stokes equations. The flow of ideal fluids forms infinite dimensional manifolds, see "V.I. Arnold, ; B.A. Khesin: Topological methods in hydrodynamics."
Someone else than me should explain the applications of complex and algebraic geometry to string theory. (And n-categories.)

All of these topics can be studied without mentioning theoretical physics, although I think it would be a shame to do that.
Comment: And Random Matrix Theory.
Comment: And classical statistical mechanics as an application of probability theory. And connections between conformal theory and vertex operator algebras. And application of topological tools in condensed matter theory. And relation of quantum groups and knot theory to many quantum and statistical physics problems (via YBE)... actually, I think it's hard to find any area of modern physics that wouldn't have deep connections with modern math too... :)
Answer: I suggest to read
Missed Opportunities by Freeman Dyson an essay that describes missed opportunities in mathematics due to the fact that mathematicians were not interested in new developments in physics (e.g. investigating the symmetries of the new Maxwell equations).
It is certainly wrong that a knowledge of physics is required for mathematics, but a widespread ignorance of developments in areas that use mathematics leads to missed opportunities and missed intuition.
Edited to add: It might be worthwhile to note that the quote does not say that mathematics or doing mathematics requires knowledge of physics, but the proper appreciation of mathematics requires it. The book sets out to give an overview of mathematics and its development and requires some physics knowledge to do so. The fact that it is perfectly possible to work happily and productively in many branches of pure mathematics without getting your neurons dirty with physical intuition is not in contradiction with this.
Answer: Just a couple of isolated -- and manifestly non-expert -- remarks.
1) I am right there with you wondering about the usefulness of physics knowledge in mathematics.  Here is my own story: I had the good fortune of attending one of the finest public high schools in the United States: Central High School.  They had pockets of strength all around (in particular my senior year English teacher was one of the most learned people I have ever met; once I compared Greg Kuperberg to him on MO, and Greg was suitably flattered!) but in retrospect it is fair to say that their programs and math science were head and shoulders above the rest.  I took every Advanced Placement science course they had to offer -- so Biology, Chemistry and two years of Physics -- and got college credit for all of them.  But somehow I wasn't thrilled with these courses: there was a lot of memorization and solving of problems calibrated to be so-and-so difficult.  As an undergraduate I went to the University of Chicago, which has one of the most extensive "core programs" among US universities.  Moreover as a math major one had to take four non-math courses in the Physical Sciences Collegiate Division beyond core requirements.  But because of all this AP, I was able to place out of all my core requirements and still have something left over to apply to the major requirements: the only "science" classes I took as an undergraduate were two quarters of computer science (in Pascal, if you can believe it).  Instead I took, as you might expect, a bunch of math courses but also -- as you might or might not expect, depending upon how well you know me -- a bunch of humanities courses of various types: Shakespeare, classical film theory, Paradise Lost...(Each of these courses was a real treat and a welcome change of pace.  I felt much more comfortable solving two problem sets a week and writing a paper every two weeks than solving four problem sets a week.  Now that I think about it, maybe things haven't changed that much: I still find writing rewarding and even somewhat relaxing and make it a substantial part of my "work time".)
As a result of this, although I got some kind of training in the very basics of physics as 17 year-old, I have taken zero serious, university-level physics courses.  For a while I was somewhat amused and somewhat self-conscious of this and asked around to see how common this was.  I met a few people who had taken as little physics as I had; I can't remember anyone who admitted to taking less.
How do I feel about this state of affairs as an adult mathematician?  I can definitely say that I wish I knew more physics than I do, or at least had the kind of robust, thoughtful understanding of the stuff I learned in high school that I do for basic mathematical concepts.  I really didn't understand electricity and magnetism very well (let's be honest: deep understanding is neither required nor tested in AP courses): e.g. things like capacitors are very mysterious to me.  This lack of understanding does come up in my work sometimes: once I refereed an arithmetic geometry paper in which there was some harmonic analysis on graphs that the authors wanted to phrase in the language of electric circuits.  That was a complete no-go for me, and I insisted that they write in the language of mathematics, not physics.  (I also borrowed a book on circuits from a colleague of mine who is a real expert in...ahem...capacity theory on algebraic curves.  But I never read it.)
So, yes, there are aspects of mathematics related to my own research interests that I don't understand very well because of my lack of physics background.  Of course there are plenty more aspects of mathematics related to my own research interests that I don't understand very well because of my lack of mathematics background in various topics: sometimes I fantasize about how dangerous I would be if I really understood algebraic stacks or derived categories, for instance.  The point being that no one (almost no one...a few people active here and/or on MO come close enough to earn my admiration and envy) has a perfect background in everything that you would need to understand all the math you want to understand.  My decision to read and write about Madame Bovary instead of, say, learn about quantum mechanics was the right one for me at the time.  I am happy to be the guy who has spent his adult life knowing something about Flaubert -- it's part of who I am.  Probably some day I will really need to learn some physics...and I don't think learning undergraduate physics is going to be any harder than learning about algebraic stacks, even if I wait until I'm in my 40's to do it.
2) [much shorter] Some parts of contemporary "theoretical physics" would be better thought of as being branches of mathematics: any differences that may exist there are more sociological than intellectual.  This is already fairly true about string theory from what I have seen: string theorists are mathematicians who are bundled up together and working furiously on a tightly bunched collection of mathematical problems.  It's like they are attending a working conference that has lasted for $20$ years: they're too busy talking to each other and making day-by-day progress to write things up according to the much more sedate standards of mainstream mathematical publishing, so it's a big flurry of arxiv preprints with arguments intended to convince each other rather than be formally complete...but does anyone really think these people aren't mathematicians?  (The non-string theory physicists sure view them that way, from all I've heard.)
The OP mentioned Matilde Marcolli.  I would say she's not even a physicist in the nominal sense of the string theorists: she's a mathematician through and through (I wish I were a mathematician like she is a mathematician).  I just consulted her wikipedia article and was mildly surprised to learn that her undergraduate degree was in physics (but her thesis was on self-equivalences of fiber bundles!). It would be very interesting to hear more about her own background in physics and her take on the subject: from my outside perspective, it seems like it is mostly a source of interesting problems and intuition.
Comment: I knew Mordell went to your high school, before checking the Wikipedia link, but hadn't known Joe Harris was a student there too.
Answer: No, one does not need to know anything about physics to learn mathematics. But learning theoretical physics has at least two advantages. Theoretical physics could be seen as a set of good examples of mathematics, so this has some didactic value. And, if you are a math researcher, a lot of new math were born out of physical problems! So it's probably a good idea to keep an eye out for what's going on in the world of physics.
Answer: One example where I think some kind of physical or real-world intuition might be indispensable is in the study of differential equations.  (OK, most of such study is already semi-applied in nature and arguably not "pure math", but pure mathematicians do occasionally draw on the theory, or need to understand specific equations.)
A central theme in understanding the solutions of ODEs and PDEs, even without solving the equations directly, is identifying quantities related to solutions that either don't change, or change in some regular fashion (e.g. they decrease over time, or whatever your name is for the variable you are interested in).  You can think of these as just formal expressions that you want to find--- but in DEs of interest to physics, they often are physically meaningful things: quantities that are conserved or dissipated in some physically understandable way as time goes on.  So in the literature, even in the pure math literature, you will see integral expressions for things that are arbitrarily given names like "energy" or "entropy", and then it's shown that for solutions of whatever DE you are interested in, they behave in some regular fashion.  And you can understand this on a formal level, but I think it helps to have a background in physics and applications to understand why you might write these things down.
(As a concrete example, the Lyapunov method of showing that a dynamical system is stable involves finding an auxiliary function $V$ with specified properties related to the system you started with.  You can think of it just as a formal thing that helps you do what you want--- prove stability--- but it is often called a "potential," and in applications you often find that functions that capture the application's sense of "potential energy", will satisfy the technical conditions you need in order to apply the method.)
Answer: A more simple example than the very good examples given above is: find the geometric shape with the minimal face area that is based upon the vertices of a cube. A mathematician would have to do a great deal of calculations, but a physicist would just drawn the cube in a bucket full of soap water.
P.S. In his book "The Music of the Primes" the author, Marcus du Sautoy, tells us about how the Quantum Mechanics helped to predict and give new information about pure mathematic research. It is a book easy to read and I enjoyed it a lot.
Answer: I came across this site after reading the question posted here and some of the answers. I really enjoyed it. There are additional links on the page pursuing the discussion, providing examples to consider.
Enjoy!  Chicken-or-egg? Math-Physics?
Answer: My own knowledge of pure mathematics is very limited, but I have observed that ideas from physics at least motivate "pure math" results. For instance, I have heard that Riemann's original proof the Riemann Mapping Theorem was based on intuition about electrostatic forces, as is Dirichlet's principle (http://en.wikipedia.org/wiki/Dirichlet's_principle). Similarly, some properties of harmonic functions stems from the intuition that an initial charge distribution on the boundary must stabilize.
So, in my own experience I have not encountered mathematics where knowledge of theoretical physics is "required" (but I have not dabbled in mathematical physics, where I'm sure this is true), but even at my elementary level I've seen a lot of mathematical ideas which are motivated by physics. Even more vaguely, I've found that some physical principles prove useful in grasping the content of mathematical theorems -- for example, thinking about degrees of freedom involved, special cases, etc.
Answer: There's a big difference between mathematical physics and theoretical physics.  The former is essentially a branch of mathematics.  A physicist would motivate, for example, generalized functions, by pointing out that ever real measurement involves an integral (average) of an "observable field" with respect to a "test function." While you can certainly master generalized functions without ever speaking of measurements, I can't help thinking you will have a limited perspective the subject.
When I visited Cambridge, the pure mathematicians would send offprint request to the applied mathematicians by post.  But then, to the former, DAMPT was the department of awful mathematicians trying to be physicists.  Vive la différence.
Comment: I reiterate that there is now quite a distinction between Maths-Physics (MP) and Theoretical-Physics (TP), with the former being, a branch of maths, while the latter is an area of physics that covers far more than only String Theory/ High Energy Physics/ Relativity, as is often assumed. Rather, all sorts of topics within the fields of Theoretical Particle Physics/ Condensed Matter Theory/ Fluid Mechanics/ Computational Physics/ Astro-Particle Physics etc. are parts of TP.
Really the distinction is that MP are mathematians who think about physics, while TP are physicists who think about maths!


Verificar retorno $.getJSON

Question: Preciso saber se retornou algo do banco de dados. Alguma sugestão?
var url = "agenda_salao_pesquisa.php?dataReserva="+busca;

$.getJSON(url, function(result) {
 //preciso saber se veio algo da consulta
            $.each(result, function(i, field) {

                var id = field.id;
                var dataReserva = field.dataReserva;
                var horaReserva =  field.horaReserva;
                var unidade = field.unidade;
                var responsavel = field.responsavel;
                var salao = field.salao;
   });
});

Comment: @ValdeirPsr nada acontece. Na verdade eu preciso preencher uma div caso não retorne nada.
Comment: a requisição traz algum valor ou retorna um valor em branco?
Answer: A função do método $.getJSON só vai ser executada se vier um objeto JSON válido. Para verificar se deu algum erro, veio vazio, ou um JSON inválido, você precisa adicionar o callback .fail ao método (a partir da versão 3 do jQuery):
Versão 3.0 ou maior:
$.getJSON(url, function(result) {
   console.log("Deu certo. JSON válido!"); // aqui é a função success
})
.fail(function() {
   console.log("Deu erro. Veio nada, veio JSON inválido etc");
});

Versão anterior a 3.0:
$.getJSON(url, function(result) {
   console.log("Deu certo. JSON válido!"); // aqui é a função success
})
.error(function() {
   console.log("Deu erro. Veio nada, veio JSON inválido etc");
});

Informações completas você pode consultar a página oficial do método.
Comment: perfeito !!!!!!!!!!
Answer: Você pode usar o if(result){}
Ele vai retornar true se o retorno **Não for: **

null
undefined
NaN
empty string ("")
0
false

Fonte
Comment: Eu preciso saber se não retornou nada, fiz if(!result){} e fiz como voce falou porem com else e nada
Comment: você já deu um `console.log(result)` para ver o que está vindo?
Comment: nao vem nada quando o resultado é vazio, quando coloco um parametro que retorna algo vem certinho

Verificar retorno $.getJSON


Como retornar um uma resposta do sql?

Question: Essa é minha primeira publicação num forum, então peço desculpas se cometer algum erro.
É o seguinte, preciso cadastrar um livro no banco, mas antes disso, precisaria saber se o autor já está cadastrado no banco, pois se nao tiver, devo cadastrar primeiro.
Então pensei em criar um metodo "verificaAutor" que consultaria a tabela Autor no banco, e caso nao houvesse o nome digitado pelo usuario, direcionaria para a tela de cadastro.
Pensei em dar um select no banco, e caso nao encontrasse nada, retornasse null,
tipo um
if (RESULTADODOBANCO = null)
         cadastroAutor.setVisible(true);

Porem nao faço ideia de como fazer isso, fiz um
String sql="SELECT nome FROM autor WHERE nome=?";
    try {
        PreparedStatement stmt = c.prepareStatement(sql);
        stmt.setString   (5, livro.getAutor());

no meu metodo verificaAutor, mas nao sei como saber se retornou null
PERDOEM-ME SE ESTIVER MUITO CONFUSO
Comment: Acho que uma abordagem para pular essa etapa de verificar se o autor que o cara digitou existe, seria, ao invés dele digitar o nome do autor na hora de cadastrar o livro, você criar um `select` com todos os autores do Banco de Dados, e o usuário selecionar qual ele quer.
Comment: Rick, obrigado pela resposta rápida. Esse trabalho tá me surtando! kk
No caso, o cliente iria estar na tela de cadastro do livro, digitaria o nome do autor, e na hora que clicasse no botao CADASTRAR, chamaria o metodo verificaAutor(), esse método verificaria no banco se o autor está cadastro, se nao, encaminharia o usuario para a tela de cadastro
Comment: No caso vc quis dizer que tivesse tipo uma tabela com o nome de todos os autores cadastrados, e caso nao encontrasse, o usuario clicasse no botao pra cadastrar o autor, isso?

Desculpe pela complicacão, sou novo em programação
Comment: Muito Bem Weslley, só dei uma sugestão de uma outra abordagem, mas já vou responder sua dúvida.
Comment: A abordagem que sugerir, é que na tela de cadastro do livro, tenha um campo do tipo `select` onde será listado todos os autores e o usuário irá escolher qual ele quer, assim não existe a possibilidade dele escolher um autor que não existe.
Answer: Muito simples amigo. Tendo o seu ResultSet em mãos, bastá você fazer um simples if no méteodo next() dele.
Exemplo:
if (rs.next()) {
    // Autor encontrado
} else {
    JOptionPane.showMessageDialog(null, "Autor não encontrado");
    // Redirecionar para o cadastro
}

Comment: O metodo next() retorna true ou false?

Ou seja, se retornar false, significa que nao encontrou nenhum registro?
Comment: Sim Weslley. Se a minha resposta te ajudou, de um certo nela, isso vai me ajudar bastante

Como retornar um uma resposta do sql?


Are these vectors linearly independent or linearly dependent?

Question: I am really confused if $\alpha_1=(e^{\pi/2},1)$ and $\alpha_2=(\sqrt[3]{110},1)$ are linearly independent or linearly dependent in $\mathbb{R}^2$. (Hoffman and Kunze, page no. $48$.)
Consider $c_1\alpha_1+c_2\alpha_2=0$. Then $c_1(e^{\pi/2},1)+c_2(\sqrt[3]{110},1)=(0,0)$. This gives two equations:
\begin{align*}
c_1e^{\pi/2}+c_2\sqrt[3]{110}&=0\\
c_1+c_2&=0.
\end{align*}
Then we get, $c_1=-c_2$ and $c_2(\sqrt[3]{110}-e^{\pi/2})=0$. Now, $e^{\pi/2}$ and $\sqrt[3]{110}$ are numerically very close. So for $0<c_2<1$, the smaller $c_2$ becomes, the closer the latter equation is to $0$.
Can we conclude that $\alpha_1$ and $\alpha_2$ are linearly dependent on basis of this?
But then if two vectors are linearly dependent, one of them is a scalar multiple of the other. I don't see how $e^{\pi/2}$ and $\sqrt[3]{110}$ are scalar multiple of each other. (I am only discussing the first components of each tuple because $1$ is clearly scalar multiple of $1$.)
Any idea if they are linearly independent or linearly dependent? Also, what about the linear independence/linear dependence of the set $\{e^{\pi/2},\sqrt[3]{110},1\}$ in $\mathbb{R}$?
This might be very simple and I am complicating things unnecessarily, nevertheless any explanation to clear my confusion would be appreciated. Thanks!
Edit: Thank you. I found each your answers helpful (except where I mentioned below). I wish I could accept more than one answers I found helpful :)
Answer: The vectors are linearly independent. Otherwise, one of them would a multiple of the other one (as you wrote). But, since the second coefficient of both vectors is $1$, that would mean that $e^{\pi/2}=\sqrt[3]{110}$, that is, that $e^{3\pi/2}=110$. But they're different.
Answer: "Close to zero" and "zero" are very much not the same thing when it comes to vectors (or, really, most things in math). The only actual solution to the equation $c_2(\sqrt[3]{110} - e^{\pi/2}) = 0$ is $c_2 = 0$; the math doesn't care if "small" $c_2$ gets you "close", it only cares about exact equality. In fact, if close enough were good enough, we could conclude by your argument that any two vectors were linearly independent - for sufficiently small $c_2$, for example, $c_2(2 - 1)$ is also very nearly zero.
Because the only true solution to your system of equations is $c_1 = c_2 = 0$, the vectors must be linearly independent.
Comment: Correct, but in numerical computation, its important to be aware of situations in which vectors are effectively linearly dependent (or matrices are effectively singular) to floating point precision even though they may be linearly independent in exact arithmetic.
Comment: @BrianBorchers Yes, that's what I was wondering when I said they are numerically close. Would such a thing happen here?
Answer: As noted in the other answers the key fact here is that $e^{\pi/2} \ne \sqrt[3]{110}$. And we can be sure of this fact also without  an explicit calculation of the two numbers, because $\sqrt[3]{110}$ is an algebraic number, but $e^{\pi/2}=(-1)^{-i/2} $ is transcendental by  the Gelfond–Schneider theorem.
This  is important for the answer to your second question:
the three numbers (see as vectors) $\{\sqrt[3]{110},e^{\pi/2},1 \}$  are linearly dependent in the vector space $\mathbb{R}$ over the field $\mathbb{R}$ ( obviously), but are linearly independent in $\mathbb{R}$ over the field $\mathbb{Q}$, because there does not existst a rational number $q$ such that $\sqrt[3]{110}=qe^{\pi/2}$
Comment: Thank you so much. It clears away my confusions.
Answer: from the second equation we get
$$c_2=-c_1$$ and then you will get
$$c_1e^{\pi/2}-c_1\cdot \sqrt[3]{110}=0$$
Comment: I already have these equations in the above post.


error message when inserting into table from another table

Question: I am trying to insert value in product table from product2 table like this.
INSERT INTO product
     SELECT *
       FROM product2
      WHERE product_code = 8002-25768;

But it is showing the following message .

ERROR 1136 (21S01): Column count doesn't match value count at row 1

please help me
Thanks
Comment: Welcome to SO. Please also add the tag 'sql' to your question.
Answer: check if tables have same structure (the same number of columns and same type for columns at the same index)
Answer: It means that product has different column count than product2; so either product2 has spare columns or product has unassigned ones. I suggest rewriting  the query into
INSERT INTO product(p1, ..., pN)
     SELECT column1, -- this will be written into p1
            ...
            columnN  -- this will be written into pN
       FROM product2
         -- do you really want subtraction: 8002-25768 = -17766??? Probably '8002-25768'
      WHERE product_code = '8002-25768';

Please, notice that each pK target column corresponds to columnK source one.
Comment: You are right sir, it is '8002-25768' not subtraction .


How can I make Apache Solr Autocomplete work in search Views?

Question: I'm trying to work with Apache Solr search in my view and make this autocomplete.
Apache solr is configured correctly and I've added a solr view. I can search on fields, that's no problem. But how can I make the autocomplete work?
Answer: Have you tried to search in google? before asking. Have you tried to do anything?
 I noticed you're constantly asking questions to which the answer can be found by spending a little time searching.
Try to do something by yourself before asking. thanks.
ApacheSolr Autocomplete module - This module should solve your problem.
Comment: I'm working with that module but documentation is very poor.


Scrolling Textview in android

Question: In my application i have to implement an automatically scrolling textview, I referred this link    .
Now I have a scrolling tetxview.But according to my requirement is that I have a string array(I have parsed and I have some string)..Consider the array may be
 string[] array=new string{"fgsdfd","gdfghdjhsd","gdfjhsgfhds"};

Now i want this array to be displayed in  that textview(which will scroll automatically).
I want like this:
 fgsdfd   gdfghdjhsd    gdfjhsgfhds------------------>this will scroll automatically

This is my textview(scrolling):
 <TextView
    android:text="Really Long Scrolling Text Goes Here.... ..... ............ .... ...."
    android:singleLine="true"
    android:ellipsize="marquee"
    android:marqueeRepeatLimit="marquee_forever"
    android:scrollHorizontally="true"
    android:id="@+id/TextView03"
    android:padding="5dip"
    android:layout_width="wrap_content"
    android:textColor="#000000"
    android:layout_height="wrap_content" />

How can I set an array of string to a tetxview..Please help me.
Answer: You can merge all of your string in one string by StringBuilder and apply it to the TextView.
I've implemented my own TextView with scrolling by wrapping textview (and maybe some other views)
private Runnable scrollRight = new Runnable()
{

    @Override
    public void run()
    {
                    // can control scrolling speed in on tick
        topScroll.smoothScrollBy(1, 0);
    }
};

and in new Thread I invoke:
while (!interruptScroll){
    try{
        Thread.sleep(50); // control ticking speed
    }
    catch (InterruptedException e){
        e.printStackTrace();
    }
    topScroll.post(scrollRight);
}

and by manually scrolling the scrollView i interrupt the scrolling (such an automatic scrolling while not interrupted by user).
Comment: And BTW - to implement autoscroll use ScrollView, smoothScrollBy and threads\timers.
Comment: k...Thanks dilix...Can u do 1 more favour..In above code the scrolling is happening very slowly..i want them to scroll little bit fast..How can i accomplish in above code..
Comment: first part is only a variable and post second part in (new Thread("you code in Runnable")).start(); and your scrolling code will be run in other thread.
Answer: Try using StringBuilder.
String[] array = { "fgsdfd", "gdfghdjhsd", "gdfjhsgfhds" };
StringBuilder sb = new StringBuilder();

for (int i = 0; i < array.length; i++) {
    sb.append(array[i]);
}
txtView.setText(sb.toString());

Answer: try for this,
<TextView
android:text="Really Long Scrolling Text Goes Here.... ..... ............ .... ...."
android:singleLine="true"
android:ellipsize="marquee"
android:marqueeRepeatLimit="marquee_forever"
android:scrollbars="horizontal"
android:id="@+id/TextView03"
android:padding="5dip"
android:layout_width="wrap_content"
android:textColor="#000000"
android:layout_height="wrap_content" />

Scrolling Textview in android


The meaning of upper 100 alpha(th) percentile

Question: Recently, I learned about the chi-square distribution. In my class, I was told about the upper $100\alpha^{th}$ percentile $\chi^{2}_{\alpha}(k)$ and given the following definition:
$$P(X<\chi^{2}_{\alpha}(k))=1-\alpha$$
$$P(X>\chi^{2}_{\alpha}(k))=\alpha$$
I have tried my best while looking online for additional information using a given name but got almost nothing( I do admit I might have looked for the wrong thing). Can someone tell me the formal name of this concept? Additionally, why would we even have something like this if a general concept of percentile should be sufficient? Also, do we use it anywhere else outside of the chi-square distribution?
Comment: The formal name is "percentile."  This requires a convention concerning whether percentiles run from smallest to largest (which is commonest) or from largest to smallest.  "Upper" is a vague attempt to indicate the latter convention is used -- but there's no problem with the vagueness because the definition makes the meaning perfectly clear.
Comment: @whuber. Thanks, I was confused by the notion of percentiles always being smallest to largest
Answer: This is just standard definition to make notation less ugly. One area you will often see this kind of notation is in building confidence intervals. Basically, $\chi^2_\alpha(k)$ is the value such that $P(X > \chi_\alpha^2(k)) = a$. This is common when building confidence intervals: you have some theory that tells you that the test statistic follows some distribution that you know how to explicitly work with, such as chi square distribution or the normal distribution (or asymptotically does).
A common situation is when you want to build a confidence interval for the mean given data $x_1,\dots,x_n$. Let $\hat{\theta}$ be the sample mean. Assuming the CLT holds, we know that asymptotically,
$$Z={\frac {{\hat{\theta}}-\theta }{\sigma /{\sqrt {n}}}} \sim N(0,1),$$
and so if I want to build a 95% CI for $\theta$, then I claim it's enough to just know values $z^*_l,z^*_u$ such that
$$P( z^*_l \leq Z \leq z^*_u) = .95$$
Why? Well I can just massage that equation given that I know $Z$ to build the CI as follows:
\begin{align*}
 .95 = P( z^*_l \leq Z \leq z^*_u) & = P\bigg( z^*_l \leq {\frac {{\hat{\theta}}-\theta }{\sigma /{\sqrt {n}}}} \leq z^*_u \bigg) \\
& = P\bigg( \hat{\theta} - z^*_l\frac{\sigma}{\sqrt{n}} \leq \theta \leq \hat{\theta} + z^*_u\frac{\sigma}{\sqrt{n}} \bigg).
\end{align*}
So great, I just need to know the values $z^*_l,z^*_u$, but I know those: they are precisely the values such that $P( z^*_l \leq Z \leq z^*_u) = .95$ holds when $Z \sim N(0,1)$, and I know exactly how to work with the CDF of the standard normal distribution, so I'm all set after I decide to choose values that are symmetric around $0$. Since I know those values, when denoting them, I may decide to just shorthand $z^*_\alpha$ as the value such that
$$P(-z^*_\alpha\leq Z\leq z^*_\alpha)=1-\alpha.$$
Comment: 1. yes, normal distribution is symmetric, hence why its the same $z_{\alpha}^*$ (worth noting that another common notation for the same value is $z^*_{\alpha/2}$). 2. Yes, the value $z^*_{\alpha}$ is sometimes denoted the critical value (also called the zscore sometimes). See https://www.statisticshowto.com/probability-and-statistics/find-critical-values/.
Comment: I see, thanks for the proper explanation. Just 2 more questions: 1) Does $P( z^*_l \leq Z \leq z^*_u)$ convert to $P(-z^*_\alpha\leq Z\leq z^*_\alpha)$ due to the property that the graph is symmetric( I am talking about the negative sign appearing). And 2) I came across the term *critical value* while looking online. Is it related to this topic as well?
Comment: Okay, thanks for the clarification


$(0,1)^n$ a normal space but $(0,1)^{\mathbb{R}}$ not normal?

Question: Is it true that $(0,1)^n$ is a normal space for any $n \in \mathbb{N}$,
but $(0,1)^{\mathbb{R}}$ is not normal?
This seems like an interesting result and rather counterintuitive and if true would also mean that $(0,1)^{\mathbb{R}}$ is not a metric space.
I came across this as an argument in an answer to an old question but there was no proof of this claim.
How would you prove this?
Answer: It is true. We can replace $(0,1)$ by $\mathbb{R}$ because they are homeomorphic as spaces. Then $\mathbb{R}^I$ is normal exactly if the index set $I$ is at most countable (and then the space is even metrisable). For uncountable $I$ the space is not normal (so certainly not metrisable).
Here it is shown that $\mathbb{N}^I$ is not normal for uncountable index set, and this is a closed subset of $\mathbb{R}^I$ so if the latter were normal, the former would be too, which it is not), and this settles it.
The original paper is here, it's theorem 3. It's due to A.H. Stone. The first link is essentially a more expanded write-up of this argument. The whole paper by Stone is worth a read (a classical paper).
Comment: @JKnecht glad you like it!


What are these brackets? What do they mean and what are they called?

Question: I came across to these brackets in my music theory book of Trinity. I don't know what are these called and what they exactly do in this music?
Answer: These brackets indicate a "first and second ending." The measures under the "1" are to be played the first time through and those under the "2" are to be played the second time.
It's a nice method of notation and can be extended to more choices. There are also things like "dal segno" and "al fine" and others. You should check them out; search the internet for music notation.
This link has a nice summary of the most basic repeats and endings.
Comment: Italian speaker here. Volta means may things, among whose [an] arched ceiling, [you] turn, [this] time. Volta subito: turn now. Prima volta: first time. Seconda volta: second time. By the way, Dal Segno means literally "from the sign" (da+il segno), Dal Capo: from the beginning, and Al Coda: to the "tail" = end.
Comment: Maybe you should add they're called "voltas", since it's still used. And more specifically prima volta and seconda volta
Comment: @CarlWitthoft "volti subito". Volti mean turn. It's also the nickname of my electrician.
Comment: @Creynders  Just be sure not to confuse with "V.S." , 'volta subito,'  indicating you better turn the page quick :-)
Comment: @CarlWitthoft true that, maybe it should be added that volta literally means turn and hence isn't restricted to the volta lines ...
Comment: @Creynders Volta literally means time. You guys need watching :-)
Comment: @Zachiel for some reason, _da capo_ is traditionally without the definite article.
Comment: @Phoog Weird. I have some scores which have "Dal Capo" (as part of the expression "Dal Capo al Coda"). After googling, I see that it is not the standard. Both are grammatically correct, of course.
Answer: The name is volta brackets. Used to mark alternate endings used in conjunction with repeat sign barlines.


What transformation should be carried out if the count is non-integer in poisson glm in r?

Question: My data is like this:
person   final   sex
1        34.20    1
2        2.00     0
3        15.58    0
4        18.00    1
5        50.06    1

I am fitting po = glm (final ~ sex, data=df, family="poisson"), there are warnings because "final" is not a integer. Final is a count of risky behaviours. What can I do to fix this issue?
Comment: What does it mean when the count of risky behaviors is non-integer, such as 34.2?
Comment: No transformation. Either your data are suitable as they come, or you should use a different method.
Comment: When *I* count things I only get integers; if I get non-integer values like 50.06 I did something other than just *counting*. Can you explain how it happens that none of your counts are integers? Are they averages of counts, for example? Are they rates per unit time? Why would you choose to model what are clearly *not counts* as Poisson? The manner in which these non-integral values arise will be important to suitable approaches to the issue.
Answer: When performing Poisson regression, it is assumed that the response variable (in your case, final) is an integer, as the goal of Poisson regression is to model count data.  In this case, a transformation would not be the appropriate solution-- my suggestion is to check how you arrived at the values for final. If final represents counts of risky behaviors it should be composed of all integers.
Comment: Say rather that many implementations assume and enforce this. Others are happy to be more generous. The fact the the normal is a limiting case of the Poisson is one illustration of the idea that it's often not outrageous to regard the Poisson as approximately continuous. Similarly in data analysis many implementations are broader-minded and to good effect. See e.g. http://blog.stata.com/2011/08/22/use-poisson-rather-than-regress-tell-a-friend/ and its references. Arguably, being Poisson even conditionally is no more central to Poisson regression than being Gaussian is to linear regression.
Comment: @Nick You're correct--I've read and appreciated that blog post before. I sympathize with Ben F's answer here, though (+1), because he rightly points out that those warning messages raise a huge flag: although (in the hands of an expert who will know enough to anticipate and ignore such warnings) Poisson regression with non-integral data can be meaningful, very likely what it indicates is the wrong procedure has been selected or that the data have been incorrectly coded.
Comment: @whuber Agreed. The detailed questions about the OP's data remain and the widespread usefulness of Poisson regression does not imply that it is a good choice for his data.


WPF ListView Databound Drag/Drop Auto Scroll

Question: I've been working with Bea's solution here for a while and finding it very helpful.  Problem now I'm having is when I drag-n-drop items within or to another ListView control and I want to scroll up/down "during" the drag (moving an item from index 30 to index 1), it's not happening.  I would have to drag to the top of the visual items in the ListView, manually scroll up, then drag again, eventually ending at the position I want.  This isn't very user friendly.
Now I found the function (DragDropHelper.DropTarget_PreviewDragOver) that I would want to do the testing of which item is being dragged over, and I'm getting that.
Dim pt As Point = e.GetPosition(DirectCast(Me.targetItemsControl, UIElement))

' Perform the hit test against a given portion of the visual object tree.
Dim result As HitTestResult = VisualTreeHelper.HitTest(Me.targetItemsControl, pt)

Now from there I can get the DependencyProperty of this visual hit
Dim lvi As ListViewItem = TryCast(GetDependencyObjectFromVisualTree(TryCast(result.VisualHit, DependencyObject), GetType(ListViewItem)), ListViewItem)

Which is of a ListViewItem.  Now in the function DropTarget_PreviewDragOver I have the "DraggedItem" which is of type Picture in Bea's example, but that can change depending on the ObservableCollection you have bound to the ListView.  Now, I want to drag the ListView up or down depending on where the mouse is on the control.  I've attempted with the below un-finished non-working code
If lvi IsNot Nothing Then
    If pt.Y <= 25 Then
        Dim lv As ListView = TryCast(targetItemsControl, ListView)
        If lv IsNot Nothing Then
            Dim index As Integer = lv.Items.IndexOf(lvi)
            If index > 1 Then
                lv.ScrollIntoView(lv.Items(index - 1))
            End If
        End If
    Else
        If pt.Y >= Me.targetItemsControl.ActualHeight - 25 Then
            Debug.Print("Scroll Down")
        End If
    End If
End If

Can someone point me in the right direction to get this ItemsControl or ListView to scroll when dragging over the items??
Thanks!
Comment: Sorry, I still haven't had time to look into this any more.  Have you ran across something since you posted?
Answer: What I did was took advantage of the ListBox.ScrollIntoView method. Basically, when you update your drop target, you can just call this method on it and wpf will do all the work. All you need to know is the index of the drop target item. This handles both vertical and horizontal scrolling.

this.listView.ScrollIntoView(this.listView.Items[index]);

When you use this method, your adorner might move with the scrolling ListBox. To fix this, I just set my adorner parent and adorner layer parent to the content of the window at the top of the visual tree (i.e. topWindow.Content).
Comment: I'll give this a try, thanks for posting 3 months later :) I thought it was forgotten about..
Comment: thanks! I did it similar

lstMessages.ScrollIntoView(lstMessages.Items[lstMessages.Items.Count-1]);
Answer: I'm still messing around with this exact same issue too.  I'm using a slightly modified version of Bea's Drag and Drop found here, which is in VB instead of C#.  When I used ScrollIntoView as described above, I could scroll down but not up.  So I messed around and came up with this as my DropTarget_PreviewDragOver:
 Private Sub DropTarget_PreviewDragOver(ByVal sender As Object, ByVal e As DragEventArgs)
        Dim draggedItem As Object = e.Data.GetData(Me.m_format.Name)
        Me.DecideDropTarget(e)
        If (Not draggedItem Is Nothing) Then
            If (TypeOf m_targetItemsControl Is ListBox) Then
                Dim lb As ListBox = CType(m_targetItemsControl, ListBox)
                Dim temp As Integer = m_insertionIndex
                Dim scroll As ScrollViewer = Utilities.GetScrollViewer(lb)
                If scroll.VerticalOffset = temp Then
                    temp -= 1
                End If
                If temp >= 0 And temp <= (lb.Items.Count - 1) Then
                    lb.ScrollIntoView(lb.Items(temp))
                End If
            End If
            Me.ShowDraggedAdorner(e.GetPosition(Me.m_topWindow))
            Me.UpdateInsertionAdornerPosition()
        End If
        e.Handled = True
    End Sub

and I had to include this utility function, taken from here
    Public Shared Function GetScrollViewer(ByVal listBox As ListBox)
    Dim scroll_border As Decorator = CType(VisualTreeHelper.GetChild(listBox, 0), Decorator)
    If (TypeOf scroll_border Is Decorator) Then
        Dim scroll As ScrollViewer = CType(scroll_border.Child, ScrollViewer)
        If (TypeOf scroll Is ScrollViewer) Then
            Return scroll
        Else
            Return Nothing
        End If
    Else
        Return Nothing
    End If

End Function

which is great and all.  Then running out what theuberk mentioned above with the adorner moving, and in the spirit of making this easy for someone else later, I added a variable to the DragDropAdorner class:
    Private m_mouseDelta As Point

Added this to the last line of DragSource_PreviewMouseLeftButtonDown:
        Me.m_mouseDelta = e.GetPosition(m_sourceItemContainer)

And turned ShowDraggedAdorner into:
    Private Sub ShowDraggedAdorner(ByVal currentPosition As Point)
    If (Me.m_draggedAdorner Is Nothing) Then
        Dim adornerLayer As AdornerLayer = adornerLayer.GetAdornerLayer(Me.m_topWindow.Content)
        Me.m_draggedAdorner = New DraggedAdorner(Me.m_draggedData, DragDropBehavior.GetDragTemplate(Me.m_sourceItemsControl), m_topWindow.Content, adornerLayer)
    End If
    Me.m_draggedAdorner.SetPosition((currentPosition.X - m_mouseDelta.X), (currentPosition.Y - m_mouseDelta.Y))
    End Sub

Comment: I finally got to implementing this and testing it really quick. Looks great!  Worked just fine!  Thanks!
Comment: I did find one problem though, I was using double clicking on the list box as an alternative to dragging items between two controls.  Using the method like above it was having problems detecting a single click between a double click and the source list would have the item added back what I just removed.  So in the mouse down function I added "If e.Clicks = 1 Then Do Drag Else Set DraggedData = Nothing", that should fix it.
Answer: Another possibility to scroll is to use the ScrollBar-Commands. You can do this without climbing down the VisualTree. If you have a styles ListBox with no border the GetScrollViewer()-Method would not work anymore.
I use the first Element of the ItemContainerGenerator as CommandTarget for the ScrollBar.LineXXXCommand:
Point p = e.GetPosition(itemsControl);
  IInputElement commandTarget = itemsControl.ItemContainerGenerator.ContainerFromIndex(0) as IInputElement;

  if (commandTarget != null)
  {
    if (p.Y < OFFSET_TO_SCROLL)
      ScrollBar.LineUpCommand.Execute(null, commandTarget);
    else if (p.Y > itemsControl.ActualHeight - OFFSET_TO_SCROLL)
      ScrollBar.LineDownCommand.Execute(null, commandTarget);

    if (p.X < OFFSET_TO_SCROLL)
      ScrollBar.LineLeftCommand.Execute(null, commandTarget);
    else if (p.X > itemsControl.ActualWidth - OFFSET_TO_SCROLL)
      ScrollBar.LineRightCommand.Execute(null, commandTarget);
  }

Calling the LineXXXCommands is similiar to clicking the Arrow-Buttons of a ScrollBar: The ScrollViewer scrolles by a certain ammount which you can configure by setting the 'SmallAmount' Property of the ScrollBar.


How to iteration on $scope variable in angularjs?

Question: I have code like this :
<code>$scope.maps = [
    new google.maps.LatLng($scope.longitude[0], $scope.latitude[0]),
    new google.maps.LatLng($scope.longitude[1], $scope.latitude[1]),
    new google.maps.LatLng($scope.longitude[2], $scope.latitude[2])];
</code>
How to simplify in AngularJS style to take more iteration?
Answer:

<code>$scope.maps = [];
$scope.longitude.forEach(function(el, i) {
  $scope.maps.push(new google.maps.LatLng(el, $scope.latitude[i]));
});

// or

$scope.maps = [];
for(var i = 0; i < $scope.longitude.length; ++i) {
  $scope.maps.push(new google.maps.LatLng($scope.longitude[i], $scope.latitude[i]));
}</code>
Comment: Cool, as expectations. Thanks @CodeiSir
Answer: Just to clarify the CodeiSir answer.
Is not "Angular" iteration what he showed is javascript. You can use it for read data too, as you use the angular ng-repeat in the html.
more info here


Should I use a temp table?

Question: I have a report query that is taking 4 minutes, and under the maximum 30 seconds allowed limit applied on us.
I notice that it has a LOT of INNER JOINS. One, I see, is it joins to a Person table, which has millions of rows. I'm wondering if it would be more efficient to break up the query. Would it be more efficient to do something like:
Assume all keys are indexed.
Table C has 8 million records, Table B has 6 Million records, Table A has 400,000 records.
SELECT Fields
FROM TableA A
INNER JOIN TableB B
ON b.key = a.key
INNER JOIN Table C
ON C.key = b.CKey
WHERE A.id = AnInput

Or
SELECT *
INTO TempTableC
FROM TableC
WHERE id = AnInput

-- TempTableC now has 1000 records
Then
SELECT Fields
FROM TableA A
INNER JOIN TableB B --Maybe put this into a filtered temp table?
ON b.key = a.key
INNER JOIN TempTableC  c
ON c.AField = b.aField
WHERE a.id = AnInput

Basically, bring the result sets into temp tables, then join.
Comment: If possible, please post the execution plan for the slow query.
Comment: 6-8 million records isnt that big and shouldnt cause a problem on properly indexed tables. I regularly deal with 400 million record tables which perform just fine for queries.
Comment: @cdotlister - Grab a screen shot of the graphical plan and upload it as an image, or get a text plan and you can then just paste that in as a "quote" text block.
Comment: The execution plan when provided will give insight into where the actual performance overhead is taking place, as well as validate correct index usage. Please share it.
Comment: Thanks chaps - will get it tomorrow. What's the best way to provide it to you?
Comment: @cdotlister - Did you manage to resolve your slow query problem? Great if you did but if not get the execution plan posted for people to help out.
Answer: If your Person table is indexed correctly, then the INNER JOIN should not be causing such a problem.  Check that you have an index created on column(s) that are joined to in all your tables. Using temp tables for what appears to be a relatively simple query seems to be papering over the cracks of an inadequate database design.
As others have said, the only way to be sure is to post your query plan.
Comment: You can do this with getting the query plan.
Comment: It came down to a very silly sub query, which was being run for each row.... Thanks for the help.

Should I use a temp table?


What's the difference between Rackspace Cloud Servers and Rackspace Cloud Sites?

Question: I'm looking to host a video-centric PHP site. Video will be served from CDN.
What does Cloud Site's $150/mo get you?
[Edit]
Seems like with cloud site you get:

web server that's ready to go
a web-based control panel
50 GB of scalable storage space.
500 GB of monthly bandwidth.
Support

Cloud server you get:

to setup everything yourself
pay-as-you-go bandwidth and storage
Comment: Shopping Questions are Off-Topic on any of the [se] sites. See [Q and A is hard, lets go Shopping](http://blog.stackoverflow.com/2010/11/qa-is-hard-lets-go-shopping) and the [FAQ] for more details.
Answer: CloudSites is a shared hosting platform, while CloudServer is a VPS product.
As far as what $150/month on the CloudSites platform gets you, that's readily available on their pricing page.
Comment: Well, for one, many companies don't have the in-house expertise to configure, secure, and maintain a VPS.
Comment: So why pay $150 for shared hosting and a cpanel?
Comment: @Matt Please see my detailed response to the question in your comment.
Answer:
So why pay $150 for shared hosting and a cpanel?

It's actually a cluster of load balanced servers with shared storage served via NFS. The database is on another server which is setup with MySQL replication to a slave for disaster recovery. The email is also served on separate servers. If that isn't enough, there's also a SSL cluster in-front if need be (additional $20/month for SSL I think the rate is). Be aware though, no SSH access and no executables will run either. However, there are cron jobs.
They've recently upgraded the web stats system which has seemed much more stable but of course you can just hook up Google Analytics or process your own logs. Their support is great with about 5-min response (at night) but usually 1-minute response times for their online chat (I haven't used the phone support).
While you're at it, you might as well check out Cloud Files and its CDN ability served via the Limelight Networks.
Check out their wiki/KB if you're looking for more info: http://cloudsites.rackspacecloud.com/ and be sure to compare SLAs and Terms with other shared hosting providers you're considering.
(FYI I don't work for them :P)


SQL query returnin previous weeks

Question: I need the SQL query which will return previous 10 weeks in format 2017W03 for given week in the sam format.
So for week given in format 2017W05 this query should return:
2017W05
2017W04
2017W03
2017W02
2017W01
2016W52
2016W51
2016W50
2016W49
2016W48

Comment: Is the prior year always assumed to have 52 weeks?
Comment: Not, of course! Otherwise I wouldn't ask this Q! Silly thing is that date `TO_CHAR(TO_DATE('01.01.2017','DD.MM.YYYY'),'YYYYIW')='201752'`
Comment: Poblem here is that week format (like `IW`) cannot be used as input in TO_DATE() function!
Answer: Here is a simplified solution, taking advantage of the fact that January 4 always falls in the first ISO week of its year. (January 1 may fall in the last ISO week of the prior year, and that would lead to complications.)
An earlier version (which can be reviewed by clicking on the "edited ..." link below the Answer) did not take advantage of this observation.
Also, credit to tbone - to get the proper ISO year/week combination, we must use the format model component IYYY for the year (in TO_CHAR), rather than YYYY.
with
     input ( str ) as (
       select '2017W05' from dual
     )
select     to_char(to_date(substr(str, 1, 4)||'0104', 'yyyymmdd') +
                     7 * (to_number(substr(str, 6)) - level), 'iyyy"W"iw') as result
from       input
connect by level <= 10
;

RESULT
-------
2017W05
2017W04
2017W03
2017W02
2017W01
2016W52
2016W51
2016W50
2016W49
2016W48

Comment: @sbrbot - Connecting the year to January 1 will cause problems in some years, because January 1 may fall in the last ISO week of the prior year. (That is why I use July 1 - it could be any other date that is guaranteed to fall in an ISO week of the current year.) However, thinking about this issue, I realized that January 4 will always fall in the first ISO week of the current year. This, then, allows for a simpler solution. I will work on it and post it in my Answer above.
Comment: I think we should probably use IYYY instead of YYYY (in the to_char) per the docs (I updated my answer too).  This would handle situation where 20170101 is actually 2016W52 for example.
Comment: Woof, gove me some time to study it! I figured out something else: `SELECT TO_CHAR(TO_DATE(SUBSTR('2017W05',1,4)||'0101','YYYYMMDD')+(SUBSTR('2017W05',6)-LEVEL)*7+1,'YYYY"W"IW') FROM DUAL CONNECT BY LEVEL <= 10` but I have to check if it works for other years.
Comment: @sbrbot - It seems your solution is similar to mine, you just wrote everything in a single formula. I could do the same thing, but I wrote it the way I did so that, at each step, you can see the result. (Remove the comma at the end of any CTE and instead type `select * from ` to see what it generates.) When you are done inspecting the code, you can rewrite my solution in a single, long formula.
Comment: using IYYY should also allow us to use 0101 instead of 0104 (check me on that ;)
Comment: @tbone  - Err... you are right about needing to use IYYY in the output. I am not sure how that helps with the input though (using 0101 instead of 0104).
Comment: Since the OP wants string as input, I'll vote for this answer, don't have any clever way of converting to date other than what you did here :)
Answer: This should give you the output you listed:
select TO_CHAR(to_date('20170201','YYYYMMDD')-(7*(level-1)),'IYYY"W"IW')
from dual
connect by level <= 10

Output:
2017W05
2017W04
2017W03
2017W02
2017W01
2016W52
2016W51
2016W50
2016W49
2016W48

Starting with the arbitrary date of Feb 1st 2017
Comment: The date is not "arbitrary" - this depends on the date being in the fifth ISO week of the year - and that did happen in 2017, but it is not guaranteed to be true every year. In addition, this hardcodes the 05; presumably the OP wants an answer where the input may be any year, any week.
Comment: @mathguy I see, so we're starting with a string not a date.  Not sure what you mean about hardcoding 05
Comment: I assume "for given week in the same format" would indicate that, yes. The OP's attempt (in a comment to my Answer) suggests the same thing.
Comment: The OP's problem is EXACTLY that, given a year and an ISO week number, there is no simple way to generate a date (for example, midnight on the Monday of that week) for that year/ISO week combination.
Comment: In my post I said for "given week", so this SQL will be part of function where week and range number will be given as input parameters and it should return range of weeks. Input week will be week as string in IYYY"W"IW format and range INTEGER. So week 2017W05 (od date 20170201 in that week) cannot be hardcoded.
Comment: Hi sbrbot It wasn't hardcoded for the approach to work, it was just used as an example.  If you have a specific function spec you wanted, it probably would have avoided confusion if you listed that in your post. (input parameters, return values).  Anyway, if your function takes a date, use my approach, if it takes a string, use @mathguy approach.  Both similar.  Hope it helps.

SQL query returnin previous weeks


Within the webbrowser , is it possible to make a  element focus?

Question: I am using a WebBrowser object to render text which is presented in a table format.
This WebBrowser is embedded within an Eclipse plugin project.  In order to comply the accessibility requirement, I need to make each entry in the WebBrower to have focus when using the up / down arrow keys to navigate through the entries so the screen reader can read out the text.
I can detect the up/down arrow keys through the java code, but I don't see the focus to be set on the entry.
document.getElementById("+newPos+").focus();
I think this is because only button / form / text field can have focus within HTML.
Is there a way for the  element to have focus?  Or is there a way for the WebBrowser object to spell out the associated text in the entry when the up / down arrow keys are pressed?
Thanks a lot.
Answer: I had problems setting a focus on a div without having a height associated with it, although I did not put any listeners on it to check.
document.getElementById(newPos).focus();

However, without altering the newPos code to that, it errored.
With heights set, and using that alteration it allowed me to switch focus between two divs.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

    The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.

Change test from 'test2' to test to see the difference. In the second example it focuses, and I can keypress and it will move up and down. I think it's something to do with the scrolling that causes it.
Hope that is of some help.
Comment: The  element I have here is a paragraph of text. Can I set focus on such  element?

Could you provide some sample code in terms of how to set the height you referred to here?

Thanks a lot
Comment: Updated with example. I don't know if that is a decent solution enough solution.


How do I add an image for each menu column?

Question: I have 4 columns in the footer, and am displaying a different menu in each block(column). I would like to add an image above the menu items for 3 out of 4 menus. What is the best way to do this? Is it possible to add a field to the menu content type as such? If this was possible I could then create a view for each block/column, by pulling in the menu items and images...
What would be the best way to do this?
Comment: You can create a menu blocks using menu block module and  alter the render of block to add the image.
Comment: Thank you for that @mohit_rocks
What exactly do you mean by "alter the render of block "?
Comment: to append the image you can use hook_block_view_alter() function. This hook allows you to modify any data returned by hook_block_view(). https://api.drupal.org/api/drupal/modules!block!block.api.php/function/hook_block_view_alter/7
Comment: Are your blocks a menublock creations, or what exactly?
Answer: It's pretty easy to do that with CSS
<code>#column-1{
  padding-top:20px; /* Space for image */
  background: transparent url('../images/bg-col-1.png') no-repeat left top;
}
</code>
Comment: I believe this is the "easiest" way to make the change from among the listed solutions. The only fly-in-the-ointment is that there's no UI for changing the image after the site is handed off, or is there?
Comment: Thank you @David-thomas
I was hoping for the image to be editable via the cms, but this is too easy not to implement as is. Thanks.
Comment: You're welcome, if you want to get slightly more fancy, you may want to consider using a CSS sprite and background-position to combine the images into one.
Answer: Additionally, if you need to add individual image for each menu item in a particular menu block you can use this module
You can try Menu Attributes module. You can add specific class and do the css.
This simple module allows you to specify some additional attributes for menu items such as id, name, class, style, and rel.


Lipschitzian function implies uniformly continuity - choice of delta

Question: On this link which is french page of uniformly continuity, it is said that if a function is k-Lipschitzian, then this function is uniformly continuous.
They say that by choosing a $\delta$ as :
$$k\delta \leq \epsilon$$
Shouldn't it be rather $\delta$ as :
$$\delta \geq \epsilon/k\quad\quad?$$
Indeed, for a fixed $\epsilon$, we want to choose a $\delta$ as :
$$\dfrac{f(x+\delta)-f(x)}{\delta} = \dfrac{\epsilon}{\delta}$$
Moreover, we have with k-Lipschizian definition :
$$\dfrac{f(x+\delta)-f(x)}{\delta} \leq k$$
So we can choose $\delta$ as :
$$\dfrac{\epsilon}{\delta} \leq k$$
which is equivalent to: $\delta \geq \epsilon/k$, isn't it ?
Any help is welcome
Answer: You want $\left|\frac{f(x+\delta)-f(x)}{\delta} \right| \leq \frac{\epsilon}{\delta}$ or to be more precise:
$$\left|\frac{f(y)-f(x)}{\delta} \right| \leq \frac{\epsilon}{\delta}, \forall y \text{ such as } |x-y| \leq \delta$$
And you have:
$$\left|\frac{f(y)-f(x)}{\delta} \right| \leq  \frac{k |x-y|}{\delta}\leq k, \forall y \text{ such as } |x-y| \leq \delta$$.
So you have to choose $\delta$ such has:
$$a \leq k \Rightarrow a \leq \frac{\epsilon}{\delta}$$
or in term of intervals:
$$(0,k] \subset \left(0,\frac{\epsilon}{\delta} \right]$$
which gives:
$$k \leq \frac{\epsilon}{\delta}$$
i.e
$$\delta \leq\frac{\epsilon}{k}$$
Comment: what does `a` represent ? $ \left|\frac{f(y)-f(x)}{\delta} \right| $ ? I have difficulties with the implication $a \leq k \Rightarrow a \leq \frac{\epsilon}{\delta}$, how do you get it ?
Comment: $a$ is just here to show that if you *want* a quantity (called $a$) smaller than $\frac{\epsilon}{\delta}$ and you *know* that $a \leq k$ then you have to choose $\delta$ such that $\frac{\epsilon}{\delta} \leq k$.

$\left| (f(y)-f(x))/\delta\right|$ as no particular signification, the good quantity to consider is $|f(y)-f(x)|$ I was just trying to stay close to your question.
Comment: I think that $a\leq k \Rightarrow a \leq \frac{\epsilon}{\delta}$ can be understood graphically.

Draw the interval $(0,k]$. You know that $a$ is in this interval. Now how can you draw the interval $(0,\frac{\epsilon}{\delta}$ such that you are certain that $a$ is in this second interval ?
Answer: The definition of Lipschitz (even the French one) says that a function $f$ is Lipschitz (on some interval $E$) if
$$|f(x) - f(y)| \leq k|x-y| $$
for some $k$, for any choice of $x,y \in E$.
Now, given any $\epsilon$, if we pick $\delta \leq \epsilon/k$, then for any pair
$x,y \in E$ with $|x-y| < \delta$
$$|f(x)-f(y)| \leq k|x-y| < k \frac{\epsilon}{k} = \epsilon $$
which means by definition that $f$ is uniformely continuous on $E$.
Picking any $\delta \geq \epsilon/k$ makes no sense, what if I pick a huge $\delta$?


How can I make use of zip codes when I am building a model for fraud detection

Question: I have gone through few articles but I am not convinced on what should I do with these. I know from business standpoint it might be good to consider fraudulent transactions happening from unknown locations. But I don't know how to use this in my data as dummy encoding might not be good solution.
how to represent geography or zip code in machine learning model or recommender system?
Comment: Have a look at [principled-way-of-collapsing-categorical-variables-with-many-levels](https://stats.stackexchange.com/questions/146907/principled-way-of-collapsing-categorical-variables-with-many-levels)
Answer:
If you have many independent observations per zip code and ample compute resources, you might try to use the zip code as-is.
Instead of including a categorical variable for zip code, you might acquire demographic data for each zip code, join it to your data set, and include those numeric variables instead.  Perhaps income level, voting behavior, etc. could conveniently describe the variance in the response, saving you hundreds or thousands of dummy variables.
Statistics has an elegant and powerful way to include categorical variables with many levels:  Random effects.  In this case, the coefficients for each zip code would be assumed to come from a normal distribution, and cleverly fit with maximum likelihood instead of least squares.  It has the effect of keeping the coefficients for the zip codes closer to each other (controlling overfitting) and keeping the standard errors of the estimates under control.  The most commonly-used function for this is lmer from the R package lme4.
Answer: Why do you think that

... as dummy encoding might not be good solution

? If it is because of memory issues, find some software that is using sparse matrices, like glmnet.
Since there are very many different zip codes, have a look at Principled way of collapsing categorical variables with many levels?  and the suggestions there.  I would try out the fused lasso.
It is also possible that for your application, some domain knowledge would be helpful for feature construction.


Performance of scrappping WEB table in VBA Excel

Question: This code is too slow. I need to transfer data from HTML table to excel worksheet. I can't use query table because the table is generated by POST method after pressing a button.
The table is 16 columns x 200+ rows
What can I do to make it work faster?
<code>'GET THE TABLE
Dim HTMLTables As MSHTML.IHTMLElementCollection
Dim HTMLTable As MSHTML.IHTMLElement
Dim HTMLDiv As MSHTML.IHTMLElement
Dim TableSection As MSHTML.IHTMLElement
Dim TableRow As MSHTML.IHTMLElement
Dim TableCell As MSHTML.IHTMLElement
Dim RowText As String
Dim row As Integer
Dim column As Integer

Dim ws As Worksheet
Set ws = Workbooks("Zautomatyzowany raport produkcyjny.xlsm").Worksheets("PressQualityHold")

row = 1
column = 1
'WAIT for IE
Do While IE.readyState <> READYSTATE_COMPLETE Or IE.Busy
Loop

Set HTMLTables = HTMLDoc.getElementsByTagName("table")

Workbooks("Zautomatyzowany raport produkcyjny.xlsm").Worksheets("PressQualityHold").Cells.Clear

For Each HTMLTable In HTMLTables
    'Debug.Print HTMLTable.ID, HTMLTable.className

    If HTMLTable.ID = "table_details" Then
        For Each TableSection In HTMLTable.Children
            'Debug.Print , TableSection.tagName

                If TableSection.tagName = "TBODY" Then
                    For Each TableRow In TableSection.Children
                        'RowText = ""
                        For Each TableCell In TableRow.Children
                            ws.Cells(row, column).Value = TableCell.innerText
                            'RowText = RowText & vbTab & TableCell.innerText
                            column = column + 1
                        Next TableCell
                        'Debug.Print , , RowText
                        row = row + 1
                        column = 1
                    Next TableRow
                End If

        Next TableSection
    End If

Next HTMLTable

IE.Quit
</code>
Comment: So I've found that one way to make it faster is to copy entire row from the HTML Table, but then it copies to one cell and later I'd have to parse the excel sheet in different way. I am not sure what would be faster.
Comment: Same happens when I try to copy entire table. Copies into 1 cell. Should I try doing that and then parsing the data? Is excel even able to hold this much data in one cell? Or maybe someone already knows easier method?
Answer: I am very sorry to spam the forum, but I have found a solution to this.
I am not sure how and why this works, but it does exactly what I wanted.
<code>'GET THE TABLE
Dim HTMLTables As MSHTML.IHTMLElementCollection
Dim HTMLTable As MSHTML.IHTMLElement
Dim HTMLDiv As MSHTML.IHTMLElement
Dim TableSection As MSHTML.HTMLTable
Dim TableRow As MSHTML.IHTMLElement
Dim TableCell As MSHTML.IHTMLElement
Dim RowText As String
Dim row As Integer
Dim column As Integer

Dim ws As Worksheet
Set ws = Workbooks("Zautomatyzowany raport produkcyjny.xlsm").Worksheets("PressQualityHold")

row = 1
column = 1
'WAIT for IE
Do While IE.readyState <> READYSTATE_COMPLETE Or IE.Busy
Loop

Set HTMLTables = HTMLDoc.getElementsByTagName("table")

Workbooks("Zautomatyzowany raport produkcyjny.xlsm").Worksheets("PressQualityHold").Cells.Clear

For Each HTMLTable In HTMLTables
    'Debug.Print HTMLTable.ID, HTMLTable.className

    If HTMLTable.ID = "table_details" Then
        For Each TableSection In HTMLTable.Children
            'Debug.Print , TableSection.tagName

                If TableSection.tagName = "TBODY" Then

                    Set TableSection = HTMLTables(2)
                    Dim clipboard As MSForms.DataObject
                    Set clipboard = New MSForms.DataObject
                    clipboard.SetText TableSection.outerHTML
                    clipboard.PutInClipboard
                    ws.Range("A1").PasteSpecial

                End If

        Next TableSection
    End If

Next HTMLTable

IE.Quit
</code>