What is entropy, really?

Question: I first saw the term "entropy" in a chemistry course while studying thermodynamics.
During my graduate studies I encountered the term in many different areas of mathematics.
Can anyone explain why this term is used and what it means.  What I am looking for is a few examples where the term "entropy" is used to describe some mathematical object/quantity and its meaning there.
Comment: Entropy isn't what it used to be.
Comment: Entropy measures our ignorance : It is the logarithm of the phase space volume of macroscopically indistinguishable states. See Roger Penrose, The road to reality, chapter 27.3 .
Comment: @MichaelHardy, von Neumann is supposed to be that someone (he was Shannon's advisor, iirc)
Comment: This could be an interesting place to start: http://ncatlab.org/johnbaez/show/Entropy+as+a+functor
Comment: I read somewhere (Therefore it's true!! Right?) that when Claude Shannon was doing his early work on information theory, he was at first uncertain what word to use to refer to the concept he later called "entropy", when that word had long been used for a different concept in physics (either a related concept or an unrelated one, perhaps depending on your philosophy).  Someone (a physicist, maybe?) told him he should call it "entropy" because nobody really knew what entropy is.
Comment: It's also worth recalling that the term has been introduced in English in 1868, from the German "Entropie", coined in 1865, on analogy of "Energie",  by German physicist Rudolph Clausius, from Greek ἐντροπία ("a turning toward").
Comment: @jjcale It measures our ignorance more because we don't know what it is than because it is the logarithm of indistinguishable something :-)
Comment: Lieb and Yngvason have a nontraditional and totally different approach, in which they define entropy in terms of a preorder on a set that has certain properties: http://arxiv.org/abs/math-ph/0204007 http://arxiv.org/abs/cond-mat/9708200
Comment: You might also look at the following book (parts of which are quite nice): http://press.princeton.edu/titles/7688.html .
Comment: @gfdsal : No, the number of coins are the number of degrees of freedom (= number of spins in the Ising model).
Comment: @jjcale but question is: why logarithm?
Comment: @gfdsal : Because the entropy should be additive in the number of degrees of freedom.
Comment: @jjcale allright. But what are you referring to "degrees of freedom"? So essentially say a coin has two sides, the degrees of freedom is 2?
Answer: Here is a simple story one can tell about the entropy
$$H = -\sum_{i=1}^n p_i \log p_i$$
of a discrete probability distribution. Suppose you wanted to describe how surprised you are upon learning that some event $E$ happened. Call your surprise upon learning that $E$ happened $s(E)$, the "surprisal." Here are some plausible conditions that $s$ could satisfy:

$s(E)$ is a decreasing function of the probability $\mathbb{P}(E)$. That is, the less likely something it is to happen, the more surprising it is that it ends up happening, and the likelihood of something happening is the only thing determining how surprising it is. For example, flipping $10$ heads in a row is more surprising than flipping $5$ heads in a row.
If $E_1$ and $E_2$ are independent, then $s(E_1 \cap E_2) = s(E_1) + s(E_2)$. That is, your surprise at learning that two independent events happened should be the sum of your surprises at learning that each individual event happened. For example, flipping $10$ coins heads in a row is twice as surprising as flipping $5$ coins heads in a row.

Exercise: These conditions imply that $s$ must be a positive scalar multiple of $- \log \mathbb{P}(E)$.
Then the expected surprisal is a positive scalar multiple of the entropy. Note in particular that $H$ is minimized if some $p_i = 1$, which corresponds to the same thing always happening and which is not surprising at all.
Comment: Nice explanation. So we're reduced to how plausible we find the conditions. The first one seems fine but the second one seems a little arbitrary -- why not weighted sum of powers, for example? I believe there are generalizations of Shannon entropy breaking this condition. So what's missing in this answer is an explanation of why to expect additivity. This is the place where I'd say it's better to put your statistical-mechanics hat on, for then the additivity of entropy as a disorder in the system seems obvious.
Comment: This is really nice, Qiaochu!
Comment: @PaulSiegel the exercise intends that $s$ is a function of $\mathbb{P}(E)$ rather than $E$. As you note, otherwise $s$ is not necessarily a multiple of $-\log \mathbb{P}(E)$.
Comment: Using the term "information content" rather than "surprise" may make additivity seem more natural (even though this doesn't quite line up with standard terminology).
Comment: @Qiaochu Yaun, I am wondering if your story originates from undergraduate books in probability, such as Ross.  He also explains why the second property is intuitive.
Comment: In general I see no guarantee that a probability space should provide enough pairs of independent events to constrain the possible "surprise" functions on it all that much.  Perhaps the right characterization involves looking at all finite probability spaces at once, though I'm not sure what that characterization would look like.
Comment: (My example above wasn't very good since $s(\{H\}) = s(\{T\})$ and $s$ is only well-defined up to a constant.  A better example would be to set $P(\{H\}) = 1/4$ and $P(\{T\}) = 3/4$ - we expect $s(\{H\}) = \log 4$ and $s(\{T\}) = \log(4/3)$, but I don't see what goes wrong if we choose $s(\{T\}) = \log(2)$, for instance, instead.)
Comment: @Timothy: Actually, "surprise" is more natural if instead of the disjoint additivity above, we actually look at "submodularity" --- the amount of possible surprise diminishes based on what we've already seen---seems to be "natural", and axiomatically related to entropy (see e.g., related discussion also in: http://arxiv.org/abs/1002.4020 -- and these ideas can be then related to Kolmogorov complexity too...
Comment: @Mustafa: no, a friend of mine told me once that "surprise is negative log probability" and this seemed like the obvious conclusion to reach from that claim.
Comment: I've encountered this characterization before, but there's something wrong.  Consider the fair coin probability space $\{H, T\}$ with $P(\{H\}) = P(\{T\}) = 1/2$.  Since  $\{H, T\}$ is independent of $\{H\}$ and $\{T\}$ we must have $s(\{H, T\}) = 0$, and since $\emptyset$ is independent of $\{H\}$ and $\{T\}$ we must have $s(\emptyset) = \infty$ (monotonicity is also used here).  But there are no other pairs of independent events, so we can't use the independence condition to fix the values of $s(\{H\})$ or $s(\{T\})$, and it seems to be that we can pick any positive value we want.
Comment: Lovely answer. (I just saw this question for the first time.)
Comment: @Qiaochu Yuan, thank you, I was just wondering because I really liked your explanation and I found a similar explanation in an undergraduate probability book (Ross).
Answer: From a modern point of view, the paradigmatic definition of entropy is that it is a number associated to a probability distribution over a finite sample space.  Let $N$ be the size of your sample space and let $p_1, p_2, \ldots, p_N$ be the probabilities of the events.  Then the entropy of the probability distribution is defined to be
$$H = \sum_{i=1}^N - p_i \log p_i,$$
where we take $0 \log 0$ to be $0$.
I've been purposely vague about the base of the logarithm; the choice of base is a matter of taste or convention.  In combinatorics or computer science or information theory, one often deals with strings of binary digits, and then it's convenient to take the base of the logarithm to be 2.  With this convention, it's an easy exercise to check that if $N=2^n$ and the probability distribution is uniform, then $H=n$; we can think of this as choosing an $n$-long binary string at random, and we often say that the entropy is "$n$ bits".  Similarly you can check that if $p_i=1$ for some $i$ (and $p_j=0$ for $j\ne i$) then $H=0$.  These two extreme cases give you some intuition for the idea that $H$ is a measure of "how random" your system is.  If your $n$-bit string is deterministic then there's no entropy, and if instead it's totally random then there are $n$ bits of entropy.  The closer your probability distribution is to uniform, the more entropy it has.
In physics, entropy was originally defined thermodynamically as $\int dQ/T$ where $Q$ is heat and $T$ is temperature.  This doesn't look at all like the "paradigmatic definition" I gave above, but the connection is that Boltzmann showed that from a statistical-mechanics point of view, the thermodynamic entropy can be recovered as $k \sum_i -p_i \ln p_i$ where the sum is over all states $i$ of your system, and $k$ is Boltzmann's constant.  (It's traditional to use the natural logarithm in stat mech.)
Entropy may also be thought of as a measure of information content.  This can sound confusing at first, since intuitively we tend to think of information as meaningful content, and if something is totally random then how can it carry any meaningful content?  An example that may be helpful is to consider compression algorithms.  Think of a text document (or photo, or video) as a sequence of bytes.  If you take a normal, uncompressed document and look at the frequency distribution of the bytes, you will see that it is far from uniform.  Some bytes are much more frequently occurring than others.  But if you then run the document through a good (lossless) compression algorithm, then the frequencies will be close to uniform.  In the compressed version, each byte contains more information about the document than in the uncompressed version.  That's how you're able to convey the same information in fewer bytes.  The increased information per byte is reflected in the greater entropy of the uniform distribution, compared to the original uncompressed probability distribution.
Comment: Due to Shannon's theorem, one can make this measure of information content precise: the entropy of a distribution is the expected amount of information you need to transmit in order to tell someone (who knows the distribution and your encoding scheme) a sample from the distribution. This is often a very useful perspective; in particular if you believe your distribution should be close to a product of independent uniform random variables, finding an efficient coding scheme and looking at entropy can often prove this (in a crude form that most of the distributions are close to uniform indept).
Comment: @gfdsal : As for your other question, no, I would not say that more information means less entropy. It's true that in everyday speech, we tend to think of entropy as "disorder" or "randomness," which is the opposite of meaningful content or "information." But that is not how the words are used in information theory. The uniform distribution has the highest entropy; strings of uniformly random bytes tend to have high information, because to tell you exactly what bytes I got, I have to communicate a lot of information to you. I usually can't say, "They're all 0" or something simple like that.
Comment: @gfdsal : If you're looking for an intuitive explanation, imagine that someone is sending you a black-and-white photo of the night sky, one pixel at a time, uncompressed. Most of the pixels are black, and a few are white (indicating stars or other celestial objects). Intuitively, the white pixels are giving you more useful information, right? It's the less probable values that are carrying the "interesting information." If you were to compress the image, you could compress long stretches of black pixels into just a few bits because they're not carrying much information.
Comment: @gfdsal : Flip your coin 1000 times, and each time you get tails, draw a black pixel, and each time you get heads, draw a white pixel. Does this help you see how to convert between the night sky and your coin?
Comment: +1 Kuddos for best intuitive explanation. I am slightly confused somehow, in a non-uniform distribution, why is low-probability events are regarded as carrying high information?  So if we have unbiased coin and chances for head is 0.2 and for tail is 0.8 then why do we say outcome head has more information? Also isnt entropy inverse of information meaning if we have more information then there is less entropy? $$-log_2(0.2)>-log_2(0.8)$$, $$2.32>0.32$$
Comment: ok so in general rare events require more bits to be encoded then more probable events. Right? Now weird thing is that I understood that in the night-sky example, celestial bodies are more useful and are sparse (low frequency) then empty black pixels. But I am unable to translate that intuition to the coin example of how heads is more informative with 0.2 probability :|
Comment: Omg yes! It makes so much sense now. Im glad I found your answer.
Answer: The explanation of entropy as information content in Shannon's original paper is quite good.  On page 10 of A Mathematical Theory of Communication (1948), you can find the following passage:

In the following pages, the author points out that such a function is uniquely given by $-\sum p_i \log p_i$ up to rescaling, names the function "entropy", mentions that the very same function appears as entropy in some existing treatments of statistical mechanics, and describes some of its useful properties.  There is a very nice discussion of information density of written language, e.g., the existence of crossword puzzles is an indication that English words have relatively low redundancy.
One example of a situation where entropy is considered is in the study of black hole thermodynamics.  In this case, one considers states given by a quantum-mechanical density matrix $\rho$, and the von Neumann entropy is defined as $-\text{Tr}(\rho \log \rho)$.  Diagonalizing the density matrix produces Shannon's entropy formula.  Beckenstein and Hawking showed that, under some simplifying assumptions, the entropy of a black hole is proportional to its surface area.  Since black holes reveal essentially no information to outsiders, one can say that the information content (in this case, a choice of internal state among all possible states that look the same from outside) depends only on the surface area.  Some people looking for a quantum theory of gravity interpret this as evidence for a holographic principle, i.e., the claim that in our universe, the information contained in any region is encoded somehow in the boundary of that region.
Comment: @Marek : The entropy of a black holes is not zero. Instead it is much greater than the entropy of the star that collapsed to the black hole.
Comment: @jjcale: in reality, the total entropy always increases, of course. But I was explicitly mentioning just classical GTR. There we have http://en.wikipedia.org/wiki/No-hair_theorem
Comment: The last paragraph seems pretty arbitrary. Entropy is omnipresent in physics, simply because of statistics of many particles. What can be surprising in the black hole case, and you forgot to mention, is that according to general theory of relativity, black holes have no degrees of freedom and their entropy should equal zero (black holes have no hair). Of course, this is simply because we did not take quantum effects into account -- microscopically black holes must still be built out of some elementary parts -> therefore they do have entropy (whether proportional to area or not).
Comment: @Marek My last paragraph was a response to the part of the question that asked: `What I am looking for is a few examples where the term "entropy" is used to describe some mathematical object/quantity and its meaning there.'  I agree that my treatment is far from thorough.
Answer: The physicist Edwin T. Jaynes wrote a lot about entropy and its uses in physics and probability theory:
http://bayes.wustl.edu/etj/node1.html some published works
http://bayes.wustl.edu/etj/node2.html some unpublished works
He insisted that the two are very much related.  In one of his works he says he had a conversation with Eugene Wigner in the 1950s in which he told Wigner that physical entropy is a measure of information and Wigner thought that was absurd, because the information one person possesses differs from that of another, whereas entropy can be measured with thermometers and calorimeters.  Jaynes says he didn't know at the time how to explain to Wigner why that was wrong, but figured it out much later.  As nearly as I understand it at this moment, Jaynes thought entropy measures the amount of information delivered by those thermometers and calorimeters.
Jaynes has a sort of cult following, many members of which are professors in the physical sciences who learned everything they know about probability from Jaynes, and they can sometimes come across a bit like religious fanatics.
OK, a short answer: The entropy in a discrete probability distribution is $\displaystyle\sum_k -p_k\log p_k$, where $\{p_k\}$ are the probabilities assigned to atoms in the probability space.  The base of the logarithm can be any number $>1$ (or maybe $<1$ in some cases?  I haven't thought about that.).  There's also relative entropy of one probablity measure $p$ with respect to another, $q$, given by $\displaystyle\sum_k p_k\log(p_k/q_k)$ and cross entropy given by $\displaystyle\sum_k -p_k\log q_k$.
Via Google, you might be able to find notes taken by scribes in Gian-Carlo Rota's probability course at MIT.  Somewhere among those, he says the resemblance between entropy of discrete probability distributions (as defined above) and entropy of continuous distributions (done similarly with integrals.) is only superficial. These observations are explored for finite discrete distributions in Section 1.4 of the book "Combinatorics The Rota Way" by Kung, Rota, and Yan. Deeper relationships are speculated upon in Rota's article "Twelve problems in probability no one likes to bring up" in the book Algebraic Combinatorics and Computer Science, Crapo and Senato eds. Those speculations are in the section entitled "Problem 4: entropy."
Comment: There are physical experiments which demonstrate that Jaynes was right. One involve mixing polymers in a particular sequence, so that they can only be unmixed given a secret key.

Another key insight of Jaynes is that entropy is subjective, it's not a measure of a system, it's a measure of the ignorance of an observer about the system.
Comment: @ArthurB : I wonder if calling it "subjective" misses Jaynes's point. Informatoin delivered by thermometers and calorimiters is not subjective. Being a measure of information, and thus of how much information is missing, is not the same as saying it's subjective.
Comment: Thank you Michael for your explanation.  I also found the historical comments interesting.
Comment: @ArthurB Do you have a reference for that? I would be very interested in that.
Answer: Entropy is the negative KL divergence of a probability distribution $p$ from a uniform distribution $\lambda$, $H(p) = -D_{KL}(p\parallel\lambda)$. Thus if you have an intuition for the KL divergence then you have an intuition for entropy.
To that end, suppose you draw a large sequence of measurements from a true distribution $p$ and want to see how likely it is that a model distribution $q$ would generate the same histogram counts. The KL divergence is the negative log average likelihood of observing the histogram you got supposing that it was generated by $q$ instead of $p$.
See,
https://arxiv.org/abs/1404.2000
In the entropy case $q$ is uniform, so the entropy measures how likely it is that draws from your distribution could have been generated from a uniform distribution.
Comment: And minus the Shannon entropy is the convex generator of the KL divergence, this latter seen as a particular case of Bregman divergence.
Comment: @Nick Alger What type of variables are you discussing? By type I mean discrete or something more general? I'm concerned that if this definition were permitted to describe any type of probability distribution then it might violate the intuitions about entropy described in the other answers.
Comment: That's a bit of a circular argument, since the KL divergence is defined that way, not the other way round.
Comment: Link is broken. I think this is the same document: https://arxiv.org/abs/1404.2000
Comment: @AaronBergman Thanks, I updated the post with the new link.
Answer: The possible definition of the entropy can be done through the enumeration of permutations.
Let's consider the set of $N$ balls, where $N_i$ is a total amount of the balls of the $i$-th colour. The total amount of unique permutations can be described by the Multinomial Coefficient: $W = {N! \over \prod{N_i!}}$.
All permutations can be enumerated and assigned with a number from 0 to (W - 1). Hence, the string of $\log_2(W)$ bits can be used to encode each permutation. Let's consider the amount of bits per item of the permutation: $S = {\log_2(W) \over N}$.
The more uniform set is (the more balls of same colour are there) - the smaller $S$ is, and vice-versa: the less uniform set is - the higher amount of bits per item of the permutation is needed.
Below provided the brief logic of transformation from this idea to the Shannon entropy.
Let's assume that amount of items is sufficiently large, so we can make use of the Stirling's approximation: $\ln(N!) = N \cdot \ln(N) - N + O(\ln N) \approx N \cdot \ln(N) - N$.
Hence, we can rewrite $S$ as:
$S = {1 \over N} \cdot \left( \log_2(N!) - \log_2(\prod N_i!) \right) \approx$
$ \approx k \cdot {1 \over N} \cdot  \left( N \cdot \ln(N) - N - \left(\sum N_i \cdot \ln(N_i) \right) + \sum N_i \right)$
where $k$ - is the transformation coefficient to the natural logarithms.
Taking into account, that $N = \sum N_i$ we can make the further transformations:
$S \approx k \cdot {1 \over N} \cdot \left( (\sum N_i) \cdot \ln(N) - \left(\sum{N_i \cdot \ln(N_i)}\right) \right) =$
$= - k \cdot {1 \over N} \cdot \sum N_i \cdot \ln {Ni \over N} =$
$= -\sum {N_i \over N} \cdot \log_2{Ni \over N}$
As far as the total amount of balls in a set is $N$ and the amount of balls of $i$-th colour is $N_i$ - the probability of selection of the ball with $i$-th colour is: $p_i = {N_i \over N}$
Hence, the entropy can be expressed as:
$S = -\sum p_i \cdot \log_2 p_i$
The more tidy derivation could also show that the Shannon entropy is an upper bound of the ${\log_2 W \over N}$, hence its value will be always slightly greater than the value of the latter.
Comment: That is essentially Wallis's derivation of the principle of maximum entropy, as recalled in (more) physical terms in Section 11.4, pp. 351-354 of Jaynes's book *Probability - The Logic of Science* and also in https://en.wikipedia.org/wiki/Principle_of_maximum_entropy#The_Wallis_derivation - a very useful remark you made is that the particular function $\frac{\log_2 W}{N}$ of $W$ is the bit word length per item of the permutation. In both references above, this choice seems arbitrary - any monotone increasing function of $W$ would do.
Comment: Similar stuff can also be seen in the textbook Thermal Physics by Kittel and Kroemer.
Answer: Entropy is a measure of disorder of a configuration. Its converse is information, which is a measure of order. Information theory seeks to understand the influences of different parts of a system on one another by comparing their entropies, or conversely by comparing their informations. One of the goals of information theory is to estimate the likelihood that one event caused another. Transfer entropy, and the closely related Granger Causality, is a tool for doing this.
First, relative entropy. Consider a pair of configutations $A$ and $B$, each with its own entropy. Assume that we know that $B$ is a controlled modification of $A$. Relative entropy, that is the entropy of $A$ minus the entropy of $B$, measures the amount of information added in the transition from $A$ to $B$.
Next, assume that both configurations, $A$ and $B$, are indexed by parameter which gives directionality (e.g. time). To measure the influence of $A$ on $B$, take the difference between the entropy of $B_t$ given its past (that is $B_{t-1}$) and the entropy of $B_t$ given both $B_{t-1}$ and $A_{t-1}$. That difference is the transfer entropy. So if the entropy of $B$ goes down "after the intervention of $A$", then we assume that $A$ causes $B$, and the magnitude of the transfer entropy estimates the likelihood of this causal relationship.
Of course correlation does not imply causation, and strictly interpreted Granger Causality suffers from the post hoc propter hoc logical fallacy. This does not prevent it from being useful.
Edit: Avishy Carmi comments by e-mail:

 There is a slight detail to add which ensures the relative entropy is always non-negative.
The relative entropy is not the mere difference of entropies of $A$ and $B$ but rather the difference between the cross entropy $H_{AB}$ and the entropy of $A$, namely $H_{AB} - H_A$. The notion of cross entropy roughly quantifies the amount of disorder in $B$ when described in the "language" or coding scheme of $A$. It is always greater than $H_A$  which, using the same analogy, can be viewed as the disorder in $A$ described in the language of $A$. That is, some information is lost due to incompatibility of a configuration ($B$) and the coding scheme used to describe it (say of $A$).
Comment: could you define a "configuration"?
Comment: @cantorhead A probability distribution.
Comment: Okay. Then are you just talking about discrete distributions or all probability distributions?
Answer: There are a bunch of answers already but none are really how I see it so here's another.
This value
$$\log \frac{1}{p} = - \log p $$
is best interpreted as the information gained when an event with probability $p$ happens, as measured in whatever units you like based on the base of the logarithm. Base 2 give bits, base $e$ gives nats, etc. An example: if a discrete event happens with probability $p = \frac{1}{8}$, then finding out that it occurred conveys (when optimally compressed) $\log_2 \frac{1}{p} = 3$ bits of information, because it takes 3 bits to describe "where in the distribution you are", a particular choice out of 8 values.
Entropy is the expected information gained per event from a particular distribution:
$$H[p] = \mathbb{E}[\log \frac{1}{p}]$$
For discrete entropy the expectation ends up looking like $H[p] = \sum - p \log p$. For continuous distributions the equivalent quantity is the differential entropy. The equivalent expectation looks like $H[p] = \int - p \log p \, dx$, but this is properly understood as only being the information gained relative to the uniform distribution $U(0, 1)$. For example, a uniform distribution $U(0, \frac{1}{2})$ has $-1$ bits of differential entropy because it can be described in one bit fewer information than $U(0, 1)$, while $U(0, 2)$ has $1$ bit of differential entropy because it requires one more bit to describe than $U(0, 1)$.
This property of entropy being "relative to another distribution" holds for discrete entropy also, and I think is necessary to properly interpret entropy in physics. In (classical) statistical mechanics entropy is the quantity of information in the microstate distribution: $H = \log \Omega$ where $\Omega$ is the "number of microstates in a macrostate". Hence it is a relative value: "if you change languages from a particular microstate parameterization to a macrostate parameterization, you have reduced the description of the system by $\log \Omega$ bits". But, and this often isn't emphasized in physics: there's no reason you couldn't do this again and again, with, say, an even finer-specified "nanostate" that has more degrees of freedom than the microstates, or some intermediate states between the micro- and macro- ones.
So IMO entropy is properly understood as a quality of a particular choice of language used to describe a system, relative to another choice of language, rather than of the system itself. In particular it is the expected amount of information communicated by describing a system in a particular language, compared to another one. And in general a system has no "canonical" list of states that you can describe it as having and the choice of a description language is arbitrary, in the same way that in general a physical problem has no canonical coordinate system.
Answer: In dynamical systems, there are two major entropies used, topological and metric. They each quantify, in technically different ways, how complexity of orbits grows as their length is allowed to grow into infinity. To make things slightly more precise, topological entropy measures the number of orbits that can be distinguished if we measured the position of points up to a fixed resolutions, and then let the resolution get finer and finer, and orbit length get longer and longer. A fairly good, not-so-technical introduction to both topological and metric entropies was given by Lai Sang-Young in Entropy in dynamical systems, available here: http://cims.nyu.edu/~lsy/papers/entropy.pdf
Answer: If you want to forgett about physical implications / motivations for a second, then you could consider the fact that the Shannon entropy $f(x):=x\ln \frac{1}{x}+(1-x)\ln\frac{1}{1-x}$
  is the unique continuous solution of the fundamental equation of information (FEI)
$$f(x)+(1-x)f\left(\frac{y}{1-x}\right)=f(y)+(1-y)f\left(\frac{x}{1-y}\right) ~~(*)$$
on  $D:=\{(x,y)~|~ x\in [0,1), y\in[0,1), x+y\leq 1\}$. This latter is motivated by quite natural axioms that certain functionals-called information-on a given probability space must satisfy.  In other words, the entropy can be defined as the (unique) solution of a functional equation much like the dilogarithm and many other important functions in Mathematics. It can be also seen as the convex generator (up to a sign) of the KL divergence, which plays an important role in information geometry and machine learning.
Comment: Does this approach extend to continuous, mixture, or singular variables? I'm interpreting the example shown as for the discrete two-state variable  with probabilities $\{x,1-x\}$.
Answer: There are a number of definitions of entropy floating around that are analogous even if they aren't equivalent.  For any outsider they should look and feel nearly the same, for an expert you may be incline to choose one over the other.

information entropy
statistical entropy
topological entropy

Exciting new variants in theoretical physics

quantum entropy
renyi entropy
holographic entanglement entropy [1]
information cohomology [2]

Please excuse my over-simplified treatment.  Also people are still thinking about what entropy to this day.  Please also In Search of a Structure, Part I: On Entropy (Gromov, 2012)

Boltzmann defined "entropy" as the number of microstates of a thermodynamic ensemble.  And his example is the asymptotics of the multinomial distribution.
A If $m = p\, n$ the binomial coefficients grow as the entropy:
$$ \binom{n}{m} = e^{p \log p + (1-p) \log (1-p)} = \mathrm{exp}\, h(X) $$
where $X$ is the random variable which is $X=1$ with probability $p$ and $X=0$ with probability $1-p$.
B What about just the map $T: x \mapsto 2x \mod 1$ on $[0,1]$. How many does a typical point $x$ have?
$$ 2^n x \equiv a \mod 1 $$
This has $2^n$ solutions, so the entropy is $\frac{1}{n}\log |T^{-n}(x)| = \log 2$.

These are the two basic kinds of entropy (probabilistic, topological) and other definitions clarify this.
See also: Information Theory and Statistical Mechanics (Jaynes)
Comment: *Boltzmann defined "entropy" as the number of microstates of a thermodynamic ensemble.* Don't you mean the log of that quantity?
Answer: Shannon showed that entropy is an natural and useful quantity in information theory. To learn more about this, I recommend the book Elements of Information Theory by Cover and Thomas.
Comment: And von Neumann's explanation to Shannon for why he (Shannon) should call his new information-theoretic quantity entropy is amusing.  Taken from http://en.wikipedia.org/wiki/History_of_entropy: "You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."
Comment: Actually, Shannon's original paper is quite a masterpiece of exposition.
Comment: This is not an answer, would have been better as a comment.
Comment: I remember reading that awesome quote from von Neumann!
Comment: actually von Neumann's entropy came before Shannon's but ... :-)

What is entropy, really?


What is the meaning of Scratchbox2?

Question: Somewhere i heard the term 'Scratchbox2'. Why this is used? what is the exact meaning of this?
What is the advantages over Scratchbox 1?
Answer: Scratchbox is cross-compilation suite.
It looks like Scratchbox 2 (sbox2) is new version of Scratchbox 1, maintained by Nokia for Maemo development. You can use it for general cross-compilation too. It is a fork of Scratchbox.
Main advantage of Scratchbox 2 is a more elegant approach on some corner cases.
It also performs better, as it uses native tools whenever possible, and otherwise emulation. For Scratchbox 2 you don't have to modify any host tools. Scratchbox 1 may require some changes and recompilations.

What is the meaning of Scratchbox2?


chmod'ing a directory with millions of files taking forever

Question: I have a directory with millions of files and sub directories and I need to change the permissions on the entire directory and its files and its take many hours to do this.   I was wondering if there is a quicker way of doing this than the normal chmod -R command?
Answer: Your chmod -R is quite optimal. There have to be one chmod() syscall per file and filesystem has to update it's metadata on block device.
Disable barriers (mount option on XFS, Ext4 etc.) on this FS if enabled but remember to enable them later.
I hope it's one time action and not recurring problem? If easy management of file permissions is more important than performance and simplicity you could try to write some overlay FUSE filesystem mangling permissions and uid/gid on-the-fly.
Answer: You'd want to write a script and execute from the root directory. I think a basic one would go something like this;
<code>chmod 777 ./stuff/stuff
chmod 777 ./stuff/stuff

chmod 666 ./stuff/stuff
chmod 666 ./stuff/stuff
</code>
Etc, etc, etc...
There might be a better script or a better way. I'm not very good at scripting.
Comment: I'm basically doing chmod -R 777 /dir/
Comment: writing a script to chmod files one at a time is guaranteed to be slower


how to run metro webservice on jetty in Maven project?

Question: how to run metro webservice on jetty in Maven project??
Answer: As per this source.
If you place your web service in a WAR file, like here, it should run fine
on Jetty, just as it would on any servlet container. There might be some
subsequent Jetty-specific configuration needed, but I would try to deploy
the WAR first on Jetty and see where it gets you.
If you want to embed Jetty, instead of running on Jetty standalone, this
might give you some pointers.
Comment: We run Metro 1.4 in an embedded Jetty 6 here under Java 5.  Works fine.  Note:  The Metro jars goes in the jetty lib, not the war.


What's the rationale for vetoing anything with a cross or a fish in our logo?

Question: In chat, I surmised that crosses and fish were no-gos as far as Jin's artwork. I can sort of see why the cross might be offensive to JW's, but why the fish?
Answer: I'm of the mind that we should avoid both of these symbols not because they don't mean really important things, or have a lot of history, but because they are really cliché. I also have another separate issue with each.

The Cross: this is problematic because it's not universal. While it is widely accepted that Christ died on a cross and this is an appropriate symbol for most of Christianity, as you mention the Jehovah's witnesses are rather offended by it. This is a pretty good reason to leave it out.
The Fish: this is a personal one to me and I find it unfortunate because it's a good symbol with a lot of history. However, this symbol seems to be the chosen symbol for an over commercialized form of Christianity. It's the Christian Bookstore and bumper sticker version of it.

Lastly, most of the discussion in chat has been about the sort of logo or main icon we want associated with this site. I think that the fish (and maybe even the cross) would make excellent motiffs in the theme as sort of understated design elements. However I'd rather see us go with something significantly less cliché for a logo/header design.
Comment: Also I won't say that http://chat.stackexchange.com/transcript/message/6182797#6182797 wasn't at least a tiny bit written to attract a chat star :)
Comment: How about the Chi Rho?  I'm glad you're dumping the fish.  Especially the one that's popular now, it's a [Vesica Piscis](http://en.wikipedia.org/wiki/Vesica_piscis).  Sacred geometry sure is fun.
Comment: @DanAndrews I'm a huge fan of the Chi-Rho, but understand that it might not be ubiquitous enough to easily ID
Comment: The cross is Constantine's sword...  Forgot to through that softball in there too.
Answer: One reasons we should have obvious Christian symbols is because we're not beliefnet or Patheos or other sites that I don't frequent. And we're not about.com and incapable of ever having a uniquely styled site.
There's no real reason for our site to be trendy or fashionable or to attract hip kids to services and there's really no reason to avoid the cross we should be embracing.
The real question is, should every Christian thing have a cross on it. Now, I believe that every altar should have a cross on it and every station of the cross should have a cross on it, every baptized person should have the sign of the cross made on them and every prayer should begin with the cross and every morning should begin with a veneration of the cross and every rosary should end with a little smooch on the cross.

The only thing I can say further is, if this were Catholicism.SE, the design would be a lot more fun

All the badges would be sacred heart badges
the star would be Our Lady and she would have a crown if stars lit up when clicked
+1 would be a lamb and -1 would be a goat
the footer of the page would be that angel holing up the moon under the image of Our Lady of Guadalupe
The 404 page would be a lost sheep.
the captcha page would be charlton Heston (in some capacity)
the error page would be a confessional with a big red light on.
the icon would be St John Bosco's beanie. (The patron of catechists).

But somewhere, there would have to be a cross, or better yet, a crucifix.
Comment: Your Catholicism.SE design reminds me of the decor of [Buca di Beppo](http://www.bucadibeppo.com/about-buca/dining-experience/).  The accepted answer icon should be a candle with Saint Thomas Aquinas (or Saint Albertus Magnus) emblazoned upon it.
Comment: If this was ever built I'd frequent it just for the decor.
Answer:
Jehovah's Witnesses don't believe Jesus died on a cross; Mormons do believe this, but don't like to focus on it in their artwork (that said, they wouldn't find it offensive); and Catholics wouldn't object to an empty cross, but do think it would make the site look rather Protestant.
The fish symbol is, I know, ancient, and is theoretically universal, but I've never actually seen it used in Ireland, even in the hyper-religious Northern Ireland. I would, in fact, think of it as an American symbol. As it's a symbol that barely exists in my culture, I'm a little reluctant to analyse it, but I'll say that it seems to me more tribal than religious in nature. Wax eagle has strengthened that feeling by saying he'd associate it with Christian commercialism, which is of course tribal by necessity. I personally would associate the symbol with what I think of as the "nasty Calvinist" types ("We're going to heaven and you're not, ha ha ha!" rubs hands gleefully and cackles a little).

Personal associations with the symbol be what they may, I can report that I strongly suspect that many Irish people would have no idea what the fish symbol meant, and wouldn't associate it with Christianity at all. And that probably holds true in many majority Catholic countries, and I suspect also in Britain.
Comment: Anyone who is gleeful over the destruction of another has NOT understood the Calvisist doctrines of grace and has no right to use that name. One of the basic premises is that none of us deserve salvation in the first place.
Comment: I decided to ask a question about the "rubs hands gleefully and cackles a little" phrase and I might have gotten [a little carried away](http://christianity.stackexchange.com/q/9866/914).  Please let me know if you'd like me to remove the quotation or fix up the way I worded the question.  It's actually one that I've struggled with at times myself.
Comment: Yeah, as far as the Ichthus fish symbol is concerned, I've really never seen it in any church decorations or priestly apparel, I think you're right, any American would know what it is but perhaps we overstate its historical significance.  [Quo Vadis](http://en.wikipedia.org/wiki/Quo_Vadis_%281951_film%29) might have had something to do with that.


When was the word "terrorism" first used?

Question: When was the word "terrorism" first used in the world? I did some research but, the internet gave me two answers. The first one said it was first used in 1794. But, the second one said it was first introduced in 1160. I want to know when was it first used in the world. Does anyone know when the word "terrorism" was first used?
Comment: @Wordster This isn't about terrorism, it's about *the word* "terrorism". You don't need a set definition in order to say when the word was first used. In any case, the English language wasn't around in the 1st-century AD.
Comment: @Wordster The fact that this is posted on ELU implies that the OP is looking for the first use of "terrorism" in English or its antecedent languages.
Comment: Part of the problem is there's no set definition of it (as per wiki).
Comment: Also, the 1160 reference was for "terrible," not "terrorism."
Comment: Good point as for practice vs.  mere term.  But the English lang. objection doesn't hold, since twice she says "in the world."  Also, its first usage re the Jacobins:  that was in FRENCH, but was it also at that time referred to as "terrorism" in English?
Comment: "Depending on how broadly the term is defined, the roots and practice of terrorism can be traced at least to the 1st-century AD Sicarii Zealots, though some dispute whether the group, which assassinated collaborators with Roman rule in the province of Judea, was in fact terrorist."  (wiki)
Answer: According to the OED, it was first used in 1795 or 1796, depending on what definition you look at.
The 1795 citation refers to the French Reign of Terror:

During the reign of terrorism, I was a close prisoner for eight long months.
Speech in Convention

The year 1796 is when it was first used in its general sense:

John Thelwall..pointed out the defects of all the ancient governments of Greece, Rome, Old France, &c.; and the causes of rebellion, insurrection, regeneration of governments, terrorism, massacres, or revolutionary murders.
The pursuits of literature
Comment: I find it interesting that "terrorism" was basically formed the same way as capitalism or fascism -- a style of governance defined by, respectively, terror, the importance of capital, and the same (forced) unity as a bundle of sticks (a fascis). Of course it means something different today -- terrorists generally aren't a recognized state government -- but the etymology is interesting.
Comment: These information had helped me a lot
Comment: Since the question was "When was the word "terrorism" first used in the world?", one could accept the French form "terrorisme" and prepone the date by one year to January 1794 ([Source](http://www.cnrtl.fr/etymologie/terrorisme/)).
Comment: @Frank The question would then be off-topic on ELU.
Comment: OED doesn't claim to always list the earliest use; 'earliest confirmed use in written form' is what they _do_ claim, if memory serves me right.
Comment: @EdwinAshworth Earliest…in English. My reading level in French is minimal and I don't have the resources to track words back outside English.
Answer: The concept goes back to the origin of states thousands of years ago. The specific word 'terrorism' instead of 'causing fear as a weapon of war at the state level' comes from the Jacobin period 'The Terror' of the 1790's, where the French 'terreur' (panic, fear) became the term 'terrorisme' (a state of panic/fear that references this period of time).
Comment: This is a really solid line of thought. Would be an even better answer if you could cite some authorities or reference works.
Comment: Citations added per @DanBron 's suggestion.
Comment: But she already mentions 1794 in her question:  she evidently thinks (?) that's not satisfactory.
Comment: So the first time the word "terrorism was made was in 1790
Answer: I would add (from the usual suspect etymonline) that the first usage from 1795 meant specifically governmental terror,

terrorism (n.)
1795, in specific sense of "government intimidation during the Reign
  of Terror in France" (March 1793-July 1794), from French terrorisme,
  noted in English by 1795 as a coinage of the Revolution, from Latin
  terror "great fear, dread, alarm, panic; object of fear, cause of
  alarm; terrible news," from PIE root *tres- "to tremble" (see
  terrible).

while the general usage came from the Irish uprising in 1798.

General sense of "systematic use of terror as a policy" is first
  recorded in English 1798 (in reference to the Irish Rebellion of that
  year).
Comment: The first use _their researchers have found_.
Answer: 'Terrorism' in 1795–1796
The vast majority of instances of terrorism that Google Books searches turn up in works published in 1795 relate to the French Reign of Terror. I didn't find any matches for terrorism or terrorist[s] from before 1795.
For example, from "On the Constitution," a speech delivered by Thomas Paine in the [French] Convention, dated July 7, 1795, reprinted in Dissertation on First-Principles of Government (1795):

"In England I was proscribed for having vindicated the French revolution, and I have suffered a rigorous imprisonment in France, for having pursued a similar mode of conduct. During the reign of terrorism, I was a close prisoner for eight long months, and remained ſo above three months after the æra of the 10th. Thermidor. I ought, however, to state, that I was not prosecuted by the People, either of England or of France. The proceedings in both countries were the effects of the despotism existing in their respective governments. But even if my persecution had originated in the people at large, my principles and conduct would still have remained the same. Principles which are influenced and subject to controul of tyranny, have not their foundation in the heart.

From Helen Williams, "Letter II" (undated) in Letters Containing a Sketch of the Politics of France, From the Thirty-first of May 1793, till the Twenty-eighth of July 1794, and of the Scenes Which Have Passe in the Prisons of Paris, volume 2 (1795):

Among the crowds who were led to the guillotine, two persons only displayed strong marks of dismay and terror. One of these persons was Madame du Barry, the mistress of Lewis XV. She had been induced to leave England, where she passed some time after the revolution, and return to France, in order to secure her property; and soon after the 31st of May was led from her beautiful pavilion at Lucienne, to a prison in Paris, by one of the agents of terrorism, who, I am sorry to add, was an Englishman. The prisons, to use a French mode of expression, in a short time became the anti- chambers of the scaffold; and Madame du Barry's mind was impressed strongly with a presage of her fate.

An an early instance in which terrorism refers more generally to a repressive system of governance, rather than specifically to the French Reign of terror occurs in Robert Adair, A Whig's Apology for His Consistency: In a Letter from a Member of Parliament to His Friend in the Borough of * * * * (1795):

If, when kept at a distance by our fears, and struggling with all the difficulties of her [France's] revolution, our danger from her opinions was such as to justify whatever our great men have said and done to repress them, what hope will be left us when France, cloathed in the double fascination of novelty and victory, comes to demand an intercourse with you on terms of conciliation and commerce?Is it just then that these opinions will lose their danger, and that our constitution may be left to take care of itself ? or is it upon a peace that we are to enforce the system of Tory terrorism with renovated vigour, and carry into yet wider effect the set of measures which it has originated?

Similarly, from Robert Bisset, Sketch of Democracy (1796):

From the cessation of limited monarchy and the substitution of democracy in its stead, Greece [in the seventh century BC] had become a scene of licentiousness and wickedness. Athens was a particularly notorious. It was entirely in that state of anarchy, which follows the destruction of an old government, before there is virtue or ability enough to frame a new one of permanent force.
Under pretence of restraining this anarchy and licentiousness, Draco established his system of terrorism. His laws and government therefore only increased the evils. The people were in the greatest confusion and misery.

'Terrorist' and 'Terrorists' in 1795
Occurrences in 1795 of the related nouns terrorist and terrorists are likewise primarily associated with the French political scene. Indeed, some early instances of terrorists specifically associate the term with Robespierre's factional supporters. For instance, from Watkin Tench, "Letter V" (dated February 5, 1795, and written from La Normandie, a prison ship), in Letters Written in France to a friend in London, Between the Month of November 1794 and the Month of May 1795 (1796):

Through these channels I draw not only abundant matter for reflection, but frequently obtain diversion. "Moderation, and down with the Terrorists!" resound, I believe, from one end to the other of the republic. It is in all respects our interest to wish that such sentiments may be more than nominal. It is certain that a general dismission of the creatures of Robespierre is taking place.

And from a review of D'Ivernois, The French Revolution at Geneva in The Monthly Review; or Literary Journal (August 1795):

Some of the revolutionists, who had more humanity than others, finding that there was a design on foot to break into the prisons, and to murder all the prisoners confined since the revolution, applied to Gase [a parson belonging to the church of Geneva] as one of the new revolutionary syndics, and desired that he would adopt measures for defeating so shocking a design, and for protecting the lives of the prisoners. The answer of the unfeeling monster was, "I had rather that three or four hundred aristocrats should perish, than that a single patriot should receive so much as a scratch."—Soulavie, the French minister sent to Geneva, and under whose auspices it would seem that all the atrocities attending the revolution in that city were perpetrated, is now a prisoner in France, where he remains still untried, under a charge of being one of Robespierre's terrorists.

One interesting early occurrence of terrorist associates the term with adherence to the views of Thomas Paine. From a review of The Rights of the Nation and the Wrongs of the Prince in The Monthly Review (July 1795):

Again—'He,' meaning the letter-writer, 'declaims with glowing and graceful innuendos on a mixed motive and double principle. He is against king, prince, and constitution, because a terrorist and a Painite ; he is against the prince and Mr. Fox, &c. because a Pittite. But whether he is only a pretended Painite, and a real Pittite, and a new species of alarmist, may seem a question to some, if not doubtful to all : because decidedly a ministerial instrument, men consider this a work of duplicity.' Again,—'It is not merely the debts of the Prince ; the reputation of royalty and monarchy are affected. It is not private honor, private justice, private credit, or a principle of private honesty between man and man that are at stake, it is become a public question to which monarchy turns, and turning, this terrorist would make "tremble too."

However, one unusual instance from 1795 the term "pulpit terrorist" in a wholly different sense—the sense of "religious sermon giver who attempts to inculcate terror in his listeners." From Peter Pindar, "The Sorrows of Sunday: An Elegy," in The Royal Tour, and Weymouth Amusements: A Solemn and Reprimanding Epistle to the Laureat (1795):

Life with the down of cygnets may be clad!
Ah! why not make her path a pleasant track?
"No!" cries the Pulpit Terrorist, (how mad!)
"No! let the world be one huge hedgehog's back."

An early summing up of 'terrorist' as pejorative term
Francois Pages, Secret History of the French Revolution, from the Convocation of the Notables in 1787 to the First of November 1796, volume 2 (1797) offers a thoughtful commentary on the evolution of terrorist as used in France in the course of the French Revolution:

Such are the horrors which have sullied our revolution, that in proportion as it has created crimes, it becomes necessary to invent words to express them. After the revolution of the 9th Thermidor (July 27) of which we have just given the details, there was excited a re-action, the more bloody in proportion as oppression had been the stronger. Those who had concurred in the tyranny of Robespierre were denominated Terrorists, Robespierrists, and Maratists ; those pure patriots who had pronounced their sentiments with any degree of energy, were also frequently confounded under these general denominations. The re-actors in their turns were called new terrorists or furorists. Thus in the course of revolution what is a virtue at one time becomes a crime at another ; and these terms crime and virtue, have no fixed or determinate sense.

This, at any rate, seems to remain constant over the years: one person's terrorist is another person's freedom fighter—and partisans on either side of a conflict are quick to apply the more attractive term to their friends and the more repellent term to their foes.


Combine two columns and add into one new column

Question: In PostgreSQL, I want to use an SQL statement to combine two columns and create a new column from them.
I'm thinking about using concat(...), but is there a better way?
What's the best way to do this?
Answer: Generally, I agree with @kgrittn's advice. Go for it.
But to address your basic question about concat(): it is useful if you need to deal with null values - and null has neither been ruled out in your question nor in the one you refer to.
If you can rule out null values, the good old (SQL standard) concatenation operator || is still the best choice, and @luis' answer is just fine:
SELECT col_a || col_b;

If either of your columns can be null, the result would be null in that case. You could defend with COALESCE:
SELECT COALESCE(col_a, '') || COALESCE(col_b, '');

But that gets tedious quickly with more arguments. That's where concat() comes in, which never returns null, not even if all arguments are null. The manual:

NULL arguments are ignored.

SELECT concat(col_a, col_b);

The remaining corner case for both alternatives is where all input columns are null in which case we still get an empty string ''. To get null instead:
SELECT CASE
          WHEN col_a IS NULL THEN col_b
          WHEN col_b IS NULL THEN col_a
          ELSE col_a || col_b
       END;

This gets more complex with more columns quickly. Again, use concat(), but add a check for the special condition:
SELECT CASE WHEN (col_a, col_b) IS NULL THEN NULL
            ELSE concat(col_a, col_b) END;

How does this work?
(col_a, col_b) is shorthand for ROW (col_a, col_b). And a row type is only null if all columns are null. Detailed explanation:

NOT NULL constraint over a set of columns

Also, use concat_ws() to add separators between elements (ws for "with separator").

An expression like the one in Kevin's answer:
SELECT $1.zipcode || ' - ' || $1.city || ', ' || $1.state;

is tedious to prepare for null values in PostgreSQL 8.3 (without concat()). One way (of many):
SELECT COALESCE(
         CASE
            WHEN $1.zipcode IS NULL THEN $1.city
            WHEN $1.city    IS NULL THEN $1.zipcode
            ELSE $1.zipcode || ' - ' || $1.city
         END, '')
       || COALESCE(', ' || $1.state, '');

Function volatility is only STABLE
concat() and concat_ws() are STABLE functions, not IMMUTABLE because they can invoke datatype output functions (like timestamptz_out) that depend on locale settings.
Explanation by Tom Lane.
This prohibits their direct use in index expressions. If you know that the result is actually immutable in your case, you can work around this with an IMMUTABLE function wrapper. Example here:

Does PostgreSQL support "accent insensitive" collations?
Comment: Nice! I had not picked up on the different `NULL`-handling of the concat() function versus the SQL-standard `||` concatenation operator. (You notice I dodged the issue in my answer by using `NOT NULL` columns.) I'm now armed with one more tool in my toolbox.
Comment: I hope I could choose multiple accepted answers. Thanks for the full explanation. All answers are appreciated!
Comment: wow, very nice explanation!
extra thanks to ROW (col_a, col_b), I've never heard of it, and it is very handy for other uses too
Answer: You don't need to store the column to reference it that way.  Try this:
To set up:
CREATE TABLE tbl
  (zipcode text NOT NULL, city text NOT NULL, state text NOT NULL);
INSERT INTO tbl VALUES ('10954', 'Nanuet', 'NY');

We can see we have "the right stuff":
\pset border 2
SELECT * FROM tbl;

+---------+--------+-------+
| zipcode |  city  | state |
+---------+--------+-------+
| 10954   | Nanuet | NY    |
+---------+--------+-------+

Now add a function with the desired "column name" which takes the record type of the table as its only parameter:
CREATE FUNCTION combined(rec tbl)
  RETURNS text
  LANGUAGE SQL
AS $$
  SELECT $1.zipcode || ' - ' || $1.city || ', ' || $1.state;
$$;

This creates a function which can be used as if it were a column of the table, as long as the table name or alias is specified, like this:
SELECT *, tbl.combined FROM tbl;

Which displays like this:

+---------+--------+-------+--------------------+
| zipcode |  city  | state |      combined      |
+---------+--------+-------+--------------------+
| 10954   | Nanuet | NY    | 10954 - Nanuet, NY |
+---------+--------+-------+--------------------+

This works because PostgreSQL checks first for an actual column, but if one is not found, and the identifier is qualified with a relation name or alias, it looks for a function like the above, and runs it with the row as its argument, returning the result as if it were a column.  You can even index on such a "generated column" if you want to do so.
Because you're not using extra space in each row for the duplicated data, or firing triggers on all inserts and updates, this can often be faster than the alternatives.
Comment: "`which takes the record type of the table as its only parameter`" : how do we find that after creating a table?
Comment: The record type for the table will generally have the same name as the table.  See the example -- specifically that there is `CREATE TABLE tbl` and `CREATE FUNCTION combined(rec tbl)`.  For the definitive answer, find the reltype column of the pg_class row for the table, and use that to find the matching pg_type row.
Answer: Did you check the string concatenation function? Something like:
update table_c set column_a = column_b || column_c

should work. More here


How Page Tables are stored in the main memory?

Question: i know that page tables are stored in memory , and each process has its own table , but each table has entries as the number of virtual pages in virtual memory so how can every process has a table and each table resides in main memory besides , the number of entries in each table is larger than the number of physical pages in main memory ...can someone explain that to me i'm very confused ,
Thanks in Advance.
Comment: I've seen this more than 1 time and I haven't found what im looking for...
Answer: Typically, page tables are said to be stored in the kernel-owned physical memory. However page tables can get awfully big since each process have their own page tables (unless the OS uses inverted paging scheme). For even a 32 bit address space with a typical 4KB page size, we shall require a 20 Bit virtual page number and a 12 bit offset. A 20 bit VPN(Virtual Page Number) implies that there would be 2^20 translations. Even if each translation i.e the Page Table entry requires 4 Bytes of memory, it amounts to 4x(2^20)= 4MB of memory, all just of address translations, which is awful.
Hence modern OSes place such large page tables in virtual kernel memory which is the Hard Disk, and swaps them to the physical memory whenever required. Thus page table is virtualized the same way each page is virtualized.
I would suggest you to go through this wonderful and easy book to get a clear under standing of Memory Virtualization and Paging related concepts:
http://pages.cs.wisc.edu/~remzi/OSTEP.
Comment: Looks decent that book... Reading now~
Comment: The pertaining sections in the book are 20.5 and 23.  They relate to VAX/VMS.  It is unclear if this applies to Linux.
Comment: However, some basic/key page tables must always stay resident in physical memory, in order to prevent circular page table fault.


What is the atmospheric pressure at the deepest point of the Kidd mine?

Question: At 2735 m (8973 ft) below sea level, the Kidd mine reportedly has the deepest non-marine point of the Earth's surface (since the Kola borehole is sealed). I wonder what the pressure down there is, as compared to 14.696 psi (1013.25 hPa) at sea level and 15.45 psi (1065 hPa) on the shore of the Dead Sea (413 m = 1355 ft below sea level standard).
Comment: [link](https://en.wikipedia.org/wiki/Mponeng_Gold_Mine)
Comment: @DavidGarcíaBodego Your link writes that it's 4 km (2.5 mi) below ground level. The Kidd mine has the deepest point below _sea_ level.
Comment: @Gionanni, Mine is 600 meters above sea level and depth is 4000 meters, so it is 3400 meters below sea level. I think that this is more than 2735 meters b.s.l.
Answer: From the book Environmental Engineering in South African Mines, The Mine Ventilation Society of South Africa, 1989, pp 451-455.
The equation required is:
$P \ = \ P_{atm}\cdot e^{\frac{-gMh}{1000R\cdot (273.15+t_{dry})}}$
Where:

$P_{atm}$ = Atmospheric air pressure
$t_{dry}$ = Dry bulb temperature
$g$ = acceleration due to gravity, 9.8 m/s2
$M$ = Molar mass of air on Earth, 28.9644 g/mol
$h$ = The difference in elevation, -2735 m
$R$ = The universal gas constant, 8.3144622 J/mol$\cdot$K

Current data is being taken from the Canadian government weather service website. The nearest town to Kidd Creek is Timmins, Ontario, approximately 25 km south.
For 14 December 2022, the dry bulb temperature on the surface is -12.6 °C and the sea level adjusted atmospheric pressure is 1032 hPa (103 200 Pa). Given the depth of the mine and heating of the air due to both auto compression, the geothermal gradient and any diesel powered equipment in the mine, the temperature of the air at the bottom of the exhaust ventilation shaft will most likely be around 30 °C.
The atmospheric pressure at a depth of 2735 m is,
$P \ = \ 103\ 200\cdot e^{\frac{-9.8(28.9644)(-2735)}{1000(8.31446) (273.15+30)}}$ = $140\ 400\ Pa = 1404\ hPa = 20.37 \ psi$


where does xenserver keep vm snapshots?

Question: I'm ssh'd into the xenserver and trying to locate the snapshots using "find /scripts | grep -i string" where string is what I called the snapshot when I took it in xencenter. This method doesn't work, so I want to know where xenserver keeps the snapshots I make?
Answer: I finally got an answer from the citrix forum:
"There are no snapshot files. Virtual disks (including snapshots) in an LVM SR are stored as volumes. You can manage volumes with commands in the lvm2 package (run rpm -vV lvm2 to see the included commands)."
Here is the full thread.
Answer: I know this is late to the game, but there is an option to use an Ext3 filesystem based Storage Repository when installing XenServer.  If this is done then the VM disks are actual VHD files stored in /var/run/sr-mount/<UUID OF SR>.
This also allows for thin provisioning, something supported in Ext3, NFS, and other more enterprise level SR types, but not supported with the default LVM SR type.
During the install, when choosing the disk(s) to use for virtual machine storage, make sure to select "Enable thin provisioning (Optimized storage for XenDesktop)".  Even if you aren't going to be using XenDesktop this still is the option that enables the Ext3-based SR versus the default LVM.
For more details, see:
Citrix XenServer 6.1.0 Installation Guide
Citrix XenServer 6.1.0 Administrator's Guide
Search for "ext3" and "thin provisioning" in those documents.
For someone who might want to attempt switching an existing install from an LVM SR to an Ext3 SR, look under the "Converting an Existing Host to Use Thin Provisioning" section in the Installation Guide, though be fully aware that this will DESTROY any existing data and VMs in the LVM SR.  So you need to back up or migrate any VMs out of the existing SR before making the switch.
In case Citrix changes the links for the documents, or when new XenServer versions become available, just search Google for "Xenserver Installation Guide" or "Xenserver Administrator's Guide" and it should be pretty easy to find.
Comment: Interesting info. Do you have a link showing this option? Would help make a great answer.
Comment: @Lizz I've updated the answer with more details and links to the relevant Citrix documents.
Answer: The short answer: it's probably not a file you can find|grep.
Storage of XenServer snapshots is dependent on the type of storage repository (SR) used, as well as the Xen version number under which the SR was originally created.  This white paper provides all the gory details.
If you are ssh'd into dom0, the command xe vdi-list is-a-snapshot=true might get you started in the right direction.

where does xenserver keep vm snapshots?


Display Name of Connected Cell Tower

Question: On older Nokia phones, the name of the cell tower you were connected to was displayed, often the name of the neighborhood you were in. Is there a way to view this data now?
Answer: OpenSignal have an App which will give you a lot of information about the tower that you are connected to, which direction it is in, which other towers are available in the area, including ones on other networks, etc.

Free App
Gives all the information that you have asked for
Allows you to monitor your coverage through the day
Allows connection testing
If you are willing to share your data it is added to the data available to everybody.

Display Name of Connected Cell Tower


Xbox one update problem

Question: I have just started playing xbox one whenever i insert a game it demands an update thats too big. I dont want to play online at the moment, I just want to play offline. I want suggestions.
Comment: What game is it?
Comment: @TimmyJim - I think the OP means any game - i.e. no matter which game it is, it forces you to download an update.  On the 360 you could cancel the update and play offline.  I presume the OP wants to know if/how to do this on Xbox One games.
Answer: You'll have to disconnect your Xbox One from the internet. There are 2 ways to do it.

Physically unplug the LAN cable and disconnect from your WiFi network.
"Go offline". Go to the "Network settings" and select "Go offline".

Xbox one update problem


sentence completion

Question: can you help me in this sentence completion question?
These days youngsters are highly _____ ; they show a lot of experience of the world and social situations. The answer is given as wise/cognizant . why it cannot be "mature" ?
Comment: Where does this question come from? What was the context? Was there a limited number of words to pick from?
Comment: 'highly wise'… I don't think so. Possibly a source to avoid.
Comment: Yes, well, whoever wrote that test doesn't really know English in context for a test. Any of the three can be used by mature is the most likely. [We use: Why can't it be x]
Comment: "wise" doesn't work well at all - both in terms of the collocation "highly wise" and the meaning of wise doesn't fit "experience of social situations".  And cognizant is also a bad fit.  (normally one is cognizant of something)  I'd suggest "...youngsters are highly sophisticated..."  So I'd agree with DoneWithThis.  This is a source to avoid.
Answer: The word "highly" can mean "extremely; greatly; very much", but it seems to typically apply to certain words and not others. Consider examples from dictionaries:

highly amusing
highly paid
highly seasoned
highly recommended

So far, the pattern seems to be "participles". At the same time, ordinary adjectives such as "highly orange" or "highly wise" sound out of place.  There is probably more to that analysis.  In any case,

mature. Yes, it's a good choice. "[quite/fairly/pretty/very] mature".
cognizant. Usually cognizant is accompanied by further details: "cognizant of the risks". It's not often used as a plain word "He is cognizant".
wise.  The meaning matches the sentence. Nevertheless  there is something vaguely contradictory about "These days youngsters are wise". Perhaps switching to "wise beyond their years" sounds more eloquent.

To answer your question, "mature" is a fine replacement in that sentence.

sentence completion


How to lock device orientation for React Native

Question: Hi I want to know how to lock device orientation for React Native for both android and iOS. I need to lock the orientation once user rotate the device and application will lock with portrait or lanscape
Comment: Try this package https://www.npmjs.com/package/react-native-orientation-locker
Answer: For Android
For Android platform all we need to do is navigate to AndroidManifest.xml file that lies inside android/app/src/main path and add the property android:screenOrientation with a value or portrait or landscape respectively. The property has to be added in xml element activity which is a child of application element.
Simply add it as shown below:

For iOS
For iOS platform we need to open Xcode, and select the desired orientation modes from the Device Orientation section. As shown below:

Load the iOS part of your React Native project
Navigate to tab General from the horizontal menu
Go to down to Deployment Info section and check/uncheck the desired device orientation modes.

And you are set! Rebuild your project to verify that the changes are configured.
Comment: @DeSilva glad it helps


Git tags as an alternative to --no-ff in gitflow

Question: In https://nvie.com/posts/a-successful-git-branching-model/,  Driessen suggests using the --no-ff flag to make the history of commits for the dev branch clear.  This is an easy flag for individuals on a multi-person team to forget to add, and it also has the disadvantage of adding extra commits to the history (commits which may not be obviously identical in the git log output).  As an alternative, I'd like to propose adding a tag of the form dev_TIMESTAMP whenever we move the dev branch pointer to a new commit (and maybe using git hooks to automate this, though I have no experience with them).  Does this make sense?  What problems would I run into?  What could be done to overcome those problems?
Comment: FYI: You can make `--no-ff` the default: https://stackoverflow.com/a/9252042/389146
Answer: The main purpose of the --no-ff flag is to:

[avoid] losing information about the historical existence of a feature branch and groups together all commits that together added the feature.

(from the blog post you linked to in your question)
If keeping the historical information about feature branches is undesirable, then consider a squash-merge instead: git merge feature --squash
This gives you one clean commit for each feature branch, and then this question is moot. The --no-ff option is only useful if you want the history of the feature branch preserved. It sounds like this is just unnecessary clutter for your team. Don't feel like you need to use it. "A successful git branching model" is often cited and used, but it is an idea. Use what makes sense for your team, and discard the things that do not.
Tags are meant to be little more than significant bookmarks in the history of your repository. Typically releases are tagged, not every merge to a branch. That isn't to say you cannot tag every major merge into develop, but it would be surprising to people familiar with most git branching strategies.
Comment: I'm not sure if keeping historical information about feature branches is desirable or not.  I suspect most of the time we don't need or want it, but sometimes we might, so I'm reluctant to get rid of it.  I guess my hope is that, by adding tags to the dev branch it'd be relatively easy to filter out feature branch noise when I don't need it.  +1 to this answer, though.  The last point about surprising people is especially good.  Maybe the next question would be "surprising and confusing" or "surprising but not confusing".
Comment: @BenjaminBerman: The problem might be feature branches that live too long and have too many commits. Especially if a feature branch as a lot of useless "merge 'develop' into 'feature'" commits, which just adds noise to the history. The problem might exist in the feature branches instead.
Comment: I'm thinking that, at least the way we currently develop, to get shorter feature branches we need to either use something like github forks + squashing, or we need to rely on something other than our VCS to backup our work.
Comment: @BenjaminBerman: You could also try [trunk-based development](https://www.toptal.com/software/trunk-based-development-git-flow) in conjunction with [feature toggles](https://martinfowler.com/articles/feature-toggles.html), but this only works if you have good automated test coverage.
Comment: @BenjaminBerman The standard way to "filter out feature branch noise" is with a `--first-parent` view. I just added an answer which touches on that. I also like the last paragraph of this answer and IMHO it's enough reason in itself to avoid tagging every merge. Regarding "If keeping the historical information about feature branches is undesirable"... I'm thinking "undesirable" might be "isn't useful in your case", which is true when devs have 30 commits of "code review comments" and "fix something"; squash is the answer to that. Also, maybe the SCM PR tool retains the branch history as well.
Answer: tl;dr: there are 3 advantages of using --no-ff in Git Flow. Your suggestion of tagging every merge into develop basically achieves one of those advantages, but doesn't really address the other two.
Details:
The 3 advantages of using --no-ff are:

The primary purpose, and the one you are offering an alternative for, is when there is more than 1 commit getting merged into develop (or another shared branch) you can see that they were merged in together. Your tagging strategy could achieve this goal without information loss.
If for some reason you need to revert the merge (either shortly after or later), it's easier to revert a single merge commit than it is to revert X number of commits between the two "merge tags".
Git has a built-in feature for looking at historical changes to a branch, in the form of git log --first-parent. This enables you to easily compare the state of the changes to a branch, which in Git Flow would normally be each of the merge commits. This is an extremely useful view to have available. In your case that view wouldn't provide much value and you'd have to manually compare the "merge tag" in question with the previous "merge tag".

Ultimately I think this comes down to personal taste. It appears your motivation is to avoid

the disadvantage of adding extra commits to the history

Yet, you'd be replacing those merge commits with at least as many tags. (Note you potentially could be adding more tags than the number of commits you'd save.) I agree with the statement in Greg Bughgardt's answer that mentions you can do it but it would be kind of odd, compared to using merge commits to represent the same thing.
Side Note: Oftentimes when people argue against Git Flow's --no-ff to create merge commits, they do so because it adds (in their opinion, unneeded) complexity to the visual graph. They then may proceed to show an example of a Git Flow graph with many merge commits from feature branches that started from older merge bases, and then compare that to a straight line graph. But that is not a fair comparison because the straight line graph only works if you rebase your feature branches first. If you also rebase your feature branches in Git Flow you don't get a complex looking graph at all; instead you end up with a "semi-linear" graph with clean looking bubbles.
Answer: It seems to me that you should be tracking your development releases, by release, not by merge.
In which case, how you do or don't track various merges, or even if a full feature gets merged or not, becomes unimportant; because, you roll back to a release, not a commit.


Is it possible to go from tree climbing to hiding in sand like some sandlions do

Question: Is this evolutionary path possible with this creature because I don't know what would be in order to go from a creature that lives in the trees to living in a treeless environment. I had an idea that these creatures had a separate lineage that would get stranded on a island because the continents on my planet were shifting and in the island they were on now has hot temperatures and not many trees. After millions of years it would become a desert like environment. I already know that they can dig they do it to hide their eggs so why not use that to avoid the heat. They would eventually become smaller and would have a dusk to dawn like lifestyle. This is because they hibernate to avoid the heat (is that right?) and when its time they leave to hunt, drink water, courtship etc.if this isn't possible then i could use this for another creature.
Comment: Just for clearity.. your drawing.. am I correct your creature looks asymmetric ? It has daggers on its feet only on one side, and a rudimentary left arm ? or is it just the perspective, the arm stretched backward, did you not finish the drawing ?
Comment: yes i did and its perspective.the creature has six limbs and two pedapalps on the front of the torso.it is symmetrical and the reason it has limbs like that is because i took insperation of a cocanout crab and has one claw on each limb besides the pedapalps and the other limbs or grasping the tree so it wont fall hope this helps
Comment: Please note that the answer to the question "can X evolve?" is for all practical purposes always "yes." Given enough time anything can evolve into anything. Just to round out the discussion, "How can X evolve?" is IMO way to broad to answer on this site (giving someone a multi-million year history is a bit much). "What evolutionary pressures could rationalize X?" is much more practical on this site as it's a finite list of things that meet a specific need. Thanks.
Answer: Absolutely possible. You just have to have the trees dissappearing, or the ecologic tree niche closing up in another way (maybe because of fitter predators moving in?) - then your creatures lineage is free to drift towards greener pastures, or in this case underground trapping.
How about combining the sand-angle and the tree-angle? Your creature is top notch at trapping in trees, but then a climatic shift gives rise to a desertification of the environment, sand is accumulating and the trees cannot cope. So there is less and less trees, and more shrubs and your creatures lineage gets more and more adept at trapping on the ground. The ones with the best adaption to moving sand get an extra boost, because now there is a lot of sand, and thus as soon as an in-sand trapping behavior is premiered, that works quite well, perhaps at first still incorporating twigs and later going all-sand.
To repeat: All you need is an evolutionary pressure towards your ideal creature, and a pressure away from the way the antecedents did it, and you are all set (in reality the creatures would also need beneficial mutations as 'steps' on their way to a new ecological niche (and would need them faster than any competing species that might end up in that niche), but in your story we can take that as a given)
Comment: thank you for the comment this helps


wcat for load testing asp.net website

Question: I am new to load testing and i want to load test my asp.net web site. After doing some investigation I came across this tool wcat by Microsoft for load testing. To me it seems to be a stress tool for putting load on one or multiple pages, more like testing the IIS and not the application itself. But what if i want to simulate a scenario, for example i have a wizard in my website that has multiple steps. The user is able to follow 4-5 steps to create a record.
Is Wcat the right tool to test my asp.net website for the wizard above? If yes, can i be pointed to some guide that does the same?
Answer: Wcat can load test web applications and navigation scenarios. However, it does not have a recorder, so you need to create your own script reflecting the navigation steps.
Fortunately, you can use Fiddler as a recorder and then export captured sessions as Wcat script as described in this blog.  Until recently, Fiddler's WCAT exporter did not support POST requests but it may have been added by now as described here.
Answer: This is the old question. Since answer has not marked yet, here is my thought.  WCAT is web application stress or load test tool. It would send send concurrent users requests to web server.
The scenario you talked about is testing a functionality of wizard. Though there are various tools available to test this, I would suggest that Selenium which could test the exact scenario you are expecting. This test tool mainly used for test an application using browser as user do. Steps of the wizard could be recorded and replayed and assertion would make sure the test results are expected.

wcat for load testing asp.net website


another solution to PDE possible?

Question: hi there,
i have the following pde:
$$(\partial_x x^4 \partial_x - \partial_t^2)y(x,t)=0$$ and found the solution $$y=a+t^2-1/x^2$$, with a a constant.
Is this solution unique? Does anyone know of any other solution, tricks to generate other solutions? Thankful for help.
Regina
Comment: Your solution can be slightly generalised to $y=A+Bt^2-B/x^2$ (both $A$ and $B$ are arbitrary constants). This equation can also be solved by using the standard method of separation of variables.
Answer: Since your equation is linear and homogeneous, linear combinations of solutions are solutions.
Basic solutions include $1$ and $t^2 - 1/x^2$ from your solution, as well as
$1/x^3$, $t$, and $(1-a/x) \exp(a/x) \exp(a t)$ and $(1-b/x) \exp(b/x) \exp(-b t)$ for any constants a and b.
Answer: PDEs look like ODEs, but only look like. The solution set of an ODE of order $n$ is usually parametrized by $n$ scalar (integration constant). On the contrary, the solution set of a PDE of order $n$ in $d$ independent variables ($d=2$ in your case) is usually parametrized by $n$ functions of $d-1$ variables. This is clear in the hyperbolic case because you just solve a Cauchy problem with initial data on a non-characteristic hypersurface. More generally, if the equation has analytic coefficients, you can apply the Cauchy-Kowalevskaia Theorem.
In conclusion, your explicit solutions are far from unique.
Answer: More generally, $F(t + 1/x) - x F'(t + 1/x)$ and $F(t - 1/x) + x F'(t - 1/x)$ for any differentiable function F.

another solution to PDE possible?


Upgrade from t1.microto to 2.micro - AWS Free Usage Tier

Question: I run my blog on a t1.micro available under aws free usage tier. I have been facing memory crunch due to only 600mb available ram. http://aws.amazon.com/free/ says that free tier provides t2.micro instances which have 1gb ram. I was wondering if its possible for me to upgrade to the same in free tier.
Answer: Yes, that should be no problem. Just stop your instance, and you can then change the instance type to t2.micro and start it again. For other instance types, you'd have to worry about losing data from your instance-store volumes, but t1.micro and t2.micro instances don't have any. However, you should note that its IP addresses will change unless you have an Elastic IP Address for it.


Has Brian Jacques's estate licensed the Redwall IP for an official role-playing game?

Question: Pretty much what it says on the tin. Is there an RPG system that has been licensed for the Redwall setting created by Brian Jacques? I expect the answer to be "No" so as a follow up, has there been any material created to re-skin an existing system to take place in that setting?
Comment: Considering the answer you have accepted, I don't think the edit here was fair—the answer you have accepted is a poor one to the question you have changed this too. I think this should be reverted and re-closed. The question as-is *might* be acceptable, but not if it's a none-too-subtle dodge around the ban on game-rec questions.
Comment: @KRyan I was wary of the question even when suggesting the edit that was eventually made to the title and still concerned when the question's body was so lightly edited. Yet the title specifies licensing, so I figured it was acceptable as, essentially, a `games-industry` question like *Who owns the license to produce an RPG based on this IP?* While not a *great* question as such information is usually pretty readily available, an answer may still require some digging (cf. [here](https://en.wikipedia.org/wiki/Mistborn#Role-playing_game)) and expertise in evaluating results, if any.
Comment: @KRyan (Not that I want to see the site deluged by questions of who holds what license for every obscure IP—such questions are better-suited to an author's forums or by contacting the author or her representatives directly—, but the Redwall IP is a big enough deal that I figured it'd be okay; we're not talking, like, [Thursday Next](https://en.wikipedia.org/wiki/Thursday_Next) here or anything, even though I wish we were.)
Answer: You want Mouse Guard.
There's no licensed game for Redwall, but the Mouse Guard RPG is the closest you'll get. Based on the comic book by David Petersen, in the world of Mouse Guard, "mice struggle to live safely and prosper amongst harsh conditions and a host of predators" — which it has in common with Redwall. David Petersen himself commented upon Redwall with respect to Mouse Guard in his blog post memorializing Brian Jacques:

Redwall was as much an influence on Mouse Guard as it was an anti-influence. It was while I was developing the ideas for my own group of mouse rangers I was handed the first book in the series. I thoroughly enjoyed Brian's writing, and his gift for including vocal accents into the text. It charged in me even more desire to pursue my love of animal stories and medieval cultures.
However, I knew instantly that I could never top what Mr. Jacques was doing. So when I say his books were a counter influence, I only mean that I had to set out in a different path from what he was already master of.

The Mouse Guard RPG deals with fantasy rodent and smallfolk culture, handles combat well, and is a beautiful game besides.