How can one derive Schrödinger equation?

Question: The Schrödinger equation is the basis to understanding quantum mechanics, but how can one derive it? I asked my instructor but he told me that it came from the experience of Schrödinger and his experiments. My question is, can one derive the Schrödinger equation mathematically?
Comment: Possible duplicate: http://physics.stackexchange.com/questions/135872/what-does-the-schrodinger-equation-really-mean/136085#136085
Comment: @Bubble related, but not a duplicate IMO since that question is asking for a physical motivation, not a derivation.
Comment: Related http://physics.stackexchange.com/q/30537/
Answer: Be aware that a "mathematical derivation" of a physical principle is, in general, not possible. Mathematics does not concern the real world, we always need empirical input to decide which mathematical frameworks correspond to the real world.
However, the Schrödinger equation can be seen arising naturally from classical mechanics through the process of quantization. More precisely, we can motivate quantum mechanics from classical mechanics purely through Lie theory, as is discussed here, yielding the quantization prescription
$$ \{\dot{},\dot{}\} \mapsto \frac{1}{\mathrm{i}\hbar}[\dot{},\dot{}]$$
for the classical Poisson bracket. Now, the classical evolution of observables on the phase space is
$$ \frac{\mathrm{d}}{\mathrm{d}t} f = \{f,H\} + \partial_t f$$
and so its quantization is the operator equation
$$ \frac{\mathrm{d}}{\mathrm{d}t} f = \frac{\mathrm{i}}{\hbar}[H,f] + \partial_t f$$
which is the equation of motion in the Heisenberg picture. Since the Heisenberg and Schrödinger picture are unitarily equivalent, this is a "derivation" of the Schrödinger equation from classical phase space mechanics.
Comment: @LoveLearning: It all depends on where you want to start. In my view, the most mysterious element of both the Schrödinger equation and the path integral is the appearance of $\mathrm{i}$. You can indeed derive the SE from the path integral (and vice versa), but then you have to explain why the heck you are integrating over $e^{iS/\hbar}$ in the first place. The procedure of geometric quantization at least gives a mathematical motivation for that, starting from classical mechanics. Of course, if you believe that we should not start from classical mechanics, then you'll not find this convincing.
Comment: @ACuriousMind +1: I think I'm going to make a T-shirt that reads "It all depends on where you want to start" and start selling it.  That way, you can just point to your chest next time.  I will also benefit immensely from wearing it when I walk around campus.  Can I mark you down for one order?
Comment: What about the "derivation" via path integrals?
Comment: Derive Schrödinger equation via path integrals can be at most a "physical" derivation, but never a mathematical deivation, since path integrals in the sense of Feynman **do not** have a mathematical meaning.
Comment: @LoveLearning See my newly added answer for more clarifications.
Comment: For completeness, I want to add that this "first quantization" method for deriving non-relativistic QM was only discovered *after* the Schrodinger equation was already known and written down through other means. It's a "hey, would you look at *that*" sort of moment in the history.
Comment: @phonon the op asked us to derive the SE, and my suggestion was to drive it via PI. OP could then go on and ask about the PI but that would be another question.
Comment: I agree with you guys regarding "mathematically deriving" some physical principle (which sounds like an oxymoron). I think I misunderstood what the OP really wanted.
Answer: Small addition to ACuriousMind's great answer, in reply to some of the comments asking for a derivation of Schrödinger wave equation, using the results of Feynman's path integral formalism:
(Note: not all steps can be included here, it would be too long to remain in the context of a forum-discussion-answer.)
In the path integral formalism, each path is attributed a wavefunction $\Phi[x(t)]$, that contributes to the total amplitude, of let's say, to go from $a$ to $b.$ The $\Phi$'s have the same magnitude but have differing phases, which is just given by the classical action $S$ as was defined in the Lagrangian formalism of classical mechanics.  So far we have: $$ S[x(t)]= \int_{t_a}^{t_b} L(\dot{x},x,t) dt $$
and $$\Phi[x(t)]=e^{(i/\hbar) S[x(t)]}$$
Denoting the total amplitude $K(a,b)$, given by: $$K(a,b) = \sum_{paths-a-to-b}\Phi[x(t)]$$
The idea to approach the wave equation, describing the wavefunctions as a function of time, we should start by dividing the time interval between $a$-$b$ into $N$ small intervals of length $\epsilon$, and for a better notation, let's use $x_k$ for a given path between $a$-$b$, and denote the full amplitude, including its time dependance as $\psi(x_k,t)$ ($x_k$ taken over a region $R$):
$$\psi(x_k,t)=\lim_{\epsilon \to 0} \int_{R} \exp\left[\frac{i}{\hbar}\sum_{i=-\infty}^{+\infty}S(x_{i+1},x_i)\right]\frac{dx_{k-1}}{A} \frac{dx_{k-2}}{A}... \frac{dx_{k+1}}{A} \frac{dx_{k+2}}{A}... $$
Now consider the above equation if we want to know the amplitude at the next instant in time $t+\epsilon$:
$$\psi(x_{k+1},t+\epsilon)=\int_{R} \exp\left[\frac{i}{\hbar}\sum_{i=-\infty}^{k}S(x_{i+1},x_i)\right]\frac{dx_{k}}{A} \frac{dx_{k-1}}{A}... $$
The above is similar to the equation preceding it, the difference relying on the hint that, the added factor with $\exp(i/\hbar)S(x_{k+1},x_k)$ does not involve any of the terms $x_i$ before $i<k$, so the integration can be preformed with all such terms factored out. All this reduces the last equation to:
$$\psi(x_{k+1},t+\epsilon)=\int_{R} \exp\left[\frac{i}{\hbar}\sum_{i=-\infty}^{k}S(x_{i+1},x_i)\right]\psi(x_k,t)\frac{dx_{k}}{A}$$
Now a quote from Feynman's original paper, regarding the above result:

This relation giving the development of $\psi$ with time will be
shown, for simple examples, with suitable choice of $A$, to be
equivalent to Schroedinger's equation. Actually, the above equation is
not exact, but is only true in the limit $\epsilon \to 0$ and we shall
derive the Schroedinger equation by assuming this equation is valid
to first order in $\epsilon$. The above need only be true for small
$\epsilon$ to the first order in $\epsilon.$

In his original paper, following up the calculations for 2 more pages, from where we left things, he then shows that:

Canceling $\psi(x,t)$ from both sides, and comparing terms to first
order in $\epsilon$ and multiplying by $-\hbar/i$ one obtains

$$-\frac{\hbar}{i}\frac{\partial \psi}{\partial t}=\frac{1}{2m}\left(\frac{\hbar}{i}\frac{\partial}{\partial x}\right)^2 \psi + V(x) \psi$$ which is Schroedinger's equation.
I would strongly encourage you to read his original paper, don't worry it is really well written and readable.

References:
Space-Time Approach to Non-Relativistic Quantum Mechanics by R. P. Feynman, April 1948.
Feynman Path Integrals in Quantum Mechanics, by Christian Egli
Comment: The Schroedinger Equation is simply the Hamiltonian ie. Kinetic + Potential energy as a function of momenta and coordinates alone, written with Quantum operators for momentum replacing the classical definition of momentum.  Hamilton's equation is well known from Classical Physics, has been tested for ~2 Centuries, and is easy to use.  The only 'new' idea is the Quantum operator for momentum, which isn't intuitive or obvious, but is used because it gives the correct answer.
Comment: Do you, per chance, have a link to Schroedinger's papers in English?
Comment: @MadPhysicist unfortunately I cannot find the very early ones in English, but at least there's his paper on ["An Undulatory Theory of the Mechanics of Atoms and Molecules"](http://materias.df.uba.ar/f4Aa2012c2/files/2012/08/schrod.pdf). Among the very first ones by Heisenberg and afterwards Schrödinger were "Über quantentheoretische Umdeutung kinematischer und mechanischer Beziehungen." and "Quantisierung als Eigenwertproblem" respectively. Try to look for the English translation of these. What are you exactly interested in? Maybe I can recommend more modern material to you.
Comment: I was particularly interested in his papers pertaining to expanding on the work of de Broglie and producing the Schroedinger equation.
Answer: According to Richard Feynman in his lectures on Physics, volume 3, and paraphrased "The Schrodinger Equation Cannot be Derived". According to Feynman it was imagined by Schrodinger, and it just happens to provide the predictions of quantum behavior.
Comment: That was in the 1960's. But I also found this 2006 paper in the American Journal of Physics: http://arxiv.org/abs/physics/0610121 which claims a derivation.
Comment: See also [this](https://en.wikipedia.org/wiki/Schr%C3%B6dinger_equation#The_wave_equation_for_particles)
Answer: Fundamental laws of physics cannot be derived (turtles all the way down and all that).
However, they can be motivated in various ways. Direct experimental evidence aside, you can argue by analogy - in case of the Schrödinger equation, comparisons to Hamiltonian mechanics and the Hamilton-Jacobi equation, fluid dynamics, Brownian motion and optics have been made.
Another approach is arguing by mathematical 'beauty' or necessity: You can look at various ways to model the system and go with the most elegant approach consistent with constraints you imposed (ie reasoning in the vein of 'quantum mechanics is the only way to do X' for 'natural' or experimentally necessary values of X).
Answer: While it is in general impossible to derive the laws of physics in the mathematical sense of the word, a strong motivation or rationale can be given most of the time. Such impossibility arises from the very nature of physical sciences which attempt to stretch the all-to-imperfect logic of the human mind onto the natural phenomena around us. In doing so, we often make connections or intuitive hunches which happen to be successful at explaining phenomena in question. However, if one had to point out which logical sequence was used in producing the hunch, he would be at a loss - more often than not such logical sequence simply does not exist.
"Derivation" of the Schroedinger equation and its successful performance at explaining various quantum phenomena is one of the best (read audacious, mind-boggling and successful) examples of the intuitive thinking and hypothesizing which lead to great success. What many people miss is that Schroedinger simply took the ideas of Luis de Broglie further to their bold conclusion.
In 1924 de Broglie suggested that every moving particle could have a wave phenomenon associated with it. Note that he didn't say that every particle was a wave or vice versa. Instead, he was simply trying to wrap his mind around the weird experimental results which were produced at the time. In many of these experiments, things which were typically expected to behave like particles also exhibited a wave behavior. It is this conundrum which lead de Broglie to produce his famous hypothesis of $\lambda = \frac{h}{p}$. In turn, Schroedinger used this hypothesis as well as the result from Planck and Einstein ($E = h\nu$) to produce his eponymous equation.
It is my understanding that Schroedinger originally worked using Hamilton-Jacobi formalism of classical mechanics to get his equation. In this, he followed de Broglie himself who also used this formalism to produce some of his results. If one knows this formalism, he can truly follow the steps of the original thinking. However, there is a simpler, more direct way to produce the equation.
Namely, consider a basic harmonic phenomenon:
$ y = A sin (wt - \delta)$
for a particle moving along the $x$-axis,
$ y = A sin \frac{2\pi v}{\lambda} (t - \frac{x}{v}) $
Suppose we have a particle moving  along the $x$-axis. Let's call the wave function (similar to the electric field of a photon) associated with it $\psi (x,t)$. We know nothing about this function at the moment. We simply gave a name to the phenomenon which experimentalists were observing and are following de Broglie's hypothesis.
The most basic wave function has the following form:
$\psi = A e^{-i\omega(t - \frac{x}{v})}$, where $v$ is the velocity of the particle associated with this wave phenomenon.
This function can be re-written as
$\psi = A e^{-i 2 \pi \nu (t - \frac{x}{\nu\lambda})} = A e^{-i 2 \pi (\nu t - \frac{x}{\lambda})}$, where $\nu$ - the frequency of oscillations and $E = h \nu$. We see that $\nu = \frac{E}{2 \pi \hbar}$ The latter is, of course, the result from Einstein and Planck.
Let's bring the de Broglie's result into this thought explicitly:
$\lambda = \frac{h}{p} = \frac{2\pi \hbar}{p}$
Let's substitute the values from de Broglie's and Einstein's results into the wave function formula.
$\psi = A e^{-i 2 \pi (\frac{E t}{2 \pi \hbar} - \frac{x p}{2 \pi \hbar})} = A e^{- \frac{i}{\hbar}(Et - xp)} (*)$
this is a wave function associated with the motion of an unrestricted particle of total energy $E$, momentum $p$ and moving along the positive $x$-direction.
We know from classical mechanics that the energy is the sum of kinetic and potential energies.
$E = K.E. + P.E. = \frac{m v^2}{2} + V = \frac{p^2}{2 m} + V$
Multiply the energy by the wave function to obtain the following:
$E\psi = \frac{p^2}{2m} \psi + V\psi$
Next, rationale is to obtain something resembling the wave equation from electrodynamics. Namely we need a combination of space and time derivatives which can be tied back into the expression for the energy.
Let's now differentiate $(*)$ with respect to $x$.
$\frac{\partial \psi}{\partial x} = A (\frac{ip}{\hbar}) e^{\frac{-i}{\hbar}(Et - xp)}$
$\frac{\partial^2 \psi}{\partial x^2} = -A (\frac{p^2}{\hbar^2}) e^{\frac{-i}{\hbar}(Et - xp)} = \frac{p^2}{\hbar^2} \psi$
Hence, $p^2 \psi = -\hbar^2 \frac{\partial^2 \psi}{\partial x^2}$
The time derivative is as follows:
$\frac{\partial \psi}{\partial t} = - A \frac{iE}{\hbar} e^{\frac{-i}{\hbar}(Et - xp)} = \frac{-iE}{\hbar}\psi$
Hence, $E \psi = \frac{-\hbar}{i} \frac{\partial \psi}{\partial t}$
The expression for energy we obtained above was $E\psi = \frac{p^2}{2m} \psi + V\psi$
Substituting the results involving time and space derivatives into the energy expression, we obtain
$\frac{-i}{\hbar} \frac{\partial \psi}{\partial t} = \frac{- \hbar ^2}{2m} \frac{\partial ^2 \psi}{\partial x^2} + V\psi$
This, of course, became better known as the Schroedinger equation.
There are several interesting things in this "derivation." One is that both the Einstein's quantization and de Broglie's wave-matter hypothesis were used explicitly. Without them, it would be very tough to come to this equation intuitively in the manner of Schroedinger. What's more, the resulting equation differs in form from the standard wave equation so well-known from classical electrodynamics. It does because the orders of partial differentiation with respect to space and time variables are reversed. Had Schrodinger been trying to match the form of the classical wave equation, he would have probably gotten nowhere.
However, since he looked for something containing $p^2\psi$ and $E\psi$, the correct order of derivatives was essentially pre-determined for him.
Note: I am not claiming that this derivation follows Schroedinger's work. However, the spirit, thinking and the intuition of the times are more or less preserved.
Answer: There are already a lot of good answers to this question. Of course, as already pointed out by many others, there is no "mathematical derivation" of Schrödinger's equation from first principles, since it is one of the axioms of quantum mechanics. However, there a many different ways to motivate this equation, from which many were already described by others here. One of them, which I like personally very much, is still missing, and hence I would like to add it here at this point.
Lets take the other axioms of quantum mechanics for granted, like the description of states using unit rays in a seperable Hilbert space and observables in terms of hermitian operators. Then there is very nice way to motivate the Schrödinger equation from physical ideas and some very deep and beautiful mathematical theorems.
The starting point is the following: A physical state is represented by a vector $\vert\psi_{0}\rangle\in\mathcal{H} $ in some Hilbert space $\mathcal{H}$. Now, we would like to analyse the time-evolution of this state. For this, we take a mapping $U:\mathbb{R}_{\geq 0}\to\mathcal{B}(\mathcal{H})$, where $\mathcal{B}(\mathcal{H})$ is the set of bounded operators in $\mathcal{H}$, such that
$$\mid\psi(t)\rangle=U(t)\vert\psi_{0}\rangle.$$
Now, by the probabilistic description of quantum mechanics, the state $\mid\psi_{0}\rangle$ should be normalized and similarely, also $\mid\psi(t)\rangle$ should be normalized. As a consequence, we see that the operator $U(t)$ has to be unitary for all $t$. Furthermore, it is easy to see that we must have that
$$U(t+s)=U(t)U(s).$$
To sum up, we see that the time-evolution operator $\{U(t)\}_{t\in\mathbb{R}_{\geq 0}}$ has to have the following properties:

$U(t)$ is unitary for all $t$ and $U(0)=1_{\mathcal{H}}$.
The mapping $U:\mathbb{R}_{\geq 0}\to\mathcal{B}(\mathcal{H})$ is a group homomorphism, i.e. $U(t+s)=U(t)U(s)$ for all $t,s$.
From the mathematical side, we may also assume that the family of operators is "strongly-continuous", i.e. $\lim_{t\to s}\Vert U(t)-U(s)\Vert=0$.

A family of such operators $\{U(t)\}_{t\in\mathbb{R}_{\geq 0}}$ is called a "strongly-continuous unitary one-parameter group" in mathematics. This type of objects is a special case of "strongly continuous one-parameter semigroups", which are well-studied in the mathematics literature (i.e. theorem of Hille-Yosida, etc.).
Now, how is this related to the Schrödinger equation? It turns out that the statement "the time-evolution operator defines a strongly-continuous unitary one-parameter group" is equivalent to say "the state satisfies the Schrödinger equation" for a suitable operator $H$! Let me discuss this in the following: First of all, it is a general fact, which is not too hard to prove, that for every (possibly unbounded operator) $A:\mathcal{D}(A)\to\mathcal{H}$, where $\mathcal{D}(A)\subset\mathcal{H}$ is some domain, the family
$$U(t):=e^{-itA}$$
defines a strongly-continous one-parameter group. Let me also stress that this expression is totally well-defined: In physics, one often just writes the exponential of an operator without discussing how to make sense out of these expressions, but this can be done in total rigour. For bounded operators, one can just take the usual exponential series, which always converges. For unbounded operators (like differential operators) one can use the "spectral theorem", which provides a rigours way to write expressions like $U(t):=e^{itA}$ (and in fact reduces to the standard exponential series for bounded operators). Using all the properties of the functional calculus coming from the spectral theorem, one can also prove that the group $U(t)=e^{itA}$ has the following properties:

The domain $\mathcal{D}(A)$ can be characterised as $$\mathcal{D}(A)=\bigg\{\vert\psi\rangle\in\mathcal{H}\,\bigg\vert\,\frac{\mathrm{d}}{\mathrm{d}t}\bigg\vert_{t=0} U(t)\vert\psi\rangle:=\lim_{t\to 0}\frac{U(t)\vert\psi\rangle -\vert\psi\rangle}{t}\, \text{exists} \bigg\}.$$
Every $\vert\psi\rangle\in\mathcal{D}(A)$ satisfies the following equation $$A\vert\psi\rangle=i\frac{\mathrm{d}}{\mathrm{d}t}\bigg\vert_{t=0} U(t)\vert\psi\rangle.$$

Now, the main point is that also the reverse is true: For every strongly-continuous unitary one-parameter group $\{U(t)\}_{t\in\mathbb{R}_{\geq 0}}$, there is a self-adjoint operator $A:\mathcal{D}(A)\to\mathcal{H}$, such that $U(t)=e^{-itA}$. This is the so-called Stone's theorem.
To sum up, we have seen that the fact that a time-evolution should preserve the norm of a state, implies that the time-evolution is described by a strongly-continuous one-parameter group $\{U(t)\}_{t\in\mathbb{R}_{\geq 0}}$. Furthermore, we have seen that (by Stone's theorem) there exists a self-adjoint operator $H:\mathcal{D}(H)\to\mathcal{H}$ such that $U(t)=e^{-itH}$. Defining a time-dependent state via $\vert\psi(t)\rangle=U(t)\vert\psi_{0}\rangle$, we also have seen that this satisfies the equation
$$H\vert\psi(t)\rangle=i\frac{\mathrm{d}}{\mathrm{d}s}\bigg\vert_{s=0} U(s)\vert\psi(t)\rangle=i\frac{\mathrm{d}}{\mathrm{d}s}\bigg\vert_{s=0} \vert\psi(t+s)\rangle=i\frac{\mathrm{d}}{\mathrm{d}s}\bigg\vert_{s=t} \vert\psi(s)\rangle=i\frac{\mathrm{d}}{\mathrm{d}t} \vert\psi(t)\rangle,$$
but this is nothing else then Schrödinger's equation! Furthemore, also the reverse is true. Let us start from Schrödinger's equation
$$H\vert\psi(t)\rangle=i\frac{\mathrm{d}}{\mathrm{d}t} \vert\psi(t)\rangle.$$
Choosing an initial value $\vert\psi(0)\rangle=\vert\psi_{0}\rangle$, then one can show that this equation has a unique solution, which is given by
$$\vert\psi(t)\rangle=e^{-itH}\vert\psi_{0}\rangle$$
and as we have seen above, the family $U(t):=e^{-itH}$ is in fact a strongly-continuous unitary one-parameter group.
To sum up, the existence of a unitary time-evolution (which physically comes from the fact that we want to preserve the probability) is equivalent to the statement of Schrödinger's equation!

Let me stress that the above arguments are only valid if one assumes that the Hamilton operator itself is time-independent. However, one can generalize many of the argument above using "two-parameter unitary groups" $U(t,s)$. In this case, the story is mathematically more involved, but still one can proof a lot of nice results using this. For more details, I would recomment the second volume of Reed and Simons book "Methods of Modern Mathematical Physics".
Answer: In Mathematics you derive theorems from axioms and the existing theorems.
In Physics you derive laws and models from existing laws, models and observations.
In this case we can start from the observations in photoelectric effect to get the relation between photon energy and frequency. Then continue with the special relativity where we observed the speed of the light is constant in all reference frames. From this when generalizing the kinetic energy we can get the mass energy equivalence. Combining the two we can assign mass to the photon, consequently we can get the momentum of a photon as function of the wavenumber.
Generalizing the energy-frequency and the momentum-wavenumber relation when have the De-Broglie relations. Which is applicable to any particles.
Assuming that a particle have 0 energy when it stands still (you can do it), although it doesn't cause too much trouble if you leave the constant term there, in the later phases you can simply put it into the left side of the equation. We can deal with the kinetic energy. Substituting the non-relativistic kinetic into the relation and reordering we can have the following dispersion relation:
$$\omega = \frac{\hbar k^2}{2m}$$
The wave equation can be derived from the dispersion relation of the matter waves using the way I mentioned in that answer.
In this case we will need the laplacian and first time derivative:
$$\nabla^2 \Psi + \partial_t \Psi = -k^2\Psi - \frac{i \hbar k^2}{2m}\Psi$$
Multiplying the time derivative with $-\frac{2m}{i\hbar}$, we can zero the right side:
$$\nabla^2 \Psi - \frac{2m}{i\hbar} \partial_t \Psi = -k^2\Psi + k^2\Psi = 0$$
We can reorder it to obtain the time dependent schrödinger equation of a free particle:
$$ \partial_t \Psi = \frac{i\hbar}{2m} \nabla^2 \Psi$$
Answer: To my mind there are two senses in which we can "derive" a result in physics. New theories try to address the shortcomings of older ones by upgrading what we already have, giving new results. They also recover old results. I suppose we can call both derivations.
For example, the TISE and TDSE were first obtained because quantum mechanics said that, where classical mechanics would imply $f=0$, we should have $\hat{f}\left|\psi\right\rangle = 0$, with $\hat{f}$ the operator promotion of $f$, which in this case is $f=E-\frac{p^2}{2m}-V$ with operators $E=i\hbar\partial_t,\,\mathbf{p}=-i\hbar\boldsymbol{\nabla}$. (Some results become the weaker $\left\langle\psi\right|\hat{f}\left|\psi\right\rangle = 0$, e.g. with $f=\frac{d\mathbf{p}}{dt}+\boldsymbol{\nabla}V$, so I'm not being entirely honest here. But we expect $\hat{E}$-eigenstates are important because the probability distribution of $E$ is conserved.)
Note that the above paragraph summarises how Schrödinger was derived in the first sense, and its ending parenthesis hints at how Newton's second law was "derived" in my second sense. And everyone talking about path integrals is hinting at a type-2 derivation for both results (path integrals obtain a transition amplitude in terms of $e^{iS/\hbar}$ with $S$ the classical action now miraculously coming out of a hat, so technically our direct recovery is of Lagrangian mechanics rather than the equivalent Newtonian formulation).
I'll leave people to fight over which, if either, type of derivation is "valid" or "better", but physical insight requires frequent doses of both. I think it's worth distinguishing them in a discussion like this.
Answer: It is common in books on path integrals to see the claim that we can derive the Schrodinger equation (indeed Ref. [2] below claims it can indeed be derived via path integrals), so it is surprising to see people say that it can't be derived.
I will say that of course there is a way to directly derive the Schrodinger equation simply (i.e. without the formal machinery of one-parameter groups and Stone's Theorem etc...) without path integrals and within the logic of (Copenhagen) quantum mechanics. It is given in (the absolutely canonical textbook of) reference [1]. Another derivation is given by Dirac in ([3], Sec. 27).
Assuming the existence of a wave function
$$\Psi(q,t)$$
(where $q$ are generalized coordinates), and assuming the wave function completely describes the state of a physical quantum system (i.e. assuming we can obtain 'complete information' about the system and do not need density matrices, to which a slight modification of what follows still applies, see [1]), this implies that the state of the system at all future instants is determined (to within the obvious limits of quantum mechanics). This means that the time derivative $\frac{\partial \Psi}{\partial t}$ of the wave function must be determined by the value of $\Psi(q,t)$ at that instant, and the  superposition principle implies this relationship has to be a linear relationship, so that
$$\frac{\partial \Psi(q,t)}{\partial t} = \tilde{\hat{H}} \Psi(q,t)$$
should hold, for some linear operator $\tilde{\hat{H}}$ (notice the hat and the tilde on $\tilde{\hat{H}}$).
Further assuming the existence of classical mechanics as a limit of the theory (a fundamental unavoidable assumption at least in Copenhagen quantum mechanics), we have to describe how a quantum wave function reduces to give a classical picture (and in fact disappears in the classical limit).
The perspective taken in [1] is that this limiting procedure is done by requiring that the wave function $\Psi(q,t)$, which is a complex number, written in complex exponential form
$$\Psi(q,t) = |\Psi(q,t)|e^{i \phi},$$
behaves in the same way that for example an electromagnetic field, which can also be written in complex exponential form (of course there we must still take the real part), behaves when we take the limit of geometric optics, where the notion of definite 'paths' can be defined, despite the wave-like nature of electromagnetism.
Since quantum mechanics says that paths don't exist, but that they should exist more and more accurately until they completely exist in a to-be-defined 'classical limit', this is a very natural assumption to make about the behavior of a wave function as we approach the 'classical limit'. What this means is that we should ignore the amplitude of $\Psi(q,t)$ like we ignore the amplitude in geometric optics approximation, and we should treat the argument of the exponential (the eikonal) like a function which determines 'classical paths'.
But there is obviously some function we know of that does precisely this, the 'classical action $S$'. Thus we assume in a 'quasi-classical' limit, where the wave function more and more accurately describes a classical system, should reduce to the form
$$\Psi(q,t) = |\psi(q,t)| e^{i S/\hbar}$$
with $|\psi(q,t)|$ being a slowly varying amplitude that we can neglect the more classical we get, and at this stage $\hbar$ is literally just a constant with the dimensions of action $[S] = [\hbar] = [E][T]$ used to ensure the argument of the exponential is dimensionless.
We now see the classical limit is taken by sending
$$\hbar \to 0$$
which means the argument of the exponential of the quasi-classical wave function will become infinite and make no sense unless the action $S$ is minimized in the $\hbar \to 0$ limit, which is exactly the condition needed for a classical action to completely describe the dynamics of a classsical system.
Therefore in the quasi-classical limit, if we take a time derivative of $\Psi(q,t)$, we can neglect contributions from the amplitude, to find
$$\frac{\partial \Psi(q,t)}{\partial t} = \frac{i}{\hbar} \frac{\partial S}{\partial t} \Psi(q,t).$$
From classical analytical mechanics we know that
$$\frac{\partial S}{\partial t} = - H$$
so we now approximately have
$$\frac{\partial \Psi(q,t)}{\partial t} = - \frac{i}{\hbar} H \Psi(q,t).$$
ignoring contributions from the amplitude, or rather
$$i \hbar \frac{\partial \Psi(q,t)}{\partial t} = H \Psi(q,t).$$
We now see that the linear operator $\tilde{\hat{H}}$ must reduce to the classical Hamiltonian $H$ (multiplied by $- \frac{i}{\hbar}$) in the classical limit, and we refer to
$$i \hbar \frac{\partial \Psi(q,t)}{\partial t} = \hat{H} \Psi(q,t)$$
(with $\hat{H}$ reducing to the classical Hamiltonian $H$ in the classical limit) as the Schrodinger equation.
The form of the operator $\hat{H}$ is still left arbitrary, indeed we have not yet invoked non-relativistic Galilean or Relativistic Poincare symmetry principles which are required to fix the form of $\hat{H}$.
We can similarly determine the momentum operator from the quasi-classical limit, noting from
$$\nabla \Psi = \frac{i}{\hbar} (\nabla S) \Psi$$
and the classical analytical mechanics definition of momentum as
$$\mathbf{p} = \nabla S$$
that the $\hat{\mathbf{p}}$ in
$$\hat{\mathbf{p}} \Psi = - i \hbar \hat{\nabla} \Psi$$
is the operator reducing to the classical momentum in the classical limit.
One immediate consequence of this perspective is by differentiating the expected value of some operator $\hat{f}$
$$
\overline{\hat{f}} = \int dq \Psi^* \hat{f} \Psi$$
we can easily derive that the time derivative of $\hat{f}$ is
\begin{align*}
\frac{d \hat{f}}{dt} &= \frac{\partial \hat{f}}{\partial t} + \frac{i}{\hbar}(\hat{H} \hat{f} - \hat{f}\hat{H}) \\
&= \frac{\partial \hat{f}}{\partial t} + \frac{i}{\hbar}[\hat{H},\hat{f}].
\end{align*}
But we also have the classical result that some function $f$ of the canonical variables satisfies
\begin{align*}
\frac{d f}{dt} &= \frac{\partial f}{\partial t} + (\frac{\partial f}{\partial q_i} \frac{d q_i}{dt} + \frac{\partial f}{\partial p_i} \frac{d p_i}{dt}) \\
&= \frac{\partial f}{\partial t} + (\frac{\partial f}{\partial q_i} \frac{\partial H}{\partial p_i} - \frac{\partial f}{\partial p_i} \frac{\partial H}{\partial q_i}) \\
&= \frac{\partial f}{\partial t} - \{H,f \}_{P.B.}
\end{align*}
Thus the time derivative of the quantum version of a classical function $f$ should reduce to these equations of motion in the classical limit, showing that in the quasi-classical limit the commutator
$$[\hat{H},\hat{f}] = \hat{H} \hat{f} - \hat{f}\hat{H}$$
reduces to
$$[\hat{H},\hat{f}] = i \hbar \{H,f\}_{P.B.} + \mathcal{O}(\hbar^2),$$
justifying the fundamental quantization rule
$$\{\dot{},\dot{}\}_{P.B.} \mapsto \frac{1}{\mathrm{i}\hbar}[\dot{},\dot{}]$$
for operators. In other words, to set up the basic principles of the theory, at a fundamental level it ultimately reduces to invoking this classical limit, which is why people take so seriously the existence of a classical limit and stress that (at least Copenhagen) quantum mechanics does not exist without this limit.
To fix the form of $\hat{H}$ we note, e.g. if we assume that we are dealing with a non-relativistic quantum system, then on taking the quasi-classical limit of $\Psi(q,t)$ we see the action $S$ in the quasi-classical limit should reduce to the non-relativistic classical Hamiltonian.
This is the fundamental idea behind why an approach such as the Schrodinger functional picture will apply in quantum field theory, we instead just assume the Hamiltonian obeys relativistic invariance principles (and something else important which I'll leave as a mystery) and depending on the fields involved in the action/Hamiltonian we then just apply the above thinking to get the Schrodinger functional equation and solve it (easier said than done).
If one thinks about it, they will notice some of this looks similar to the path integral approach (e.g. the quasi-classical limit). Indeed the above is constructed on the basis of assuming 'no paths' existing. What do we do when setting up path integrals? We instead assume all possible paths affect our results. Reference [2] makes explicit how assuming the existence of $e^{iS/\hbar}$ (with $S$ evaluated on all possible paths between two points) is a postulate in setting up quantum mechanics from a path integral perspective.
This is the logically consistent (within the principles of quantum mechanics) way to justify those rules of sending classical quantities to their operator analogues. It's completely inconsistent to just take a classical equation and just insert the operator forms without at least going through something equivalent to this dance, but with the above proviso's it's as consistent as one can expect. It's of course why books claim the Schrodinger equation can be derived from a path integral perspective, i.e. within that logical framework books like [2] say it can be derived, so of course it can also be derived within the canonical Copenhagen framework also.
References:

Landau and Lifshitz, "Quantum Mechanics", 3rd Ed.
Kaku, "Quantum Field Theory: A Modern Introduction", Ch. 8.
Dirac, "Principles of Quantum Mechanics", 4th Ed.


Systemd vs init.d Specify systemd dependencies in LSB headers

Question: I have a systemd based system which contains one System V style init.d script with LSB headers. The init.d script must only be started after all filesystems in fstab are mounted. How can I specify this type of dependency in the LSB headers?
Comment: Yes, sure: setup logic goes to your script, dependency logic goes to unit file - separation of concerns as an added bonus.
Comment: Write a systemd unit file for the old service? This should take about five minutes.
Comment: Hmm, my init.d script contains more setup logic than will fit into a unit file. That means that I'd have to create a unit file that would start my init.d script as a forking service?
Answer: When you say you check for a file system mount, it's clear you'll run it if found and mounted, but what's not clear is that if it's not found, what do you want it to do?
I ask because one possible answer is yes, in LSB, it is the run level that determines this. In Linux/Unix boot up, file systems are available at run level 1. So, set in your LSB header "Default-Start: 2 3 4 5". Then, put the filesystem mount entry in /etc/fstab and optionally set it to 'bootwait' to hang your system and prevent transitioning to run level two until it mounts, no matter how long it takes.  This is actually how some systems are configured when the (remote) file system is super critical.
Otherwise, the answer is no, you cannot check the existence of a mounted partition ONLY within an LSB header entry itself. And, since you allow this particular system to boot without this particular file system, the file system is obviously not 'important' enough to hang the system awaiting mount availability.
A consideration is if you want it to not run at all when the file system is not mounted because you're trying to satisfy a "Required-Start:" dependency in another init script?  Hopefully not as you can see you are sliding down a very slippery slope of init script dependencies.
Hopefully, you want it to not run because if it did, it messes stuff up like filling the root file system (as opposed to being an init dependency)? Then, you can let it run, but just code the init script to check and exit gracefully, as appropriate.  The logic to check a file system mount and, if not found, exit, is probably one line of code. It can be inserted just after of the LSB header.


Having issues changing this assembly into C

Question: ...
.L5:
 movl 8(%ebp), %eax #  A(param) is at ebp + 8
 movl 12(%ebp), %edx # B(param) is at ebp + 12
 movl %edx, %ecx
 subl %eax, %ecx
 movl %ecx, %eax
 addl $1, %eax
 movl %eax, %edx
 shrl $31, %edx
 leal (%edx,%eax), %eax
 sarl %eax
 addl %eax, 8(%ebp)
 movl 8(%ebp), %eax
 cmpl 12(%ebp), %eax
 jge .L5

I'm having issues reverse engineering this x86 and what I'm thinking right now is that it's a do-while but the expression that I am getting doesn't seem right and without using variables it is very long. I think at the shrl $31, %edx, the register edx should equal ((B-A)+1)>>31? Am I way off?
Answer: No, that's correct.


Stabilizers and Quotients of a Nilpotent Lie Algebra

Question: Let $\frak{g}$ be a finite-dimensional complex nilpotent Lie algebra. What conditions on $\xi\in\frak{g}$ ensure the existence of a (canonical or non-canonical) surjective morphism $\frak{g}\rightarrow\frak{g}_{\xi}$ of complex Lie algebras? Also, what are some of the known decomposition theorems for finite-dimensional complex nilpotent Lie algebras?
Comment: What do you mean by $\mathfrak{g}_\xi$?
Comment: I am referring to the stabilizer of $\xi$ under the adjoint represenation of $\frak{g}$. (ie. $\frak{g}_{\xi}=\{\eta\in\frak{g}:[\eta,\xi]=0\}$.
Comment: With no motivation, the first question seems a little senseless to me. For the second question, there is no general decomposition result in the spirit of the decompositions of semisimple Lie algebras, or the decomposition of an arbitrary finite-dimensional Lie algebra (in char 0) as a semidirect product of a semisimple by a solvable one.
Comment: Theorem 1.13.19 in Dixmier's "Enveloping algebras" is a decomposition theorem for _representations_ of finite dimensional complex nilpotent Lie algebras. It states that every finite dimensional such representation is uniquely the direct sum of its generalised eigenspaces.
Comment: You can consider that if $\phi:\mathfrak{g}\to\mathfrak{g}_xi$ is an epimorphism of Lie algebras. Then $\mathfrak{g}\mid ker(\phi)\equiv \mathfrak{g}_\xi$.

In conclusion $\mathfrak{g}\equiv \mathfrak{g}_\xi\oplus ker(\phi)$ as a vector space, where $ker(\phi)$ is an ideal and $\mathfrak{g}_\xi$ is a subalgebra.

Is that the kind of decompositions you are looking for? As a direct sum of an ideal and a subalgebra?
Answer: A well-known decomposition theorem for nilpotent Lie algebras is the weight space
decomposition, studied by R. Carles, L. J. Santharoubane and others (in the nilpotent case).
Here $L$ is a finite-dimensional nilpotent
Lie algebra over an algebraically closed field of characteristic zero, and $Der(L)$ its
derivation algebra, $T$ a torus on $L$, that is, a commutative subalgebra
of $Der(L)$ consisting of semisimple endomorphisms. This defines naturally a linear representation of $L$, and the elements of $T$ can be diagonalized simultaneously. Therefore
we have the decomposition
$$
L=\bigoplus_{\alpha \in T^{*}}L^{\alpha},
$$
where $L^{\alpha}$ is the set of $x\in L$ with $t(x)=\alpha (t)x$ for all $t\in T$. The equivalence class of a weight system is an invariant of $L$. It has been studied intensively for nilpotent Lie algebras, e.g., for the classification in low dimensions, and for
the cohomology of nilpotent Lie algebras.


Do Java primitives go on the Stack or the Heap?

Question: I just know that the non-primitives (the objects) go on the heap, and methods go on the stack, but what about the primitive variables?
--update
Based on the answers, I could say the heap can have a new stack and heap for a given object? Given that the object will have primitive and reference variables..?
Comment: Each thread has its own stack, not each object. If there is only one thread, then there´s only one stack.
Answer: Primitives defined locally would be on the stack.  However if a primitive were defined as part of an instance of an object, that primitive would be on the heap.
public class Test {
    private static class HeapClass {
        public int y; // When an instance of HeapClass is allocated, this will be on the heap.
    }
    public static void main(String[] args) {
        int x=1; // This is on the stack.
    }
}

With regards to the update:
Objects do not have their own stack.  In my example, int y would actually be part of each instance of HeapClass.  Whenever an instance of HeapClass is allocated (e.g. new Test.HeapClass()), all member variables of HeapClass are added to the heap.  Thus, since instances of HeapClass are being allocated on the heap, int y would be on the heap as part of an instance of HeapClass.
However, all primitive variables declared in the body of any method would be on the stack.
As you can see in the above example, int x is on the stack because it is declared in a method body--not as a member of a class.
Comment: If you think about it that makes perfect sense since the lexical scope of the local variable will determine when it needs to be deleted off the stack. :-). When an object is ready for garbage time the whole thing must go. I always have trouble with static members of a class though. I cant remember where these values go. Let's say if `int y` was a static member :-P
Answer: All local variables (including method arguments) go on the stack; objects and all their fields are stored in the heap. Variables are always primitives or references to objects.
Java implementations may actually store objects on the heap in such a way that it still complies with the specification. Similarly local variables may be stored in registers or become indistinct through optimisation.
Answer: primitives can be found in both places.
class Foo
{
   public int x;
   public static void Main()
   {
      int y = 3; // y is on the stack
      Foo f = new Foo();  // f.x is probably on the heap
   }
}

except you shouldn't really care unless you're building a JVM. A really clever optimizer might decide that since the Foo that f points to never escapes Main, and is never passed to another function that it is safe to allocate it on the stack.
With regards to the update:
The stack and the heap aren't distinguished by what is stored in them, but rather the operations provided for them. The stack allows you to allocate a piece of memory in a LIFO fashion, you can't deallocate a piece until all the pieces younger than it have also been deallocated. This conveniently aligns with how a call stack is used. You can put anything on the stack as long as it is ok for that thing to go away when your function returns. This is an optimization, as it is very quick to allocate and deallocate from a stack since it only supports being used in this manner. One could store all the local variables for a function on the heap in an implementation if one wanted to. The heap is more flexible, and consequently more expensive to use. It would not be accurate to say that an object has a stack and a heap, as I said, what distinguishes the stack from the heap is not what is in it, but the available operations.
Comment: @Logan: `// f.x is probably on the heap` -> Are you saying it depends on JVM implementation?
Answer: Primitive values are allocated on the stack unless they are fields of an object, in which case they go on the heap. The stack is used for evaluation and execution, so no it doesn't make sense to say that objects with primitive fields have a stack—it is still considered to be part of the heap. Even Stack objects are allocated on the heap.
Comment: when I pass primitive from one object to another object's method, local variables always goes to stack right? (even the method is static or not)


worker showing offline in nanopool my rig shows it is hashing. Need help please!

Question: I am using nanopool and it did work after initial start up. I rebooted the rig and now my worker is offline. I am hashing and connected ot pool1 us-pool pool2 eu-pool.
Has anyone else had this problem?
Please help and thank you in advance for any assistenace.
Answer: Same here. Glad to hear I'm not alone but still curious what is going on. Are you using SMOS or Windows for mining? I have a rig on Windows and rig on SMOS and both show offline but are hashing fine.
I do have a rig that has been running for weeks just fine and today it shows it has rebooted 5 times. I wonder if nanopool is having issues or something.
Answer: Same problem here. After running nicely for days (ubuntu) now my worker it show as offline on nanopool. However n. of shares found is increasing (hashing as normal) but last share submitted it shows as hours ago. Could be that there is problem with the pool or possibly anything else?
Answer: ive had the same problem for the past 5 hours. was worried it ws only me but i suppose others are having the same problem. could be a nanopool back end issue.
hope they resolve it soon for us. nanopool has been acting up a bit in recent time and i hope its not something permanent with them as theyve been quite good.


How does confusion affect enemy AI?

Question: So, there are a few weapons in Terraria that give the enemies confusion. if the player gets confusion, then their controls are inverted. I'm wondering how the confusion works for the enemy's AI since they don't have button controls.
Answer: As per the Terraria wiki, the Confused status effect causes mobs to move away from the player instead of towards them, and mobs with ranged attacks will be unable to shoot those attacks.  Note that a vast majority (about 70%) of mobs are immune to the Confused debuff - it's mostly mobs that have either the slime or fighter AI that are affected.
Answer: Confusion causes Enemy AI's to move away from the player and mobs with ranged attacks cannot shoot them at you. Intrestingly, 70% of mobs are immune to the confused debuff, and including all the bosses. Some of the other mobs that actually apply this to the player are Giant bat, Light Mummy, Clown, Brain Scrambler, Brain of Cthulu and Creeper.

How does confusion affect enemy AI?


Generics and System.Collections

Question: After moving to .NET 2.0+ is there ever a reason to still use the systems.Collections namespace (besides maintaining legacy code)?  Should the generics namespace always be used instead?
Answer: For the most part, the generic collections will perform faster than the non-generic counterpart and give you the benefit of having a strongly-typed collection. Comparing the collections available in System.Collections and System.Collections.Generic, you get the following "migration":

    Non-Generic               Generic Equivalent
    ------------------------------------------------------------
    ArrayList                 List<T>
    BitArray                  N/A
    CaseInsensitiveComparer   N/A
    CollectionBase            Collection<T>
    Comparer                  Comparer<T>
    DictionaryBase            Dictionary<TKey,TValue>
    Hashtable                 Dictionary<TKey,TValue>
    Queue                     Queue<T>
    ReadOnlyCollectionBase    ReadOnlyCollection<T>
    SortedList                SortedList<TKey,TValue>
    Stack                     Stack<T>

    DictionaryEntry           KeyValuePair<TKey,TValue>

    ICollection               N/A (use IEnumerable<T> or anything that extends it)
    IComparer                 IComparer<T>
    IDictionary               IDictionary<TKey,TValue>
    IEnumerable               IEnumerable<T>
    IEnumerator               IEnumerator<T>
    IEqualityComparer         IEqualityComparer<T>
    IList                     IList<T>

ICollection is immutable (no members to change the contents of the collection) while ICollection<T> is mutable. This makes the interfaces similar in name only while ICollection and IEnumerable<T> differ by very little.
From this list, the only non-generic classes that don't have a generic counterpart are BitArray and CaseInsensitiveComparer.
Comment: KeyedCollection is an appropriate alternative to DictionaryBase, depending on the data you were storing.
Answer: In some circumstances the generic containers perform better than the old ones. They should at least perform as well as the old ones in all circumstances. And they help to catch programming errors. It's a rare combination of a more helpful abstraction and better performance, so there isn't much of a reason to avoid them. Only if you are forced to by a crummy library you have to work with that was written before generics.
Answer: I saw an interview with Anders Hejlsberg from the c# team and he was asked if there was anything he regretted with the previous releases of .net. Not having generics in asp.net 1.0 was the first thing he mentioned. Not having it there meant they had to implement workarounds that would stick with the .net libraries and soon become legacy code.
I never use the System.Collections namespace and from his statement this seem to be the correct path.
Answer: About the only bad thing I can think of when using Generics is variance, so for example, if you have a List<Person> and you want to pass it to a method that takes List<object> you can't because List<Person> cannot be cast to List<object> directly.
This problem is solved in .NET 4.0.

Generics and System.Collections


Is this a wet liner (wet sleeve) engine?

Question: This is Porsche 911 (996) Carrera engine. It seems to me that liners can be removed from the block. I think this is a wet liner engine. Am I right?
Comment: If it is, then I would expect to see metal bars to hold the liners in place just in case the crank gets rotated. The friction of the rings can cause the liners to shift but it depends on how the liners are fitted. Some engines the liners are designed to be easily removed , 6 litre diesels come to mind.
Comment: The space around the cylinders is the water jacket but this engine does no appear to have removable cylinder sleeves.
Answer: In an effort to reduce costs during production, Porsche utilized a type of insert-mold casting process to directly incorporate Lokasil cylinder liners into the case. While this is a neat way to reduce the total number of parts used in the engine, this design basically casts a wearable part into the engine case. There is no factory replacement for the liners, when they wear, the factory expects you to buy a new engine case. In addition, the design of the cylinder liners allows them to "float" within an area filled with coolant.
Excess vibration and twisting from the normal operation of the engine appears to be causing some cracking in these liners, resulting in a small chunk of the liner breaking off. When this failure happens, you will see oil and coolant begin to mix together or a slight unexplained coolant loss.
If your engine experiences this failure, it can be rebuilt using LN Engineering's Nickasil liners installed. They take your old case, machine out the cracked or damaged Lokasil liners, and install an aluminum Nickies insert, which is stronger and more reliable than the factory cast-in liner.
Porsche parts and info
Comment: @NarimanAsgharian Yes some engines have the sleeves cast in, some don't.
Comment: So, all the liners which are available for sale for this engine are aftermarket parts. I thought the liners are removable. Based on your explanation, Porsche doesn't offer genuine spare parts (liners) but in a real wet liner engine (like some diesel engines), you can replace the liners with OEM parts supplied by the factory.


drop down list in rails

Question: I have something basic generated from nifty_scaffold in a partial form: _form.html.erb

    <%= f.label :group_id %>

    <%= f.text_field :group_id %>

Instead of a text field I want to convert the above from text_field to a drop down list which will be populated with groups which I set below.
My new action in Employee controller looks like this:
  def new
    @employee = Employee.new
    @groups = Group.all
  end

How do I make a drop down list where it will be populated with all groups in @groups variable
Additionally, how will edit action work? there I will want the assigned group to be preselected.  Since I am using a partial form, same form will be used in edit as well.
Answer: <%=  select("employee", "group_id", Group.all.collect {|p| [ p.name, p.id ] }, { :include_blank => true })%>

works!

drop down list in rails


SharePoint Security issue

Question: I work for an IT company and I have come to know that besides all the security provided by SharePoint there is still document insecurity.  Like when I send someone an attachment in the network (in the company) there is no guarantee that the recipient will not misuse it by sending it outside the network. So if someone has any suggestions for me I'd appreciate it.
Comment: If the document is sent as an attachment on an email, SharePoint security is irrelevant. You may be able to limit what can be done with an individual document via IRM, but if a person has read access to a document in SharePoint, there's not a lot you can do to keep them from using the content inappropriately.
Answer: Nothing is 100% secure, Because if a person who has right permission and he decided to misue of the documents than he can. But their are couple of things to try.

Information Rights Management (IRM) is a persistent file-level technology from Microsoft. It uses permissions and authorization to help prevent sensitive information from being printed, forwarded, or copied by unauthorized people. After permission for a document or message is restricted by using this technology, the usage restrictions travel with the document or email message as part of the contents of the file. read complete details here
Enable the Auditing on your site, By that you will know about who is view/editing /downloading the docuemnts. Configure audit settings for a site collection and View audit log reports
Never send a document in attachment rather send the link to the document.
Answer: Eric - Omegacron is correct. Once a document is attached to an email it is out of your SharePoint installation forever. The best practice is to teach your users how to share links instead of documents. Some more extreme companies have accomplished this by disabling email attachments period. But in practice most companies will not do this and most people will keep on hitting "attach" instead of "link". A co-worker wrote a blog about this awhile back: http://info.easydynamics.com/blog/secure-messaging-storage-audit-trail
Answer: SharePoint is a content management platform. It enables storing document and data in multiple forms. It comes with security at various levels from

Web application
Site collection
Site
Library
Item level

Normally a user is give permissions based on his Role in the organization. If a user can misuse the role then there won't be any control (and its not the job of SharePoint).
To tackle things happening around the network you should use tool/setting available in your network devices. You can set rules in email servers to track down who is sending email to outside network. Or you can disable attachments sending to outside network

SharePoint Security issue


Viapath as a maximum flow problem

Question: Let $G = (V, E)$ be a graph and $a$, $b$, $x$ $\in V \ $  different vertices. I have seen stated that the problem of finding a simple path from $a$ to $b$ passing through $x$ can be formulated as a maximum flow problem. Does anyone
have a pointer for this ? Also any references to problems that can be reformulated, in not completely obvious ways, to maximum flow problems would be valued.
Comment: This smells like homework.  (I have assigned this problem as homework.)
Comment: Could you be more specific about your *simple path* definition ?
Comment: It is a path where no vertex appears twice
Answer: The reduction is the following.
Consider a graph as directed (that means, each edge $(u,v)$ is actually a pair of arcs pointing in the opposite directions). Split each vertex $v \in V$ to 2 different vertices $v_{in}$ and $v_{out}$ and fix the adjacent edges such that all in-arcs ends in $v_{in}$ and all out-arcs starts from $v_{out}$. Also add an arc $(v_{in}, v_{out})$ of unit capacity for each such pair. This guarantees that the flow will not pass through any original vertex more than once (which is necessary for path simplicity).
Then add a sink $t$ and arcs $(a_{out},t)$ and $(b_{out}, t)$ of unit capacity. Find a maximum flow from $x_{out}$ to $t$. If a simple path $a-x-b$ exists in the original graph, the parts $x-a$ and $x-b$ will not share a vertex (except from $x$) and the maximum flow will have size 2 so you will be able to reconstruct the answer from the flow. The opposite is true, too.
Upd.: The aforesaid was written under assumption that "simple path" means "a path with no repeated vertices". In other case there is no need for vertices splitting.
Comment: Your assumption is correct. Thank you for the demonstration. Quick thinking !
Comment: Do you mean from $x_{in}$ to $t$, to avoid going through $x$ twice?
Comment: @GeoffreyIrving Just make sure $(x_{in}, x_{out})$ capacity is 2. Well, basically it's the same thing (you can cancel loops through $x$ easily). I guess I was thinking about _shortest path_ problem, where loops are impossible so it doesn't matter.
Answer: Try following approach for grapf $G=(V,E)$:

Create new sink $V'=V \cup t$, and edges $E' = E \cup \{ (a,t), (b,t) \}$.
make all edges' capacity 1 unit (I assume discrete maxflow).
Make $x$ your source.
Maximal flow $F$ will be 2 iff path from $a$ to $b$ exist. Searched path will be:$P = F - { (a,t),(b,t) }$

To be specific, for simplicity of description, above approach assumes bidirectional graph.
For directed graph it applies as well, after adjusting a little bit.

Viapath as a maximum flow problem


Literature recommendations for the relationship between fundamental physics and pure mathematics?

Question: So, I have been reading up on the works by Kenneth Wilson, mainly his 3 statements that he concluded to be true about our universe. His first: 'There exists a hierarchy to our universe'. From this i understand that he means to convey that the different disciplines are built upon the former. I.e, chemistry upon physics and physics upon maths. If i have understood this right then Ken is conveying that, fundamentally, physics is built upon some fundamental mathematics and that's why everything is the way that it is. I.e, the reason there are a certain number of fundamental forces can be boiled down to some pure mathematics that is just 'true' (Like 2 + 2 = 4, albeit more complicated). Is there a branch of physics that deals specifically with trying to find how these fundamental physical laws link to pure mathematical concepts? I have been struggling to find textbooks on the topic, providing my thought process is correct.
Comment: I am even more sure now that Tong was using an analogy, and was actually talking about physical scales (which is the central idea of Wilson's work).  See e.g. the section "Nature is Organized by Scale" which begins on the bottom of page 3 of [these notes](https://www.damtp.cam.ac.uk/user/tong/sft.html), in which he does the same thing.
Comment: Do you have some context for that quote? Given Wilson's landmark work with the renormalization group, in the absence of more information I would interpret "there exists a hierarchy to our universe" to be a reference to physical length/energy scales.
Comment: I was watching a colloquia given by David Tong on the work of Wilson. It was an admittedly brief segment. I think that this is definitely the main context behind this quote, however I think to avoid confusion in the audience he simplified the idea.
Answer: If I recall correctly Feynman's The Character of Physical Law (1965) covered this pretty well. The format is accessible to laypeople.
To answer your implied question about the nature of the relationship between math and physics:
No mathematical statement describes the world. Math is the process of starting with a set of arbitrary axioms and determining other statements that must be true given those axioms, and there is neither a need to test those statements against observation nor the possibility of ever doing so. Put another way, math is the study of the nature of internally consistent systems.
The universe is, as far as anybody can tell, internally consistent. Therefore, if we can make up some arbitrary axioms that seem to match all of our observations, we can apply what we know about the nature of internally consistent things (math) and determine other things that would be true if our axioms were right. By refining our axioms, we can start making really good predictions. By making really good observations, we can start guessing really good axioms. That process is physics.
So, physics (and the other sciences) are not built on a foundation of mathematics, but a foundation of observations. However, all studies of internally consistent things are inherently mathematical because mathematics is the study of internal consistency itself.


Do I need to use a waterproof sealant in my entryway?

Question: Do I need to use a waterproof sealant in my entryway, over the backerboard and under the porcelain tile or is the thinset adequate?  Sub floor is above grade, 3/4" plywood. thanks!
Answer: You will only need the backer board for your entry way. No other waterproofing needed. If it was your bathroom at the shower entrance, it would be a good idea, but since it will only be an occasional wetting from tracking in water or snow, and porcelain tile is impervious, it will handle it very well. You should seal your grout joints too, so the color remains, not getting changed by conditions over time
Comment: Grout sealer is your friend.


Is "The Infinite" a metaphor that can refer to God?

Question: First, I am not entirely sure this can be viewed as a metaphor, but I feel this is an edge case, because like the "Absolute" it refers to a quality of God, but "Infinite" even if it's a quality of God, sounds sketchy at best. What do you think?
Comment: A better term is "Metonymy".
Comment: Can you provide more context? Is this a novel? A self-help book? Who's referring to "The Infinite"?
Comment: Look up Eyn Sof.
Comment: I'm voting to close this question as off-topic because it may be more suited for another Stackexchange site
Comment: I honestly don't know why it was migrated. It should be on the writing stackexchange.
Answer: I'll play devil's advocate and Answer: Yes. In terms of making the metaphor (or any metaphor) 'work' your success depends on your novel as a whole.
Imagine a novel written in the style of Hemingway--very sparse, concrete language. Few metaphors. In such a case, dropping 'The Infinite" into the narrative, in the absence of any clues or what have you, will not conjure a picture of God.
But. Imagine a novel about an alien culture--alien in many, many ways. Perhaps they communicate through dance, and have tails, and never sleep. Who knows, but they are clearly not at all like humans. They refer to love as 'The Communion." They refer to food as "The Power." To their jobs as "The Definition" or "The Hours of the Day." And to God as "The Infinite."
Two extreme cases here--hopefully illustrating the point that 'The Infinite' can refer to God in a fairly obvious manner depending on your execution. Imagine one being says to another, "Pray to the Infinite." We get the meaning easily, especially if we already know that something like "The Red" means an injury.
Your novel may be in between these two extremes, but as with so many of these issues, it depends on your execution.
Answer: Speaking as a scientist, "Infinite" wouldn't mean "God" to me, it would refer to an opinion about time and/or space; which some people consider infinite.
Answer: "Absolute" is not a common reference to god, at least not to most people.  I have heard of "absolute being" as a reference to god...
"infinite" never heard of that word used that way either...
https://www.thesaurus.com/browse/god
There are a lot of words for god...
BUT this being a metaphor.. you can do whatever you want, if you can make it work, it works.
but if you want to say "I saw infinite last night in my dreams" out of the blue, you should have no real expectation to be understood.
Answer: I'll give a second vote to yes as in many Judeo-Christian beliefs, God is described as Omnipotent (All Powerful), Omniscient (All Knowing), and Omnipresent (Present in all possible spaces).  The idea of god as a being of such magnitude has inspired tropes of having any working knowledge of God is just a little too much knowledge for humans to handle, with works like Kevin Smith's "Dogma" God is so powerful that he tasks a specific Angel to speak God's divine message to humans, least their heads explode ("We went through several "Adams" before we worked that out).  In the tv series "Joan of Arcadia" Joan passes out after learning a small part of God's Plan and comes to several hours later, having forgotten any element of it.  Even in the Bible, Mose asks if he can see God and God responds that Moses cannot see God's face, but lets him gaze upon God's back, which causes Moses' face to glow brightly from the energy for several days, forcing Moses to wear a cloth over his face until it stops glowing, and the Angels' catchphrase greeting "Fear Not, For I am an Angel of the Lord" was due to the fact that the Angels weren't humans with wings (this occured after Christianty made its way to Rome) but rather bizarre and almost Lovecraftian images that shocked most people they appeared before.
Referring to God as all things is common in rituals and God's capacity for forgiveness in Christianity is often described as infinite.  It's certainly appropriate and can even be a bit subtle to refer to God as the Infinite by an alien race that is perhaps more math oriented in their language as opposed to humans which are more metaphoric.
Comment: and yet..  the  Judeo-Christian-islam god has  a lot enemies....  who are just ordinary men.. women and children...  according to many of the self proclaimed believers.
Comment: @ashleylee:  Define enemies and list possible examples.  I think you're either misreading something.


virtual box windows 7 screen size problems

Question: I have windows 7 installed in virtual box with ubuntu 14.04. I cannot get it into full screen...there is at least 3 inches of black on both side and an inch at the top and bottom. I already installed guest additions and that had no effect. Does anyone know a fix?
Answer: This is usually caused when the VM is running at a lower resolution than the host display. If guest additions are installed I would restart the VM then try to adjust the resolution within the VM to that of the display.
Comment: I tried to change the resolution within the virtual Windows 7 and the resolution setting for the guest screen is lower than the host but there is not an option to increase it as much as the host screen. In other words the max resolution option is lower for the guest. I have no clue how to change that.


Problems about value log2(IP/Input) less than zero in ChIP-seq?

Question: I have a question about the normalization for ChIP-seq. I used CPM to normalize my bam files of each IP and Input. Then I calculate the coverage of gene bodies for all genes on the genome. I have WT and different mutants. Take H3K4me1 as an example, I first calculate the matrix over gene bodies for H3K4me1 IP and Input. Then I used the matrix of H3K4me1 divided by the matrix of input. then log2(H3K4me1/Input) and plot this value over entire gene bodies. I found all the regions are lower than 0 and it seems that my IP does not work. In fact, this is not the case since I have checked the antibody's efficiency via ChIP-qPCR and it works well. Moreover, the distribution of H3K4me1 is consistent with the published data. But the problem is the profile of log2(H3K4me1/Input) is lower than 0.  I don't know how to deal with such problems? I would appreciate it if someone can give me some suggestions on that.  Thanks!
Answer: Seconding what Devon says:

Use a proper normalization technique such as TMM from edgeR. For applying this to bigwig files you could follow my tutorial at biostars.org.

My two cents beyond that:

I would not use enrichment over input as input is hard to normalize and can be sparse depending on sequencing depth. Therefore, you might get biased enrichments that do not well reflect reality, especially if counts in IP or input (of both) are low. I personally only use input for peak calling, and for plots like this then compare IPs among each other.

So the strategy would be to first normalize your bigwigs (I guess you use these for the plots?) using the strategy as linked above, and then use these robust CPMs to make the plots without any input samples, only using the IPs. You can use MA-plots to explore whether samples are properly normalized. I made a little code suggestion here in this biostars post on how to do this starting basically from a matrix of raw counts using edgeR. This is basically (from the TMM normalization standpoint) the same as in the first link, but then doing pairwise MA-plots to see whether TMM properly centers the majority of points along y-axis being somewhat zero. Normalization is key in any experiment, therefore I always recommend to use these kinds of plots for diagnostics.
Comment: Dear ATpoint, sorry for replying to your message so late and thank you very much for your suggestions!  Yes, you are right, I first want to normalize the raw bigwig files once I get the normalized bigwig files, I will try to calcualte the matrix of genebodies for genes interested in each sample.  I will have a try to use TMM method you recommended to see whether it works on my data or not!  I will let you know once I got the results.   Thanks!
Comment: Hi, ATpoint, I read the link you sent to me. I have a question about the regions in the matrix . If I understand properly,  for example, I have 5 strains and each strain have a peak region list, you means that pooling the peak coordinates of each sample as one region list,  then use the feature count to calculate the coverage of each region for all samples, if I am wrong, please correct me. I am looking forward to your reply! Best.
Answer: This is a common issue caused by the normalization method not being particularly robust (most likely it's using the mean signal for normalization, which will be a bit upward skewed when you have regions of high enrichment in the ChIP). The basic ways to fix this are:

Use a more robust normalization (e.g., the median signal, or even just manually select un-enriched regions to calculate the normalization factors)
Manually tweak the normalization factors so the background (i.e., log2 ratio in regions where you don't expect any enrichment) is 1. That will generally make the regions you're plotting more biologically realistic.

Note that there can be a negative enrichment in some areas, due to things like exclusion of histones, so don't be overly zealous if you manually tweak the normalization factors.
Comment: Dear Devon,  Thanks for your kind reply and valuable suggestions! I have tried to use the median value of the non-enriched region to calculate the scaled factor of each IP vs their input. I found it works for all modifications but failed for H3 IP. I guess this is due to the peaks for H3 are widely spread on the genome and when I use MACS2 to do the peak calling for H3, it won't work efficiently. Do you have any suggestions for H3 IP normalization? I am looking forward to your reply.  Thanks!
Comment: You may have to manually tweak normalization values to get reasonable results.
Comment: What I did: first did peak annotation, then extracted the genes(name it as gene list 1)  with enriched peaks. Then I calculated the matrix of the gene body of genes (not included in gene list 1 ), further calculate the median value for each sample (for both IP and input).  In the peak annotation step, I tried Homer and Rgmatch, unfortunately, the results of these two software are quite different.
Comment: Therefore, I recalculated the median value given the coverage of the non-peak region (just escaped the peak annotation step). It seems the results of using Homer were much closer to the results directly given the scaled factor for the non-peaks region.  I checked the default parameter of Homer, they defined TSS as from -1kb to +100bp and TES as -100bp to +1kb. Since S.cerevisiae. genome is not big like human or mouse, I am not sure how much effect using the default parameter.   I am pretty interested in the 500bp up-and downstream of gene bodies.
Comment: The Homer source code is based on perl and I haven't used Perl before.  Do you have any other suggestions? Thanks!


Topics in Differential geometry $\cap$ Algebraic geometry

Question: I find (both: differential and algebraic) geometry fascinating. I'm just beginning my graduate studies, but I'd like to know some topics/theorems in the intersection of these two (since I don't really know where to go).
Comment: I am the one who asked this question. I don't know how it happened, but I lost my account. I would like to thank the people that took some time to answer, I really appreciate it. (Please move this to a comment.)
Comment: Love the title!
Comment: Another interesting topic is the study of the Kahler-Einstein metric and K-stability. You can have a look at the https://www.youtube.com/watch?v=v0NxDUKQVoE
Answer: 1) One very nice topic is Hodge theory.
It allows you to decompose the De Rham cohomology vector spaces of a smooth projective complex variety  into a direct sum of vector spaces having an algebro-geometric interpratation: $$ H^r(X,\mathbb C)=\oplus_{p+q=r}   H^q(X,\Omega^p)                    $$ where $\Omega^p$ is the bundle of algebraic differential $p$-forms.
2) A more restricted topic is Kodaira's embedding theorem which tells you when a compact complex manifold $X$  is projective algebraic in terms of a differential 2-form associated to a so-called Kähler metric on $X$.
3) Finally (for this post!) there are powerful "vanishing theorems" which tell you that certain line bundles on projective smooth complex varieties  have zero  positive dimensional cohomology as soon as you can put a metric on these bundles with certain "positivity" properties.
The best known of these results is the Kodaira-Nakano theorem: for a positive line bundle $L$ on the smooth complex projective $n$-dimensional variety $X$ we have $$H^i(X,\Omega^j\otimes L)=0 \;\text{for} \;i+j\gt n$$
Griffiths-Harris's book is a very rich reference for all of these topics and many more which are at the intersection you require.
Answer: One topic that lies right at the intersection of these two fields is the study of the Gauss map of subvarieties in projective space.
An $k$-dimensional manifold embedded in Euclidean space $\mathbf R^n$ has a Gauss map to the Grassmannian $Gr(k,n)$, taking $p$ to the tangent space at $p$. Gauss studied this map for surfaces in $\mathbf R^3$, and this is one of the basic objects of interest in the differential geometry of surfaces.
To see examples of how this can be applied to algebraic geometry, you could look at the long paper of Griffiths–Harris studying the Gauss map of smooth subvarieties of projective space. I don't know the paper well enough to say much about it, but in summary they use this differential-geometric approach to obtain a number of classification results about subvarieties in "special position" in projective space. For example, they prove that a nondegenerate smooth surface in $\mathbf P^5$ with degenerate secant variety but nondegenerate tangent variety must be the Veronese surface.
Answer: I would suggest that you have a look at Serre's paper GAGA. In this paper, he constructs an analytification functor from the category of complex algebraic varieties to the category of complex analytic spaces. It preserves the underlying point-set of a variety, but changes the topology from the Zariski-topology to a classical analytic one. If the variety happens to be smooth, its analytification is a complex manifold.
This functor is implicitly used when differential geometers talk about smooth varieties. They really mean to refer to the associated complex manifolds. It is often fruitful to study manifolds coming from varieties in this way, as they tend to be a bit more tame than arbitrary manifolds.
Answer: In a completely different direction, you might wanna look at Arakelov Geometry.
Arakelov geometry is an area of geometry that is motly interrested in Diophantine problems. The idea behind it is that in order to grasp arithmetic properties, it won't be sufficient to only study the algebraic geometry of the objects over $\text{Spec } \mathbb Z$, because $\text{Spec } \mathbb Z$ is not proper (compact if you prefer), so we have to compactify the situation, and to do so, we need to add an "analytic fiber" (which is a smooth complex manifold), and endow it with hermitian data.
We then have to find analogues of statements and theorems in classical algebraic geometry in this new algebro-hermitian setting, which involves of course algebraic geometry on the variety itself, and complex differential geometry on the "fiber at infinity", and to treat them in an unified setting.
This relates to deep differential concepts and question, such as index theory for instance, or questions related to the curvature on specific bundles (e.g the quillen curvature on the determinant of cohomology) and of course analytic torsion.
Maybe to give you a taste of the mixing of the two styles of geometry, let me state an example. On a riemann surface, classical algebraic geometry tells you that is a good idea to study cycles over it. In the arakelov context you study so called "arithmetic cycles", which consists of pairs $(Z,g)$ where Z is a cycle on the surface (the geometric part), and g is a so called green function for $Z$, which you might think of as a potential for the electric field generated by the uniformly charged distribution of charges supported by $Z$ (you put charges at every point of $Z$ and you $g$ is then a potential for the electric field generated).
You can take a look at these notes, or those ones, by Soulé, but they're in French
Comment: Oh, and a nice way to see Arakelov theory is as a generalisation of intersection theory, so it includes three areas of your title, algebraic geometry, differential geometry and $\cap$. =)
Comment: Could you explain why   $\text{Spec } \mathbb Z$ is not compact? By the way, proper is a relative concept and it doesn't make sense to say that $\text{Spec } \mathbb Z$ is not proper.
Comment: It's a manner of speaking. The long-discovered analogy is that Spec Z should be the analog of a proper curve over a field k. In the context of arakelov geometry, Spec Z is the analog of an affine open subset in a "proper" object.
A basic observation why it should be so is the product formula which is considered the analog of the fact that the degree of a principal divisor is 0 on a proper variety. The arithmetic degree of a principal divisor is 0 if you add up the "infinite fiber". This is an indication that this should be considered as the compact object, and Spec Z is an open subset if it.
Comment: More generally, if you want reasonnable analogs of classical resulats in projective geometry to work (Riemann-Roch, degree formula etc...) you need to add up that infinite fiber that compactify $\text{Spec }\mathbb{Z}$. It's a purely "interpretative" statement. To my knowledge, nobody has ever created a theory where one actually endows $X\coprod X(\mathbb C)$ with a topology, and develop a theory on this space.
Comment: Thanks for your comments, @A.Rodriguez.


How to transfer weight paint from bone to symetrical bone

Question: I have my dog rig generated by rigify. Did the additional weight painting and forgot to turn on symmetry. Now I have left side properly weight painted, and right one isn't. Tried to symmetrize painting by removing one side and then symmetry in edit mode, didn't work. Tried with vertex groups by copying proper one, then Mirror Vertex Group, but it always shows some vertices are mirrored, other failed. Topology is symmetrical and bones are symmetrical too. What is best solution? On first two pictures you see left side which is properly painted, and right one which is not. Also, on third there are middle bones (spine) however, it isn't symmetrical to sides.
Comment: You could duplicate the vertex group, mirror it, change its name?
Comment: Take the left one bone, in the Object Data properties click that small arrow right of "Vertex Groups" then copy Vertex Group, then you have _copy of that bone. Then, again click on arrow and press mirror vertex group and that is it. However, for me it doesnt mirror them exactly as I would want it.
Answer: Best way I found is that you just remove half of the mesh which is not painted. And then use Mirror modifier and apply it. You have same mesh and weights are mirrored.


Thread vs async execution. What's different?

Question: I believed any kind of asynchronous execution makes a thread in invisible area. But if so,

Async codes does not offer any performance gain than threaded codes.

But I can't understand why so many developers are making many features async form.
Could you explain about difference and cost of them?
Answer: The purpose of an asynchronous execution is to prevent the code calling the asynchronous method (the foreground code) from being blocked.  This allows your foreground code to go on doing useful work while the asynchronous thread is performing your requested work in the background.  Without asynchronous execution, the foreground code must wait until the background task is completed before it can continue executing.
The cost of an asynchronous execution is the same as that of any other task running on a thread.
Typically, an async result object is registered with the foreground code.  The async result object can either raise an event when the background task is completed, or the foreground code can periodically check the async result object to see if its completion flag has been set.
Answer: Concurrency does not necessarily require threads.
In Linux, for example, you can perform non-blocking syscalls. Using this type of calls, you can for instance start a number of network reads. Your code can keep track of the reads manually (using handles in a list or similar) and periodically ask the OS if new data is available on any of the connections. Internally, the OS also keeps a list of ongoing reads. Using this technique, you can thus achieve concurrency without any (extra) threads, neither in your program nor in the OS.
If you use threads and blocking IO, you would typically start one thread per read. In this scenario, the OS will instead have a list of ongoing threads, which it parks when the tread tries to read data when there is none available. Threads are resumed as data becomes available.
Having the OS switch between threads might involve slightly more overhead in the form of context switching - switching program counter and register content. But the real deal breaker is usually stack allocation per thread. This size is a couple of megabytes by default on Linux. If you have a lot of concurrency in your program, this might push you in the direction of using non-blocking calls to handle more concurrency per thread.
So it is possible to do async programming without threads. If you want to do async programming using only blocking OS-calls you need to dedicate a thread to do the blocking while you continue. But if you use non-blocking calls you can do a lot of concurrent things with just a single thread. Have a look at Node.js, which have great support for many concurrent connections while being single-threaded for most operations.
Also check out Golang, which achieve a similar effect using a sort of green threads called goroutines. Multiple goroutines run concurrently on the same OS thread and they are restrictive in stack memory, pushing the limit much further.
Answer:

Async codes does not offer any performance gain than threaded codes.

Asynchornous execution is one of the traits of multi-threaded execution, which is becoming more relevant as processors are packing in more cores.
For servers, multi-core only mildly relevant, as they are already written with concurrency in mind and will scale natrually, but multi-core is particularly relevant for desktop apps, which traditionally do only a few things concurrently - often just one foreground task with a background thread. Now, they have to be coded to do many things concurrently if they are to take advantage of the power of the multi-core cpu.
As to the performance - on single-core - the asynchornous tasks slow down the system as much as they would if run sequentially (this a simplication, but true for the most part.) So, running task A, which takes 10s and task B which takes 5s on a single core, the total time needed will be 15s, if B is run asynchronously or not. The reason is, is that as B runs, it takes away cpu resources from A - A and B compete for the same cpu.
With a multi-core machine, additional tasks run on otherwise unused cores, and so the situation is different - the additional tasks don't really consume any time - or more correctly, they don't take away time from the core running task A. So, runing tasks A and B asynchronously on multi-core will conume just 10s - not 15s as with single core. B's execution runs at the same time as A, and on a separate core, so A's execution time is unaffected.
As the number of tasks and cores increase, then the potential improvements in performance also increase. In parallel computing, exploiting parallelism to produce an improvement in performance is known as speedup.
we are already seeing 64-core cpus, and it's esimated that we will have 1024 cores commonplace in a few years. That's a potential speedup of 1024 times, compared to the single-threaded synchronous case. So, to answer your question, there clearly is a performance gain to be had by using asynchronous execution.
Answer:
I believed any kind of asynchronous execution makes a thread in invisible area.

This is your problem - this actually isn't true.
The thing is, your whole computer is actually massively asynchronous - requests to RAM, communication via a network card, accessing a HDD... those are all inherently asynchronous operations.
Modern OSes are actually built around asynchronous I/O. Even when you do a synchronous file request, for example (e.g. File.ReadAllText), the OS sends an asynchronous request. However, instead of giving control back to your code, it blocks while it waits for the response to the asynchronous request. And this is where proper asynchronous code comes in - instead of waiting for the response, you give the request a callback - a function to execute when the response comes back.
For the duration of the asynchronous request, there is no thread. The whole thing happens on a completely different level - say, the request is sent to the firmware on your NIC, and given a DMA address to fill the response. When the NIC finishes your request, it fills the memory, and signals an interrupt to the processor. The OS kernel handles the interrupt by signalling the owner application (usually an IOCP "channel") the request is done. This is still all done with no thread whatsoever - only for a short time right at the end, a thread is borrowed (in .NET this is from the IOCP thread pool) to execute the callback.
So, imagine a simple scenario. You need to send 100 simultaneous requests to a database engine. With multi-threading, you would spin up a new thread for each of those requests. That means a hundred threads, a hundread thread stacks, the cost of starting a new thread itself (starting a new thread is cheap - starting a hundred at the same time, not so much), quite a bit of resources. And those threads would just... block. Do nothing. When the response comes, the threads are awakened, one after another, and eventually disposed.
On the other hand, with asynchronous I/O, you can simply post all the requests from a single thread - and register a callback when each of those is finished. A hundred simultaneous requests will cost you just your original thread (which is free for other work as soon as the requests are posted), and a short time with threads from the thread pool when the requests are finished - in "worst" case scenario, about as many threads as you have CPU cores. Provided you don't use blocking code in the callback, of course :)
This doesn't necessarily mean that asynchronous code is automatically more efficient. If you only need a single request, and you can't do anything until you get a response, there's little point in making the request asynchronous. But most of the time, that's not your actual scenario - for example, you need to maintain a GUI in the meantime, or you need to make simultaneous requests, or your whole code is callback-based, rather than being written synchronously (a typical .NET Windows Forms application is mostly event-based).
The real benefit from asynchronous code comes from exactly that - simplified non-blocking UI code (no more "(Not Responding)" warnings from the window manager), and massively improved parallelism. If you have a web server that handles a thousand requests simultaneously, you don't want to waste 1 GiB of address space just for the completely unnecessary thread stacks (especially on a 32-bit system) - you only use threads when you have something to do.
So, in the end, asynchronous code makes UI and server code much simpler. In some cases, mostly with servers, it can also make it much more efficient. The efficiency improvements come precisely from the fact that there is no thread during the execution of the asynchronous request.
Your comment only applies to one specific kind of asynchronous code - multi-threaded parallelism. In that case, you really are wasting a thread while executing a request. However, that's not what people mean when saying "my library offers an asynchronous API" - after all, that's a 100% worthless API; you could have just called await Task.Run(TheirAPIMethod) and gotten the exact same thing.


How to track identity changes in EF Core?

Question: I have a service in my application which creates a User, and saves it to the database. Here's a method of my service:
    public async Task<UserDTO> CreateUserAsync(User newUser)
    {

        var result = (UserDTO)await _userRepository.CreateUserAsync(newUser);

        if (result != null)
        {

            await _userRepository.SaveChangesAsync();

        }

        return result;

    }

And a method from a UserRepository:
public async Task<User> CreateUserAsync(User newUser) => (await _dbContext.AddAsync(newUser)).Entity;

Here's a User class:
public class User
{

    public int UserId { get; set; }
    public string Username { get; set; }
    public string Password { get; set; }
    public string Name { get; set; }
    public string Surname { get; set; }

}

The problem is when the user is being added via service, UserId is not known yet. It has default value 0, then ef core saves it to a database, finding a proper UserId. But value returned by my methods has no UserId updated - it is still 0, and i would like to return updated value. How to achieve that in a proper way?
Answer: newUser WILL have an Id read from the database.
Your code is casting from User to UserDTO, which is unlikely to work.
Comment: Thanks, casting user to UserDTO after saving changes, solved the problem


Is it necessary to pour additives to fuel?

Question: What additives, if any, should be added and how often?
Answer: It depends on how the vehicle is used. If you are running the vehicle and burning through a tank of gas every week, fuel addatives are probably unneccisary on a regular basis. If you aren't running the vehicle often, an additive like Sta-Bil can help keep the fuel fresh while in the tank.
As far as additives go, some additives work better than others. BG products have a very good reputations, but are a little expensive (and can only be purchased at an authorized dealer which is usually a garage - not at a Walmart or such). If used correctly, Seafoam is a good product as well. It is also a lot cheaper an can be readily purchased at a Walmart or Autozone. Over time, varnish does build up inside the intake tract. These types of products can help to clean them up.
Answer: None and unneccessary would be the answer to your question. The fuel companies spend good money on their fuel formulations and pretty well cover all that is needed. There is though a vibrant after market that will try to convince you other wise. Over many years I have yet to come across an after market product that really does do what it says on the tin in way of improvement. USA and European governments exert stringent controls on fuel supplys and suppliers.


How to create proper NEXT & PREV buttons for retrieving next & prev objects in Django

Question: I would like to put NEXT & PREV links to my detail page to get easily to Previous or Next item. I dont want to do it with DATE or ID, because it is totally impractical.
I dont want to use:
Model.get_next_by_FOO(**kwargs)
Model.get_previous_by_FOO(**kwargs)

How can I setup a column with for example numbers and sort by that. If I do integers it will sort like:
1 12 13 14 2 21 24 3 etc....doesnt work either.
What would be the simplest way for the user in CMS specify order and it will sort it and then I can use the NEXT and PREV.
models.py
class Art(models.Model):
    pub_date = models.DateTimeField('date published')
    title = models.CharField(max_length=200)
    title_url = models.SlugField(max_length=200)
    image = models.ImageField(blank=True, null=True, upload_to='uploaded_images')
    graphics = models.ImageField(upload_to='images/art/',blank=True)
    def image_thumbnail(self):
        return '' % (self.graphics, self.graphics)
    image_thumbnail.allow_tags = True
    image_thumbnail.short_description = 'graphics'
    description = models.TextField(blank=True)

    def __unicode__(self):
        return self.title

views.py
def artdetail(request, art_title_url):
    art = Art.objects.all().filter(title_url=art_title_url)

    return render_to_response('nart-detail.html', {'art':art)

How can I on this example make next & prev buttons in the template to get next ot previous object and if I got to the end it will hide the next button etc....
My major concern is to make it proper way maybe by sort (number column) I am thinking wrong, I would like to know the best way how to achieve this that user is happy with easy customization.
Answer: I hope this will be of some help:
http://www.dajaxproject.com/pagination/


What can we do to encourage new members to be more welcomed?

Question: I feel like we have a reasonable supply of new members coming by, but they don't seem to stick around as much as we would like. What can we do to encourage these members to stick around for more than it takes to find an answer to a question or two?
Comment: Maybe you should put this on [chat] as well so that it gets more publicity.
Answer: I think that it would help if we got to their questions faster so they didn't have to wait around. Also I think it would help if we lowered the 20 point requirement for Chat to maybe 5 points so that the newer users could join in the conversation quicker and then stimulated more conversation in Chat.


xxx.eot web font file won't work properly on IIS 6.0 server

Question: I'm using two web fonts 'NanumGothic.eot'
and 'NanumGothic.woff', with CSS style like below.
This system is hosting web services, based on Windows Server 2003
and IIS 6.0 and local remote MS-SQL Server 2008.
<code>@font-face {
        font-family: ngweb;
        src:url('NanumGothic.woff') format('woff'); /* Modern Browsers */
}
@font-face {
        font-family: ngie;
        src:url('NanumGothic.eot'); /* IE6-IE8 */
}
</code>
.woff file works fine at NON-IE 6, IE 7, and IE 8 web browsers, even on IE 9(works fine here).
From IE 6 to IE 8, they are works with .eot font file that I know.
But, this server doesn't present web font at all(on IE 6 ~ IE 8).
My IE 8 browser security setting is enabled web font downloadable.
Other similar system's are service same .eot file in good way.
I searched on Google, about sytle CSS tag and applied them on the system, but there's no improvements.
Why do this to me MS???
Server system reboot could solve this problem? really?
Answer: I cannot say for IIS6 but in IIS7 you need to add a mime type for the font. Have you looked into this?


Altering routing of user/login

Question: I have a use case where I need to adjust the landing of the route, user/login to direct to a custom developed user/register page once clicked. What's the quickest method of doing this without overloading the core User module with changes?
Comment: hook_init function in your custom module or you cantry rules module for custom redirect of the path.
Answer: You can also use Login Destination module to redirect without touching any code

The Login Destination module allows you to customize the destination
  that a user is redirected to after logging in, registering to the site
  (7.x), using a one-time login link or logging out (7.x). The
  destination can be an internal page or an external URL. It is possible
  to specify certain conditions like referring pages or user roles and
  make the destination depend upon them. You may use PHP snippets to
  provide custom conditions and destinations. It is also possible to
  keep users on the currently visited page after logging in or out.
Answer: You could easily use Rules if you don't want to touch code
event: drupal initializes
condition: url: user/login and user/register
action: redirect to your url
Rules:

The Rules module allows site administrators to define conditionally
  executed actions based on occurring events (known as reactive or ECA
  rules). It's a replacement with more features for the trigger module
  in core and the successor of the Drupal 5 workflow-ng module.
Comment: Is it possible with Rules to define behavior based on login from specific sources? I have a ctools-based Multi-step registration and it appears that when I use Triggers to route logged in users to the new URL, it was affecting users who registered with multi-step.
Comment: so far as I know, it should pick it up, unless you are overriding the session data, you would be using core login functionality just in your own format - only takes about 6 minutes to find out if it would work...

Altering routing of user/login