Comprehensive list of activation functions in neural networks with pros/cons

Question: Are there any reference document(s) that give a comprehensive list of activation functions in neural networks along with their pros/cons (and ideally some pointers to publications where they were successful or not so successful)?
Comment: no, it makes a pretty big difference.
Comment: https://en.wikipedia.org/wiki/Activation_function is a good resource; you can use many others, including `sin(x)`, see https://openreview.net/pdf?id=Sks3zF9eg.
Comment: I don't know enough about ANNs, but unless the activation functions differ substantially in shape it will be very difficult to tell them apart. For a discussion of an analogous situation, you could see my answer here: [Difference between logit and probit models](http://stats.stackexchange.com/a/30909/7290).
Comment: For Video Tutorial on Activation Functions, Visit:
https://quickkt.com/tutorials/artificial-intelligence/deep-learning/activation-function/
Answer: I'll start making a list here of the ones I've learned so far. As @marcodena said, pros and cons are more difficult because it's mostly just heuristics learned from trying these things, but I figure at least having a list of what they are can't hurt.
First, I'll define notation explicitly so there is no confusion:
Notation
This notation is from Neilsen's book.
A Feedforward Neural Network is a many layers of neurons connected together. It takes in an input, then that input "trickles" through the network and the neural network returns an output vector.
More formally, call $a^i_j$ the activation (aka output) of the $j^{th}$ neuron in the $i^{th}$ layer, where $a^1_j$ is the $j^{th}$ element in the input vector.
Then we can relate the next layer's input to it's previous via the following relation:
$$a^i_j = \sigma\bigg(\sum\limits_k (w^i_{jk} \cdot a^{i-1}_k) + b^i_j\bigg)$$
where

$\sigma$ is the activation function,
$w^i_{jk}$ is the weight from the $k^{th}$ neuron in the $(i-1)^{th}$ layer to the $j^{th}$ neuron in the $i^{th}$ layer,
$b^i_j$ is the bias of the $j^{th}$ neuron in the $i^{th}$ layer, and
$a^i_j$ represents the activation value of the $j^{th}$ neuron in the $i^{th}$ layer.

Sometimes we write $z^i_j$ to represent $\sum\limits_k (w^i_{jk} \cdot a^{i-1}_k) + b^i_j$, in other words, the activation value of a neuron before applying the activation function.

For more concise notation we can write
$$a^i = \sigma(w^i \times a^{i-1} + b^i)$$
To use this formula to compute the output of a feedforward network for some input $I \in \mathbb{R}^n$, set $a^1 = I$, then compute $a^2, a^3, \ldots, a^m$, where $m$ is the number of layers.
Activation Functions
(in the following, we will write $\exp(x)$ instead of $e^x$ for readability)
Identity
Also known as a linear activation function.
$$a^i_j = \sigma(z^i_j) = z^i_j$$

Step
$$a^i_j = \sigma(z^i_j) = \begin{cases} 0 & \text{if } z^i_j < 0 \\ 1 & \text{if } z^i_j > 0 \end{cases}$$

Piecewise Linear
Choose some $x_{\min}$ and $x_{\max}$, which is our "range". Everything less than than this range will be 0, and everything greater than this range will be 1. Anything else is linearly-interpolated between. Formally:
$$a^i_j = \sigma(z^i_j) = \begin{cases} 0 & \text{if } z^i_j < x_{\min} \\ m z^i_j+b & \text{if } x_{\min} \leq z^i_j \leq x_{\max} \\ 1 & \text{if } z^i_j > x_{\max} \end{cases}$$
Where
$$m = \frac{1}{x_{\max}-x_{\min}}$$
and
$$b = -m x_{\min} = 1 - m x_{\max}$$

Sigmoid
$$a^i_j = \sigma(z^i_j) = \frac{1}{1+\exp(-z^i_j)}$$

Complementary log-log
$$a^i_j = \sigma(z^i_j) = 1 − \exp\!\big(−\exp(z^i_j)\big)$$

Bipolar
$$a^i_j = \sigma(z^i_j) = \begin{cases} -1 & \text{if } z^i_j < 0 \\ \ \ \ 1 & \text{if } z^i_j > 0 \end{cases}$$

Bipolar Sigmoid
$$a^i_j = \sigma(z^i_j) = \frac{1-\exp(-z^i_j)}{1+\exp(-z^i_j)}$$

Tanh
$$a^i_j = \sigma(z^i_j) = \tanh(z^i_j)$$

LeCun's Tanh
See Efficient Backprop.
$$a^i_j = \sigma(z^i_j) = 1.7159 \tanh\!\left( \frac{2}{3} z^i_j\right)$$

Scaled:

Hard Tanh
$$a^i_j = \sigma(z^i_j) = \max\!\big(-1, \min(1, z^i_j)\big)$$

Absolute
$$a^i_j = \sigma(z^i_j) = \mid z^i_j \mid$$

Rectifier
Also known as Rectified Linear Unit (ReLU), Max, or the Ramp Function.
$$a^i_j = \sigma(z^i_j) = \max(0, z^i_j)$$

Modifications of ReLU
These are some activation functions that I have been playing with that seem to have very good performance for MNIST for mysterious reasons.
$$a^i_j = \sigma(z^i_j) = \max(0, z^i_j)+\cos(z^i_j)$$

Scaled:

$$a^i_j = \sigma(z^i_j) = \max(0, z^i_j)+\sin(z^i_j)$$

Scaled:

Smooth Rectifier
Also known as Smooth Rectified Linear Unit, Smooth Max, or Soft plus
$$a^i_j = \sigma(z^i_j) = \log\!\big(1+\exp(z^i_j)\big)$$

Logit
$$a^i_j = \sigma(z^i_j) = \log\!\bigg(\frac{z^i_j}{(1 − z^i_j)}\bigg)$$

Scaled:

Probit
$$a^i_j = \sigma(z^i_j) = \sqrt{2}\,\text{erf}^{-1}(2z^i_j-1)$$.
Where $\text{erf}$ is the Error Function. It can't be described via elementary functions, but you can find ways of approximating it's inverse at that Wikipedia page and here.
Alternatively, it can be expressed as
$$a^i_j = \sigma(z^i_j) = \phi(z^i_j)$$.
Where $\phi $is the Cumulative distribution function (CDF). See here for means of approximating this.

Scaled:

Cosine
See Random Kitchen Sinks.
$$a^i_j = \sigma(z^i_j) = \cos(z^i_j)$$.

Softmax
Also known as the Normalized Exponential.
$$a^i_j = \frac{\exp(z^i_j)}{\sum\limits_k \exp(z^i_k)}$$
This one is a little weird because the output of a single neuron is dependent on the other neurons in that layer. It also does get difficult to compute, as $z^i_j$ may be a very high value, in which case $\exp(z^i_j)$ will probably overflow. Likewise, if $z^i_j$ is a very low value, it will underflow and become $0$.
To combat this, we will instead compute $\log(a^i_j)$. This gives us:
$$\log(a^i_j) = \log\left(\frac{\exp(z^i_j)}{\sum\limits_k \exp(z^i_k)}\right)$$
$$\log(a^i_j) = z^i_j - \log(\sum\limits_k \exp(z^i_k))$$
Here we need to use the log-sum-exp trick:
Let's say we are computing:
$$\log(e^2 + e^9 + e^{11} + e^{-7} + e^{-2} + e^5)$$
We will first sort our exponentials by magnitude for convenience:
$$\log(e^{11} + e^9 + e^5 + e^2 + e^{-2} + e^{-7})$$
Then, since $e^{11}$ is our highest, we multiply by $\frac{e^{-11}}{e^{-11}}$:
$$\log(\frac{e^{-11}}{e^{-11}}(e^{11} + e^9 + e^5 + e^2 + e^{-2} + e^{-7}))$$
$$\log(\frac{1}{e^{-11}}(e^{0} + e^{-2} + e^{-6} + e^{-9} + e^{-13} + e^{-18}))$$
$$\log(e^{11}(e^{0} + e^{-2} + e^{-6} + e^{-9} + e^{-13} + e^{-18}))$$
$$\log(e^{11}) + \log(e^{0} + e^{-2} + e^{-6} + e^{-9} + e^{-13} + e^{-18})$$
$$ 11 + \log(e^{0} + e^{-2} + e^{-6} + e^{-9} + e^{-13} + e^{-18})$$
We can then compute the expression on the right and take the log of it. It's okay to do this because that sum is very small with respect to $\log(e^{11})$, so any underflow to 0 wouldn't have been significant enough to make a difference anyway. Overflow can't happen in the expression on the right because we are guaranteed that after multiplying by $e^{-11}$, all the powers will be $\leq 0$.
Formally, we call $m=\max(z^i_1, z^i_2, z^i_3, ...)$. Then:
$$\log\!(\sum\limits_k \exp(z^i_k)) = m + \log(\sum\limits_k \exp(z^i_k - m))$$
Our softmax function then becomes:
$$a^i_j = \exp(\log(a^i_j))=\exp\!\left( z^i_j - m - \log(\sum\limits_k \exp(z^i_k - m))\right)$$
Also as a sidenote, the derivative of the softmax function is:
$$\frac{d \sigma(z^i_j)}{d z^i_j}=\sigma^{\prime}(z^i_j)= \sigma(z^i_j)(1 - \sigma(z^i_j))$$
Maxout
This one is also a little tricky. Essentially the idea is that we break up each neuron in our maxout layer into lots of sub-neurons, each of which have their own weights and biases. Then the input to a neuron goes to each of it's sub-neurons instead, and each sub-neuron simply outputs their $z$'s (without applying any activation function). The $a^i_j$ of that neuron is then the max of all its sub-neuron's outputs.
Formally, in a single neuron, say we have $n$ sub-neurons. Then
$$a^i_j = \max\limits_{k \in [1,n]}  s^i_{jk}$$
where
$$s^i_{jk} = a^{i-1} \bullet w^i_{jk} + b^i_{jk}$$
($\bullet$ is the dot product)
To help us think about this, consider the weight matrix $W^i$ for the $i^{\text{th}}$ layer of a neural network that is using, say, a sigmoid activation function. $W^i$ is a 2D matrix, where each column $W^i_j$ is a vector for neuron $j$ containing a weight for every neuron in the the previous layer $i-1$.
If we're going to have sub-neurons, we're going to need a 2D weight matrix for each neuron, since each sub-neuron will need a vector containing a weight for every neuron in the previous layer. This means that $W^i$ is now a 3D weight matrix, where each $W^i_j$ is the 2D weight matrix for a single neuron $j$. And then $W^i_{jk}$ is a vector for sub-neuron $k$ in neuron $j$ that contains a weight for every neuron in the previous layer $i-1$.
Likewise, in a neural network that is again using, say, a sigmoid activation function, $b^i$ is a vector with a bias $b^i_j$ for each neuron $j$ in layer $i$.
To do this with sub-neurons, we need a 2D bias matrix $b^i$ for each layer $i$, where $b^i_j$ is the vector with a bias for $b^i_{jk}$ each subneuron $k$ in the $j^{\text{th}}$ neuron.
Having a weight matrix $w^i_j$ and a bias vector $b^i_j$ for each neuron then makes the above expressions very clear, it's simply applying each sub-neuron's weights $w^i_{jk}$ to the outputs $a^{i-1}$ from layer $i-1$, then applying their biases $b^i_{jk}$ and taking the max of them.
Radial Basis Function Networks
Radial Basis Function Networks are a modification of Feedforward Neural Networks, where instead of using
$$a^i_j=\sigma\bigg(\sum\limits_k (w^i_{jk} \cdot a^{i-1}_k) + b^i_j\bigg)$$
we have one weight $w^i_{jk}$ per node $k$ in the previous layer (as normal), and also one mean vector $\mu^i_{jk}$ and one standard deviation vector $\sigma^i_{jk}$ for each node in the previous layer.
Then we call our activation function $\rho$ to avoid getting it confused with the standard deviation vectors $\sigma^i_{jk}$. Now to compute $a^i_j$ we first need to compute one $z^i_{jk}$ for each node in the previous layer. One option is to use Euclidean distance:
$$z^i_{jk}=\sqrt{\Vert(a^{i-1}-\mu^i_{jk}\Vert}=\sqrt{\sum\limits_\ell (a^{i-1}_\ell - \mu^i_{jk\ell})^2}$$
Where $\mu^i_{jk\ell}$ is the $\ell^\text{th}$ element of $\mu^i_{jk}$. This one does not use the $\sigma^i_{jk}$. Alternatively there is Mahalanobis distance, which supposedly performs better:
$$z^i_{jk}=\sqrt{(a^{i-1}-\mu^i_{jk})^T \Sigma^i_{jk} (a^{i-1}-\mu^i_{jk})}$$
where $\Sigma^i_{jk}$ is the covariance matrix, defined as:
$$\Sigma^i_{jk} = \text{diag}(\sigma^i_{jk})$$
In other words, $\Sigma^i_{jk}$ is the diagonal matrix with $\sigma^i_{jk}$ as it's diagonal elements. We define $a^{i-1}$ and $\mu^i_{jk}$ as column vectors here because that is the notation that is normally used.
These are really just saying that Mahalanobis distance is defined as
$$z^i_{jk}=\sqrt{\sum\limits_\ell \frac{(a^{i-1}_{\ell} - \mu^i_{jk\ell})^2}{\sigma^i_{jk\ell}}}$$
Where $\sigma^i_{jk\ell}$ is the $\ell^\text{th}$ element of $\sigma^i_{jk}$. Note that $\sigma^i_{jk\ell}$ must always be positive, but this is a typical requirement for standard deviation so this isn't that surprising.
If desired, Mahalanobis distance is general enough that the covariance matrix $\Sigma^i_{jk}$ can be defined as other matrices. For example, if the covariance matrix is the identity matrix, our Mahalanobis distance reduces to the Euclidean distance. $\Sigma^i_{jk} = \text{diag}(\sigma^i_{jk})$ is pretty common though, and is known as normalized Euclidean distance.
Either way, once our distance function has been chosen, we can compute $a^i_j$ via
$$a^i_j=\sum\limits_k w^i_{jk}\rho(z^i_{jk})$$
In these networks they choose to multiply by weights after applying the activation function for reasons.
This describes how to make a multi-layer Radial Basis Function network, however, usually there is only one of these neurons, and its output is the output of the network. It's drawn as multiple neurons because each mean vector $\mu^i_{jk}$ and each standard deviation vector $\sigma^i_{jk}$ of that single neuron is considered a one "neuron" and then after all of these outputs there is another layer that takes the sum of those computed values times the weights, just like $a^i_j$ above. Splitting it into two layers with a "summing" vector at the end seems odd to me, but it's what they do.
Also see here.
Radial Basis Function Network Activation Functions
Gaussian
$$\rho(z^i_{jk}) = \exp\!\big(-\frac{1}{2} (z^i_{jk})^2\big)$$

Multiquadratic
Choose some point $(x, y)$. Then we compute the distance from $(z^i_j, 0)$ to $(x, y)$:
$$\rho(z^i_{jk}) = \sqrt{(z^i_{jk}-x)^2 + y^2}$$
This is from Wikipedia. It isn't bounded, and can be any positive value, though I am wondering if there is a way to normalize it.
When $y=0$, this is equivalent to absolute (with a horizontal shift $x$).

Inverse Multiquadratic
Same as quadratic, except flipped:
$$\rho(z^i_{jk}) = \frac{1}{\sqrt{(z^i_{jk}-x)^2 + y^2}}$$

*Graphics from intmath's Graphs using SVG.
Comment: Welcome to CV. +6 this is fabulously informative. I hope we'll see more like it in the future.
Comment: someone should update this with Elu, Leaky ReLU, PReLU and RReLU.
Comment: This would be an interesting paper with a nice list of references. For instance http://arxiv.org/abs/1505.03654 . Feel free to contact me if you decide to write a paper and want other references.
Comment: @Viliami don't you love it when people around you say "someone should do something" and go on with their day? I sure do. Good day!
Comment: there's also the smooth rectified linear function of the form $\log(1+\exp(x))$, and probit.
Comment: Okay, I think I added Logit, Probit, and Complementary log-log, however I don't have a deep understanding of these topics, so I may have misunderstood their written form. Is this correct?
Answer: One such a list, though not very exhaustive: http://cs231n.github.io/neural-networks-1/

Commonly used activation functions
Every activation function (or non-linearity) takes a single number
and performs a certain fixed mathematical operation on it. There are
several activation functions you may encounter in practice:

Left: Sigmoid non-linearity > squashes real numbers to range between [0,1]
Right: The tanh non-linearity squashes real numbers to range between [-1,1].

Sigmoid. The sigmoid non-linearity has the mathematical form $\sigma(x) = 1 / (1 + e^{-x})$ and is shown in the image above on
the left. As alluded to in the previous section, it takes a
real-valued number and "squashes" it into range between 0 and 1. In
particular, large negative numbers become 0 and large positive numbers
become 1. The sigmoid function has seen frequent use historically
since it has a nice interpretation as the firing rate of a neuron:
from not firing at all (0) to fully-saturated firing at an assumed
maximum frequency (1). In practice, the sigmoid non-linearity has
recently fallen out of favor and it is rarely ever used. It has two
major drawbacks:

Sigmoids saturate and kill gradients. A very undesirable property of the sigmoid neuron is that when the neuron's activation
saturates at either tail of 0 or 1, the gradient at these regions is
almost zero. Recall that during backpropagation, this (local) gradient
will be multiplied to the gradient of this gate's output for the whole
objective. Therefore, if the local gradient is very small, it will
effectively "kill" the gradient and almost no signal will flow through
the neuron to its weights and recursively to its data. Additionally,
one must pay extra caution when initializing the weights of sigmoid
neurons to prevent saturation. For example, if the initial weights are
too large then most neurons would become saturated and the network
will barely learn.
Sigmoid outputs are not zero-centered. This is undesirable since neurons in later layers of processing in a Neural Network (more on
this soon) would be receiving data that is not zero-centered. This has
implications on the dynamics during gradient descent, because if the
data coming into a neuron is always positive (e.g. $x > 0$
elementwise in $f = w^Tx + b$)), then the gradient on the weights
$w$ will during backpropagation become either all be positive, or
all negative (depending on the gradient of the whole expression
$f$). This could introduce undesirable zig-zagging dynamics in the
gradient updates for the weights. However, notice that once these
gradients are added up across a batch of data the final update for the
weights can have variable signs, somewhat mitigating this issue.
Therefore, this is an inconvenience but it has less severe
consequences compared to the saturated activation problem above.

Tanh. The tanh non-linearity is shown on the image above on the right. It squashes a real-valued number to the range [-1, 1]. Like the
sigmoid neuron, its activations saturate, but unlike the sigmoid
neuron its output is zero-centered. Therefore, in practice the tanh
non-linearity is always preferred to the sigmoid nonlinearity. Also
note that the tanh neuron is simply a scaled sigmoid neuron, in
particular the following holds: $ \tanh(x) = 2 \sigma(2x) -1  $.

Left: Rectified Linear Unit (ReLU) activation function, which is zero when x &lt 0 and then linear with slope 1 when x &gt 0.
Right: A plot from Krizhevsky et al. (pdf) paper indicating the 6x improvement in convergence with the ReLU unit compared to the tanh unit.

ReLU. The Rectified Linear Unit has become very popular in the last few years. It computes the function $f(x) = \max(0, x)$. In
other words, the activation is simply thresholded at zero (see image
above on the left). There are several pros and cons to using the
ReLUs:

(+) It was found to greatly accelerate (e.g. a factor of 6 in Krizhevsky et
al.) the
convergence of stochastic gradient descent compared to the
sigmoid/tanh functions. It is argued that this is due to its linear,
non-saturating form.
(+) Compared to tanh/sigmoid neurons that involve expensive operations (exponentials, etc.), the ReLU can be implemented by simply
thresholding a matrix of activations at zero.
(-) Unfortunately, ReLU units can be fragile during training and can "die". For example, a large gradient flowing through a ReLU neuron
could cause the weights to update in such a way that the neuron will
never activate on any datapoint again. If this happens, then the
gradient flowing through the unit will forever be zero from that point
on. That is, the ReLU units can irreversibly die during training since
they can get knocked off the data manifold. For example, you may find
that as much as 40% of your network can be "dead" (i.e. neurons that
never activate across the entire training dataset) if the learning
rate is set too high. With a proper setting of the learning rate this
is less frequently an issue.

Leaky ReLU. Leaky ReLUs are one attempt to fix the "dying ReLU" problem. Instead of the function being zero when x < 0, a leaky ReLU will instead have a small negative slope (of 0.01, or so). That is, the function computes $f(x) = \mathbb{1}(x < 0) (\alpha x) + \mathbb{1}(x>=0) (x) $ where $\alpha$ is a small constant. Some people report success with this form of activation function, but the results are not always consistent. The slope in the negative region can also be made into a parameter of each neuron, as seen in PReLU neurons, introduced in Delving Deep into Rectifiers, by Kaiming He et al., 2015. However, the consistency of the benefit across tasks is presently unclear.

Maxout. Other types of units have been proposed that do not have the functional form $f(w^Tx + b)$ where a non-linearity is applied
on the dot product between the weights and the data. One relatively
popular choice is the Maxout neuron (introduced recently by
Goodfellow et
al.) that
generalizes the ReLU and its leaky version. The Maxout neuron computes
the function $\max(w_1^Tx+b_1, w_2^Tx + b_2)$. Notice that both
ReLU and Leaky ReLU are a special case of this form (for example, for
ReLU we have $w_1, b_1 = 0$). The Maxout neuron therefore enjoys
all the benefits of a ReLU unit (linear regime of operation, no
saturation) and does not have its drawbacks (dying ReLU). However,
unlike the ReLU neurons it doubles the number of parameters for every
single neuron, leading to a high total number of parameters.
This concludes our discussion of the most common types of neurons and
their activation functions. As a last comment, it is very rare to mix
and match different types of neurons in the same network, even though
there is no fundamental problem with doing so.
TLDR: "What neuron type should I use?" Use the ReLU non-linearity, be careful with your learning rates and possibly
monitor the fraction of "dead" units in a network. If this concerns
you, give Leaky ReLU or Maxout a try. Never use sigmoid. Try tanh, but
expect it to work worse than ReLU/Maxout.
...
License:
The MIT License (MIT)
Copyright (c) 2015 Andrej Karpathy
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.*

Other links:

tanh activation function vs sigmoid activation function
Answer: I don't think that a list with pros and cons exists. The activation functions are highly application dependent, and they depends also on the architecture of your neural network (here for example you see the application of two softmax functions, that are similar to the sigmoid one).
You can find some studies about the general behaviour of the functions, but I think you will never have a defined and definitive list (what you ask...).
I'm still a student, so I point what I know so far:

here you find some thoughts about the behaviours of tanh and sigmoids with backpropagation. Tanh are more generic, but sigmoids... (there will be always a "but")
In Deep Sparse Rectifier Neural Networks of Glorot Xavier et al, they state that Rectifier units are more biologically plausible and they perform better than the others (sigmoid/tanh)
Comment: This is the "correct" answer. One can produce a list but pros and cons are completely data-dependent. In fact, learning activation functions is much more reasonable in theory. The reason there's not much research focus on it is that sigmoid "just works". In the end, your only gain is convergence speed which is often unimportant
Answer: Just for the sake of completeness on Danielle's great answer, there are other paradigms, where one randomly 'spins the wheel' on the weights and / or the type of activations: liquid state machines,
extreme learning machines and echo state networks.
One way to think about these architectures: the reservoir is a sort of kernel as in SVMs or one large hidden layer in a simple FFNN where the
data is projected to some hyperspace. There is no actual learning, the reservoir is re-generated until a satisfying solution is reached.
Also see this nice answer.
Answer: An article reviewing recent activation functions can be found in
"Activation Functions: Comparison of Trends in Practice and Research for Deep Learning" by Chigozie Enyinna Nwankpa, Winifred Ijomah, Anthony Gachagan, and Stephen Marshall

Deep neural networks have been successfully used in diverse emerging domains to solve real world complex problems with may more deep learning(DL) architectures, being developed to date. To achieve these state-of-the-art performances, the DL architectures use activation functions (AFs), to perform diverse computations between the hidden layers and the output layers of any given DL architecture. This paper presents a survey on the existing AFs used in deep learning applications and highlights the recent trends in the use of the activation functions for deep learning applications. The novelty of this paper is that it compiles majority of the AFs used in DL and outlines the current trends in the applications and usage of these functions in practical deep learning deployments against the state-of-the-art research results. This compilation will aid in making effective decisions in the choice of the most suitable and appropriate activation function for any given application, ready for deployment. This paper is timely because most research papers on AF highlights similar works and results while this paper will be the first, to compile the trends in AF applications in practice against the research results from literature, found in deep learning research to date.


Effective Data read /write speed

Question: I plan to by new core i7 laptop with eSATA 5400 rpm HDD and usb 3.0 and  usb c ports.If I connect 500 GB Segate fast SSD  (external), can I get practical best reed/ write speed of the SSD or will it have any limitations due to read/ write speed of HDD?
Thanks.
Answer: Depends on what you are going to do with it.
When copying from the hard drive to the SSD, you will be limited to the hard drive speed.
When running applications from the SSD, the applications will most likely use the hard drive to store their application data files and cache, which can slow the applications down.
If you are just storing files on the drive to read from, then the hard drive should not affect the speed.

Effective Data read /write speed


How to sign out from Facebook on a lost iPod

Question: My iPod was stolen. I changed my Facebook password and it still says 2 devices are connected. How can I remotely logout of Facebook on the lost iPod?
Comment: Could you rephrase your question so that it is actually a _question_? It is a bit hard to read what you really want to accomplish.
Comment: It could be that you are logged in another session other than the one on the iPod. Check Punchlinern's answer.
Answer: I assume you want to end the Facebook session (log out) on the iPod? To do that, follow these steps:
Enter settings

Go to the Security tab and click the Where You're Logged In section

Here you'll find all logged in sessions. You can choose to End activity on all those associated with the iPod. In the below picture you see those associated with my Android device as an example. Look for iPod or iOS instead.

It can also be a good idea to remove it from the trusted browsers, still in the Security tab in settings:
Comment: Haha, I like to use the so powerful features of MS Paint.
Comment: Just one comment... your red arrows should be HAND DRAWN. ;)
Answer: Simply go to security tab in Facebook settings.
There is an option to remove devices
Answer: Try to login to your account with a wrong password or click on forgot your password option from any other device. Facebook will ask for your phone or email id to send you the code or password reset link. click that link and then facebook will ask for your new password and an option at the bottom to log you out of other devices. check that option and enter a new password. you will be logged out of all the other devices.
Answer: In addition to removing the device from your Facebook account as @Punchlinern suggested, I would recommend erasing your device from Apple's web site. That would take care of other files, accounts, etc.


What distinguishes the declaration, the definition and the initialization of a variable?

Question: After reading the question, I know the differences between declaration and definition. So does it mean definition equals declaration plus initialization?
Comment: Initialization is for variables. Definition can apply also for functions, where you define the body.
Comment: So do you mean for a variable, it's right.
Answer: Declaration
Declaration, generally, refers to the introduction of a new name in the program. For example, you can declare a new function by describing it's "signature":
void xyz();

or declare an incomplete type:
class klass;
struct ztruct;

and last but not least, to declare an object:
int x;

It is described, in the C++ standard, at §3.1/1 as:

A declaration (Clause 7) may introduce one or more names into a translation unit or redeclare names introduced by previous declarations.

Definition
A definition is a definition of a previously declared name (or it can be both definition and declaration). For example:
int x;
void xyz() {...}
class klass {...};
struct ztruct {...};
enum { x, y, z };

Specifically the C++ standard defines it, at §3.1/1, as:

A declaration is a definition unless it declares a function without specifying the function's body (8.4), it contains the extern specifier (7.1.1) or a linkage-specification25 (7.5) and neither an initializer nor a function- body, it declares a static data member in a class definition (9.2, 9.4), it is a class name declaration (9.1), it is an opaque-enum-declaration (7.2), it is a template-parameter (14.1), it is a parameter-declaration (8.3.5) in a function declarator that is not the declarator of a function-definition, or it is a typedef declaration (7.1.3), an alias-declaration (7.1.3), a using-declaration (7.3.3), a static_assert-declaration (Clause 7), an attribute- declaration (Clause 7), an empty-declaration (Clause 7), or a using-directive (7.3.4).

Initialization
Initialization refers to the "assignment" of a value, at construction time. For a generic object of type T, it's often in the form:
T x = i;

but in C++ it can be:
T x(i);

or even:
T x {i};

with C++11.
Conclusion

So does it mean definition equals declaration plus initialization?

It depends. On what you are talking about. If you are talking about an object, for example:
int x;

This is a definition without initialization. The following, instead, is a definition with initialization:
int x = 0;

In certain context, it doesn't make sense to talk about "initialization", "definition" and "declaration". If you are talking about a function, for example, initialization does not mean much.
So, the answer is no: definition does not automatically mean declaration plus initialization.
Comment: Not quite: `int x;` is a definition as well as a declaration.
Comment: @Angew, thanks, I've added a more complete *definition*.
Comment: @Tony I think (please someone correct me if Im wrong so I learn) 'extern int x' makes it just a declaration, meaning that definition is found elsewhere. Whilst 'int x' actually defines it albeit it gets appointed with random garbage value, unless you initialize it specifically like 'int x = 5'.
Comment: I think the question that OP mentioned in his post answers definition and declaration pretty well. But the original question was `So does it mean definition equals declaration plus initialization`.
Comment: @Tahlil, added a conclusion. Thanks for making me notice.
Answer: Declaration says "this thing exists somewhere":
int foo();       // function
extern int bar;  // variable
struct T
{
   static int baz;  // static member variable
};

Definition says "this thing exists here; make memory for it":
int foo() {}     // function
int bar;         // variable
int T::baz;      // static member variable

Initialisation is optional at the point of definition for objects, and says "here is the initial value for this thing":
int bar = 0;     // variable
int T::baz = 42; // static member variable

Sometimes it's possible at the point of declaration instead:
struct T
{
   static int baz = 42;
};

…but that's getting into more complex features.
Comment: @JamesKanze: Decided to keep it super simple for these purposes
Comment: Very well explained, except that initialization is a bit more complex than that.  (This wouldn't be C++ if it were that simple.)  Initialization includes things like the zero initialization of variables with static lifetime, and default constructors, as well as what you show.  (And to add to the confusion: in C, initialization can be the first time the variable is assigned to; e.g. in statements like "taking the value of an uninitialized variable".  It wouldn't surprise me if some of this also slipped into C++.)
Comment: Oh, and there are also special cases where C++ permits the specification of the initialization in a declaration.
Comment: This doesn't make sense, static member variables allocate memory.
Comment: I'm no expert (btw, `baz` needs to be `const` or `constexpr`, otherwise it won't compile), but I think the compiler allows initialization without definition because that value can be used in expressions that can be evaluated at compile time. So if you have something like `int x = T::baz;`, the compiler just does search-and-replace on it and the expression becomes `int x = 42;`. If you push `baz` onto a function call, `foo(T::baz);`, then you should allocate memory for it via a definition.
Answer: For C, at least, per C11 6.7.5:

A declaration specifies the interpretation and attributes of a set of
  identifiers. A definition of an identifier is a declaration for that
  identifier that:

for an object, causes storage to be reserved for that object;
for a function, includes the function body;
for an enumeration constant, is the (only) declaration of the identifier;
for a typedef name, is the first (or only) declaration of the identifier.

Per C11 184.108.40.206-10:

An initializer specifies the initial value stored in an object ... if
  an object that has automatic storage is not initialized explicitly,
  its value is indeterminate.

So, broadly speaking, a declaration introduces an identifier and provides information about it. For a variable, a definition is a declaration which allocates storage for that variable.
Initialization is the specification of the initial value to be stored in an object, which is not necessarily the same as the first time you explicitly assign a value to it. A variable has a value when you define it, whether or not you explicitly give it a value. If you don't explicitly give it a value, and the variable has automatic storage, it will have an initial value, but that value will be indeterminate. If it has static storage, it will be initialized implicitly depending on the type (e.g. pointer types get initialized to null pointers, arithmetic types get initialized to zero, and so on).
So, if you define an automatic variable without specifying an initial value for it, such as:
int myfunc(void) {
    int myvar;
    ...

You are defining it (and therefore also declaring it, since definitions are declarations), but not initializing it. Therefore, definition does not equal declaration plus initialization.
Comment: In C++, initialization is significantly different.  (C also has the equivalent of zero initialization of objects with static lifetime.)
Comment: @JamesKanze: Yes, unfortunately this question is tagged with both C and C++, which isn't very helpful on those points where they differ.
Comment: Quite.  For the compatible types, the _intent_ (at least originally) in C++ was that they behave the same as in C; in fact, I think they do still behave the same in all of the cases where the languages are themselves compatible.  But the wording used to define this behavior is quite different.
Answer: "So does it mean definition equals declaration plus initialization."
Not necessarily, your declaration might be without any variable being initialized like:
 void helloWorld(); //declaration or Prototype.

 void helloWorld()
 {
    std::cout << "Hello World\n";
 }


the image below is a device that came came with my 1993 toyota carina e. i want to know it name and used. thank you

Question: the picture below is a device that came with my 1993 toyota carina e
Comment: @DiligentLove Trying to be gentle here  - you can't really sub in a pic of something that looks-kinda-sorta-like-it instead of giving us actual details. If you have the *exact* same thing then I'd be saying I'm not surprised you have issues - it's the ECU from a Renault Laguna so it probably wouldn't work very well in a  Toyota Carina. But I don't think that's what you have. How about you wind it back a few steps and tell us what problems you are having, what symptoms your car is exhibiting and we can try work from there?
Comment: Looks like a car alarm control unit to me.
Comment: That exact same picture is on this website as an ECU for a Renault Laguna - https://www.pieces-okaz.com/en/computer-engine-ecu/11449-engine-control-unit-ecu-renault-laguna-1997-20-dgm48574.html
Comment: So that is not the OP's own photo.
Comment: yes but i have the same thing in my 1993 carina e and i want it name and functions because am having some electrical issues. thank you
Answer: It's an engine control module, and it's not the problem.
AKA "computer".  However, "computers" usally get blamed for any electrical problem, first because old-guard mechanics hate the things and long for the simple old days, and second because their functionality is fairly opaque and hard to analyze/test, and third, because resetting a computer often clears the problem (at least temporarily).  Replacing a computer is a rather expensive way to reset a computer.
The engine computer relies on a variety of sensors and actuators to manage the engine. For instance, it has a coolant temperature sensor that tells it how hot the engine is, and an oxygen sensor to sense whether the right amount of fuel is being injected.  It has an EGR actuator that recycles exhaust gases when necessary to improve smog/fuel economy.  It might have as many as two dozen sensors or actuators.  Those fail all the time.  And fortuantely, they're cheap.
I have yet to see a computer that is actually dead.  However, I have seen many, many faulty sensors or actuators, and occasionally, a damaged wiring harness.
In particular, a "Check Engine" light is the computer saying "I am working properly, but one of my sensors is borked or telling me the engine is borked".  This is the point where idiots replace the computer :)
Mind you, this is only likely to bear on engine-related problems, or possibly speedometer.  If something is amiss in the body wiring, e.g. Funky radio, power windows not working, etc. -- for that, you need to look to a "Body control module" (if there even is one in a '93, not that likely) which is a separate computer that manages comfort and convenience devices in the body... Or plain old wiring issues like stuck switches, frayed wires etc.
Comment: Boss thanks for your explanation i really appreciate but my car already has ECU and this one too. But my car uses petrol and LPG gas and i don't know if the ECU in the picture works with the LPG
Comment: @DiligentLove  This is information you should have put in your question.  Is there any more information that you are keeping from us.  For all we know this device could have been in a cardboard box in the car trunk.
Comment: Then it must be the Body Control Module I also mentioned. Again, probably not the problem.


How to transfer iTunes library between user accounts on Windows?

Question: I have Windows 7 and the latest iTunes. My original user account, where my library is, seems to have contracted a virus so it's sky-rocketing my internet usage. I created a new user account, but I need to transfer my library, since I want to delete my old account to delete the virus. I've tried using an external hard drive, home sharing, and changing the library location to a public file, but nothing seems to work.
Answer: Under the old user, launch iTunes, then Preferences, then the Advanced tab and discover the "iTunes Media folder location".
Switch to the new user and launch iTunes. From the advanced tab, verify that the new user location path corresponds to the new user name. If it doesn't, quit iTunes, create the iTunes Music folder (using the path structure of the old location as a guide).
Relaunch iTunes. Set the location to the new folder you just created.
Quit iTunes and use Windows Explorer to copy everything in from the old user folder to the new user folder. Launch iTunes from the new user and your music should be there.


how come for any given f(x) a y could have multiple values of x, while x has only a single y value

Question: I guess i might be answering my own question here, but is it because of how we chose to represent our functions? That is, for the lack of better term, strech it along the x axis? Has it it been f(y) would it be the other way around?
Answer: A function from some set $X$ to $\mathbb R$ is, by definition, a specific type of relation, which is a collection of pairs, $f\subseteq \{(x, y)\in X\times \mathbb R\}$. $f$ is a function if each $x\in X$ appears in precisely one pair in $f$, which is why you can write $f(x) = y$  instead of $(x,y)\in f$.
The reason why we are interested in functions is not really mathematical (mathematitians don't mind analyzing other types of relations) but more "real-life" in that very very often, functions are a very good way of describing things in the world. For example, if you look at how some object moves through time, then at each point in time, it is on only one location. Therefore, it is easy to see that you can see the position of the object as a function of time.
Answer: By definition, f(x) is a manipulation on the number x such that we get another number, y.
Since any function must be clearly defined, when we input a value of x we should get a unique number, not a set of numbers. Therefore, one value of x must only return one value of y. On the other hand, nothing is stopping a specific value of y to return multiple x values, and is commonplace in periodic functions, like trigonometric functions.
So in a way, you did answer your own question. This property is due to the limits of our definition, not a mathematical phenomenon.
Answer: The definition of a mapping states that each anticident in the initial set i.e each $x$ shall have one and only one image in the final set moreover this mapping can be a non injective mapping thus there exist maybe more than 1 $image$ $y$ having more than one anticident $x$ ! Note that the anticidents are different as stated in the beggining so you can call the relation $f$ a mapping. Thats why every $x$ in a mapping is related to only 1 image while there may exist Images that are related to several different Anticidents.


Heraldic Law and Jurisdiction of armorial assumption

Question: In countries where armigers (persons or corporations bearing a coat of arms) must prove their entitlement to such (eg. most legal traditions within the UK, Canada, and other Commonwealth realms) what laws govern armigers bearing foreign arms (eg. a German, French, or American armiger in the UK)?
Comment: Do you mean specifically and only in England and Wales, or do you mean in any country which regulate coats of arms (such as Canada, Denmark, Norway)?
Comment: @user6726, ideally the latter, but the former would be sufficient if Scotland, which has different heraldic law from England were factored in.
Answer: I am not sure 'laws' is the appropriate term: though the Court of Chivalry still exists it has sat only once in the last 300 years (to determine whether a theatre had the right to display the municipal arms). The decision as to who is entitled to display a coat of arms is made by the heraldic authority in each country (the College of Arms for England and Wales and much of the Commonwealth; Lyon King of Arms for Scotland).  These authorities work closely with each other (unsurprisingly when you consider that one of their original functions was to broker peace treaties), so it is likely that if arms granted in one country will be recognised by the heralds in another; but the decision is made individually.
I found one interesting quote, though my Google-fu does not allow me to identify the source:

M . J. Sayer writes some interesting things about registrations of
  foreign arms with the College of Arms in his "English Nobility: The
  Gentry, the Heralds and the Continental Context" (Norfolk Heraldry
  Society, 1979).  On pages 17-18 Sayer writes "where foreign arms are
  recognized in England, they of course rank in England as ensigns of
  nobility, even if the family was not noble abroad, an anomaly
  reflecting the greater success of the English crown's control over
  arms."  A footnote continues: "Sir Anthony Wagner kindly informs me
  that 'foreign arms for which the authority of a document from a
  Sovereign or heraldic authority can be produced have a head start
  towards recognition here over Burgher arms.  Nevertheless, though
  acceptance of the document may be the first stage, it has often been
  thought necessary that there should be a second stage of confirmation
  of the right to use the arms in this country.  One sees that there
  could in theory, and sometimes in practice, be a clash with the design
  of existing English arms.  At the present day we keep a record
  entitled Foreign Arms where such documents are entered when accepted
  as valid, but over and above this there needs to be acceptance by
  Patent or otherwise for use in England.' (30 March 1978)...Grants by
  foreign authorities to persons not within their jurisdictions, i.e. to
  nationals of a third country, are not accepted for registration."
The College of Arms has at times contacted foreign governments to
  ascertain the status of their native heraldic authorities.

Finally, I am surprised by your reference to American armigers; it is my understanding that the Article I of the US Constitution forbids the states to create titles of nobility.
Comment: Congress cannot create noble titles, but American citizens aren't forbidden from having them.
Comment: An armiger is not specifically a noble, this is, in fact, an incredibly Anglo-centric view of heraldry. On the Continent, it was not uncommon for everyone living in a city to have their own arms, even among non-citizens. For example: the famed Rothschild banking family took their name from the arms of their old residence, *Haus zum Rotschild* (Redshield House) even though, as Jews, they weren't accorded citizenship until after the Napoleonic Wars, they still were accorded the right to bear arms, as city residents.
Comment: Courtesy tag @Stackstuck


Magento grid not fully displayed

Question: I have Installed Magento 2.4.5 but grids on the admin side are all right aligned ( not displaying on full screen )
Comment: If any third party extension or something install then, disable that from config.php and then run all the commands and then check.
Comment: Check Magneto console or log file for the real error.
Comment: @DhirenVasoya getting these kind of errors in logs, Broken reference: the 'page.actions.toolbar' tries to reorder itself towards '', but their parents are different: 'page.main.actions' and '' respectively. [] []
Comment: Thank you issue solved @DhirenVasoya
Answer: It is caused by installing third-party extensions or sometimes users' own custom extensions. I would suggest disabling those if you have made any custom modules or installed extensions, by running the bin/magento module:disable <NAME> or from the config.php file located in app/etc folder.
After that, run the bin/magento setup:upgrade command and it should be fixed.

Magento grid not fully displayed


How to change the default app for callto/tel links from Facetime to Skype/other?

Question: Callto/tel links inside Chrome open up for me in Facetime. How can I change the default application used for opening such links? (I'd like to set it to Skype)
Answer: Two options:
1. Do this via Facetime (Preferences > Settings > Default for calls)
2. If this doesn't work, try RCDefault App: http://www.rubicode.com/Software/RCDefaultApp/
For me, Skype was not showing in Facetime's list, but RCDefault App made it possible to add a custom app there. What I haven't yet managed to make work is that the number would get copied to Skype.
By the way, this has been answered also at https://superuser.com/questions/1132108/how-to-configure-the-default-app-to-open-tel-links-in-macos-x


can't debug datamodule in DLL at application server delphi

Question: I wanted to debug a data module that I've created and which is a part of a DLL, but for some reasons, I can't debug it. The debugger doesn't seems to be hitting the break point, is there a way i can debug my Dll/ data module.
Answer: Check list:

be sure your DLL have included debug info
if it's COM DLL - be sure it is registered on same place where you build it
try to rebuild the DLL, rebuild the App if you have changed the interface
check in EventLog when debugger loads the DLL does it says it have or dont have debug info.
Answer: To debug the DLL, the project loaded in Delphi must be the DLL itself. You enter a host application (Your application server executable), the one which will load the DLL. Make sure the host application load the DLL exactly where you have built it. You cannot move the DLL once constructed.
If the DLL must be in some fixed place, be sure to set the DLL's project options so that the DLL is produced at the right place.
And of course you must activate the debug build configuration for the DLL.
Using a tool such as ProcessExplorer (https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer), you can see if the DLL you built was actually the one loaded by your application server.


Is there a way to know the actual permissions of an app now before installing?

Question: I'm not a fan of the current so-called "simplified" permissions in the Play store, and would like a way to check the exact permissions of an app before installing. This need not be a tool that does the installation itself, I just need a site or app that helps me look up the exact permissions from the app before I install it. Is there such a tool available?
Answer: You don't need an external tool for this. On the versions of the Play Store with the new "simplified" permissions dialog, you can still find a full list of permissions from the app's store page (the screen with the icon at the top, description, screenshots, &c.). Scroll down to the bottom of the page. Under "Additional Information" you'll see "Permissions" with a link "View details". Click this link to see a dialog with the list of individual permissions.
However, if you do want an external tool, AppBrain is an alternative web front-end to the Play Store, and lists the permissions individually for each app. From there you can click through to install the app from the real Play Store. Not all apps appear on AppBrain, but many of them do, especially the popular apps.
There's also a resource offered by Izzy, a good friend of this site. His website offers a curated list of apps, searchable by permissions. It also gives a fuller description of each permission, and highlights interesting combinations. Because apps are added to his site by a manual process, it's a very small list compared to the huge number of apps on the Play Store, but consists of a good range of recommended and popular apps in different categories.
Comment: *Not all apps appear on AppBrain* → You can make them appear (see [my answer](http://android.stackexchange.com/a/77321/16575) ;)
Comment: Your last suggestion in your answer is by far the best. 99% of the time nobody gives a flying f about the permission itself - its what th3y intend to do with that permission that is of concern! Example: you install an app that requires contact permissions but you never know why and so you just assume is cause it needs to access contact info to share info with a contact thru the app but instead it uploads all your contacts to their servers without your permission! Omg I just had am idea oh never mind. I seem to be swearing more than usual lately, is really pissing me off
Answer: In his answer, Dan already pointed to my list of apps by real-life topics, which offers some extras as well. Currently, it only covers a small subset of what's available on Google Play, though (for some numbers: ~10,000 apps = ~1% of Google Play, including roughly 5% of the apps available in Aptoides (curated) main repository, and a third of the apps available on F-Droid – with some overlappings).
One more resource to mention for apps not covered there is Appbrain, which is a different front-end to Google Play Store. So you should find all apps there, and they also show "concerns" (what Dan put as "interesting combinations") – a very useful thing to be warned about potential danger. If an app is missing from their list, you can easily make it appear by copy-pasting its package-name (or full Google Play URL) on this page. It might take some minutes then, but it will eventually be available (usually within 10..30 minutes).

Update:
Galen recently also introduced his app "StripSearch". I've used it a little (and had an intensive discussion with him – partly playing "beta tester" ;) This app not only shows you the permissions before installing an app – it even allows you to filter your app searches by permissions, e.g. to exclude unwanted ones (privacy concerns) or specifically require others (such as using the real flash-light when in need of a flashlight app, practically excluding those using the display or other means). Runs pretty well, definitely worth a try. You can find StripSearch at Google Play.
Answer: While this isn't used for viewing permissions (better off following the advice given by the other answers), I feel that I am obligated to endorse
XPrivacy , an Xposed module for restricting permissions. If you are ever worried about extensive permissions, use this. Its completely open source too!
Answer: I think that a mixture of XPrivacy and PlayPermissionsExposed will do the trick. They are part of the popular Xposed framework. For GB devices, you can use liudongmiao's Xposed Installer GB at the XDA developers forum.
Make SURE to read the instructions. I'm not responsible if you do not.
If you run into any problems, please feel free to comment.
PS: The XPosed framework needs root, unless you manually decide to flash in recovery. If you are not willing to do so, look at the other answers.


Es gibt hier jede Menge hohe Berge drum herum

Question: In what case is Berge here? Nominative or genitive?
I understood that "Menge" might express discrete values (uncountable large numbers) and in this case it comes with "unbestimmten Artikel: e.g eine / jede. Is that correct?
Answer: Berge is neither nominative nor genitive in your example:
It is in fact accusative. The non-personal use of "geben" (for "sth exists") generally rules the accusative.
Note you could properly express the same fact with a genitivus partitivus such as

Es gibt hier jede Menge hoher Berge

"jede Menge" literally means "any amount of", so "a lot". Very much so as in English.
Comment: "Wen/Was gibt es hier jede Menge drum herum ? - Berge" Correct ?
Comment: @Doomenik Yes, correct. But the standard way of "asking" for a subject/object to find the proper case tends to help learners much less than native speakers sometimes think.
Comment: Yes ty your probably right.
Answer: It is accusative case.

Nom: hohe Berge

Hohe Berge sind schön.

Gen: hoher Berge

Beim Anblick hoher Berge möchte Lisa jodeln.

Dat: hohen Bergen

Bei den hohen Bergen scheint die Sonne.

Acc: hohe Berge

Ich sehe hohe Berge.

So, »hohe Berge« can only be nominative or accusative case. Genitive is out of the game.
It is possible, but rare, that something other than the subject appears in nominative case in a German sentence. The subject in this sentence is the first word »es«, which is an expletive subject. So it is very unlikely, that something else than the high mountains are in nominative case. But this is just a rule of thumb.
What really helps is the phrase »es gibt« itself (there is or there are in English). The thing(s) that is/are, always appears in accusative case. This means, that the nominal group »jede Menge hohe Berge« has to stand in accusative case.
But you asked for »hohe Berge«, so we have to split up the four-word phrase into its components:
There are two ways to express every amount of high mountains in German:

Es gibt jede Menge hoher Berge

As shown above, hoher Berge is genitive case, and this variation is called »genitivus partitivus«. It expresses, that the amount (which stands in accusative case) consists of high mountains. (Note, that here hoher Berge is an ad-on to jede Menge. So the rule »es gibt needs accusative case applies to jede Menge.)

Es gibt jede Menge hohe Berge

This is the same construction as in »es gibt zwei hohe Berge« (there are two high mountains) or »es gibt viele hohe Berge« (there are many high mountains). This means: Here the words jede Menge (which in this construction are a quantifier, not an object) are an ad-on to hohe Berge, and the sentence works without this ad-on too:

Es gibt hohe Berge

Now it is clear: Since »es gibt« always needs in object in accusative case, hohe Berge must stand in this case. And adding a quantifier doesn't change this case.
Comment: Wenn ich versuche "Ich sehen hohe Berge." in die richtige Form "Ich sehe..."  zu bringen, beklagt sich stackexchange dass zu wenig geändert wurde. Magst du sehen ins Singular bringen?


TypeScript: Typing the result of an external module's non-mutating extension function

Question: Let's say I'm using external package "foo". Its main export is an object which has an .extend() method that can be used to add functionality by creating a derived object (the original is left unchanged). It looks something like this:
import foo from 'foo';

const extendedFoo = foo.extend(
  // Some complex definition
);

const result1a = extendedFoo.bar.existingMethod();
const result2a = extendedFoo.bar.addedMethod();

const result1b = foo.bar.existingMethod();
const result2b = foo.bar.addedMethod();  // error!

Unfortunately the provided types for the package aren't great and .extend() is declared to return any, which is unhelpful. I can see a couple of ways to fix the issue:
Option 1, module augmentation/interface merging:
import foo from 'foo';

declare module 'foo' {
  interface Bar {
    addedMethod(): string;
  }
}

const extendedFoo = foo.extend(
  // ...
) as foo.Foo;

Option 2, interface extension:
import foo from 'foo';

interface ExtendedBar extends foo.Bar {
  addedMethod(): string;
}

interface ExtendedFoo extends foo.Foo {
  bar: ExtendedBar;
}

const extendedFoo = foo.extend(
  // ...
) as ExtendedFoo;

The issue with option 1 is that if the original foo module is used anywhere, TypeScript will think it has this added method when it doesn't.
The issue with option 2 is that it's oversimplified and incomplete: in reality the Bar interface is used in numerous places in the type definitions, e.g. as the type of a parameter in some functions, as part of big union types or other more complex types, etc. Accounting for all these would make my extended types both lengthy and brittle, whereas with option 1 all I had to do was change the relevant part at its source.
Of course the real solution would be for the provided types to be rewritten - for example a bunch of them could be made generic, including the base Foo type, and the type params propagated through. But assuming that isn't an option, is there any other way I can get the desired result while avoiding the issues I mentioned?
Answer: A type intersection with generics should do it !
interface Foo {
    extend<T>(obj: T): Foo & T;
}

declare let foo: Foo;

const extendedFoo = foo.extend({
    bar: (): number =>  { return 42 }
});


Did the McCallisters bother to call home?

Question: In "Home Alone", wouldn't Kevin's Parents try to call home? I don't remember them trying to call the house to check up on Kevin. I know the power lines were down due to a tree branch the night before but they must of been fixed in 1 day because when Kevin called 911 it was working and when some of the other Family Members called neighbors there were Answering Machines so the phone lines must have been working again but the McCallisters didn't know about it.
Comment: Except the dad called the neighbor and left a message from Paris. Why wouldn't he just call home and they were only gone a little over two days as they explain when the mom refuses to wait for the flight the family comes home on.
Answer: The phone lines were down due to the storm.
This is addressed in the dialog. From the script:

As Kay enters the van, the power man approaches her. "Mame, I just want you to know that your power is fixed, but your phone lines are a mess. it's going to take Ma bell a couple of days to patch it up, especially around
the holidays."

When he calls the cops near the end of the movie, this is several days later, so it is reasonable to assume that the phone lines have been fixed by then. His parents did not continually try to call throughout the movie; they were too busy just trying to get home, and they didn't know when phone service would be restored again.
Now, there is also an often talked about plot hole that Kevin orders a pizza, and he shouldn't have been able to do that if the phones were down. This is likely just a plot hole in the movie, but there are also possible explanations for it, such as that it was only the long distance lines that weren't working, while local calls would still work.
Comment: the power going out is the reason they all get up late too, and knock on effect of this is Keven being left HOME ALONE


Infuse Claymore on a strength build?

Question: Has anyone with a strength build infused their +10 Claymore?  I'm a bit hesitant to waste my infusing stones if the effects aren't all that great.  Can you level up the infusion like you can with Titanite for normal weapon upgrades?
Answer: In Dark Souls 2 you can consider infusions and the +N upgrades as two separate properties of the weapon. For example, if you have a +10 Claymore and infuse it with lightning, you'll have a +10 Lightning Claymore. Further, you can "uninfuse" it and it'll go back to just a +10 Claymore. There's no upgrading of the infusions themselves. That said, if you infuse a less than +10 weapon, you can still continue upgrading it to +10 with titanite, and it will retain its infusion as you upgrade it.
Whether or not to infuse a weapon is another question. Each infusion does something different to the weapon. Something like Lightning will take away some of the base damage on the weapon and replace it with lightning damage. This also affects the scaling. A purely strength based character probably wouldn't want to infuse with lightning, since doing so would likely lower the scaling off of strength and add scaling for lightning (which is affected by Faith). A Strength/Faith hybrid build might do a lightning infusion, though.
There's so many combinations, and each type of infusion works differently (Raw increases base damage but lowers scaling, Mundane adds scaling based on your lowest stat, etc). My best advice would be to experiment with different types, as there's not necessarily a "right" answer (keep in mind that you can preview the changes before you actually go ahead with the infusion). It depends a ton on your build and playstyle.
Comment: Great answer! Welcome to the site! A good note is that after you have progressed through a point in the game, there will be a merchant that sells at least 1 of every single stone in the game. You really dont have to worry about making the 'wrong' infusion, just play around like @Meorrow said.

Infuse Claymore on a strength build?


What is the name for the kind of abbreviation that employs an underlined superscript to the terminal characters?

Question: Sometimes abbreviations are made by placing the terminal characters in superscript and underlining them, as in "1st," "Mc," and "No" (see examples below). What is this process called, and what is a general name for any one of these abbreviations (the abbreviation a whole, not the superscript characters alone) so made?

Image Credit: commercial art from a box of PG Tips tea (license: fair use).
Comment: Typographically, they are superscript and subscript as what they are traditionally known as *superiors* and *inferiors* respectively. The underlined **№** is a US version of the UK **No**. (https://en.wikipedia.org/wiki/Numero_sign)
Comment: But these are not mere superscripts, but underlined superscripts that are paired with one (or more?) preceding characters. In your "No" example, the "N" is normal size. Read the question again. I am asking about the abbreviation as a whole, not just the superscript part.
Comment: I was intrigued by this question and even more so when I could not find any howto example, neither in my Perrouseaux nor in my "Lexique des Règles typographiques". I guess the answer by @htmlcoderexe explains why we are not (yet) finding the technical name for this beast: it is hardly in use anymore. Abbreviations with superscript very much in use, esp. in French, but I no longer find the underline in modern documents in French. Please post here, if you find the answer yourself.
Answer: This seems to be known as an Ordinal Indicator.
Here's the blurb for convenience, some formatting lost from Wikipedia like the actual subscripts:

In written languages, an ordinal indicator is a character, or group of characters, following a numeral denoting that it is an ordinal number, rather than a cardinal number.
In English orthography, this corresponds to the suffixes -st, -nd, -rd, -th in written ordinals (represented either on the line 1st, 2nd, 3rd, 4th or as superscript, st, 2nd, 3rd, 4th).
Also commonly encountered are the superscript (and often underlined) ordinal indicators º and ª, originally from Romance, but via the cultural influence of Italian by the 18th century widely used in the wider cultural sphere of Western Europe, as in 1º primo and 1ª prima "first, chief; prime quality".
The practice of underlined (or doubly underlined) superscripted abbreviations was common in 19th-century writing (not limited to ordinal indicators in particular, and also extant in the Numero sign №), and was also found in handwritten English until at least the late 19th century (e.g. "first" abbreviated 1st or 1st).
Comment: Re answer of @htmlcoderexe (first sentence). I don't think it would be proper to call "Mc," "No," or "St" ordinal indicators. Perhaps there is no single word that means "underlined superscripted abbreviation."
Comment: @AnaNimbus as far as I have looked, there's no specific term for it. I guess that three word combination might be what you're looking for.
Answer: I'd be surprised if there were a name for this type of abbreviation. In British English, abbreviations have a full-stop after them. E.g. "Esq." for "Esquire". Contractions are less standardised. Modern practice in British English is usually to simply omit the full-stop. E.g. "Mr" for "Mister" (this distinction is rarely made in US English). Where the contraction causes a change in pronunciation, an apostrophe is used. E.g. "shan't" for "shall not". In the past, up until the late C19th, contractions were often shown by placing terminal letters in superscript and underlining usually followed by a full-stop - this was particularly the case in variants of secretary hand. This was done with a wide variety of words that we would not longer consider writing in this way - such as Mr., but also names (like "Wm. " for "William"). The practice persists in exceptional cases - notably with ordinals (e.g. "1st") and some Latin abbreviations (e.g. "no. " for "numero").
Comment: Great, do you have some source (some links perhaps?) to back this info?


Is scroll within a card good or bad? (In desktop)

Question: I am designing a board view which has card design with huge data in it. So, I am using scroll in a card. Is that a good user experience or is there any better solution?
Comment: "[Inception scrolling](http://baymard.com/blog/inline-scroll-areas)" should be avoided, according to the smart folks at Baymard, who base their design principles on research.
Comment: Additional ch[aracters] needed to meet the minimum comment length.
Comment: @KenMohnkern Inception is scrollable area within a bigger scrollable area. Here it doesn't look like there is a full-page scroll. If the small box is the only vertical scroll, there is no inception here.
Comment: @R.. what's *10ch*?
Comment: Good point, @Agent_L. I didn't catch that.
Comment: Bad. (plus 10ch)
Answer:
Desktop: You can use expand and scroll methods.

According to Material Design:

On desktop, card content can expand and scroll internally

Mobile: Don't use a scroll inside the card, instead expand it.

Card collections only scroll vertically.
Card content that exceeds the maximum card height is truncated and does not scroll, but the card can be expanded. Once expanded, the card may exceed the maximum height of the view. In this case, the card will scroll with the card collection.

Do.
Cards may be expanded to reveal more content, without using scrolling.

Don't
On mobile, avoid placing scrollable space within a card, as it could cause two sets of scroll bars to be displayed, if one is already present.
Comment: @Medinoc: Or even worse, too large. I often find nested scrollable areas that are taller than the screen, so that you have to use the outer scrollbar *and* the inner scrollbar while browsing the nested content. Utterly awful.
Comment: Also on desktop, *never* use a scrollable area that can't be resized, because it will always be too small.
Comment: You're right, I remember seeing these on the PCs of coworkers who can't seem to use Remote Desktop (of which we sometimes need multiple levels due to our partners' VPN rules) properly.
Answer: I can see you are trying to fit two scroll in a single view.
This is not preferred.This is against Affordance.
There could be two events happening there is lot of chance users could go wrong while scrolling(Swiping).
There would also be a sudden jerk when there is a change in event.
Try going for Tabs.
Hope this helps !!! Cheers
Comment: This is the use case for deskop(Not mobile)
Answer: We always talk about context with UX design (the 'it depends' factor), so there are probably times when it is appropriate to introduce scrolling into cards.
In this instance, you mentioned that this is a board so I assume it would have lots of cards inside it arranged in some kind of order, and that you are using scrolling as a way to keep the size of the cards consistent. However, this appears to be to the detriment of the user as the way you have structured and presented the content in the cards makes it overwhelming for users to have to process so much information (just imagine looking at 5-10 of those cards on a page).
The ideal way to create a clear and logical information architecture is to look at the level and depth of information that you are presenting in each card or type of card, and see if you can find the right level where you provide enough information for users to scan on a board, and let them drill into the information by directing them somewhere else rather than trying to scroll and work within the cards when there is so much space on a desktop.
For example you might just want to provide the top three most important growth metrics, and if a user clicks on like a view more link then they can get taken to a detailed information page about the company, or perhaps even a page where companies are sorted and compared by those metrics (it just depends on what the user really wants to do, or how you want them to use the information). I think these are good examples of structuring the data and aligning it to the tasks that users might want to perform rather than just overwhelming them at the first step.
Answer: Scrolling within that card with all that small type could put users off.
Have you considered having a "More information" button for the user and then have the card grow vertically to accommodate the additional information? In Material Design this expansion would typically also include increasing the z-index and shadow of the card to indicate it's expanded. Tapping anywhere on the card would send it back to its default view.
Alternatively you could expand the height of the card by clicking on a "More" link that would slide down the additional data.
Answer: Let me break down this for you before you design. Ask these questions before jumping to your photoshop or whatever you use...

Designing for desktop
Is it going to responsive so that it is also view-able on mobile?
Is there a need to show huge information inside a single card?
Are you sure about your information architecture?
Are you sure about your choice of layout?

Two additional things

Are you taking UX design decision?
Or are you taking somebody else ux input to design the UI?

Now time to answer your question
But first thing first. If you are not taking UX design decision and just participating the UI design exercise then it would be difficult for you to convince other people about your UI design decision.
Okay! lets jump to your design.
Desktop only-Is this just going to run only on desktop, no mobile? If yes, then you can proceed with this. Though still this is a bad design decision. I would tell you about this more in the bottom.
Responsive ui-
If no, then it is surely a bad design decision. "On mobile, avoid placing scrollable space within a card, as it could cause two sets of scroll bars to be displayed, if one is already present." - material design guidelines
Why do you need put so much information into that small card? If you have to then it is the sign of bad design of information architecture. You haven't put a lot of thought while structuring your information hierarchy.
Why do you need cards as you layout? are you desinging dashboard or something like that? If yes, then it is better if you think about something else. Cards are good but good for mobile. They are meant to have some action just as expand, dismiss or some other sort of CTA. Though they look good as UI on desktop to but as experience wise they are not as good as expected because cards are useful when you don't have multiple columns. For example, facebook has card ui but it only has one column of cards which is fine. I assume you would have multiple column of cards, right?If yes, then you need to think of something else. Do not let dribbble effect eat. UI's on Dribbble are eye pleasing but not usable.
Also, If it is not in your hand to take decision on this then go ahead. I will suggest some improvement. First don't put scroll-view anywhere in the card, neither on desktop nor on mobile. Rather use a CTA like to click to expand in the same view or open a modal-Window to show full information. Choose whatever you think suits better to your app.
I would probably give you better insight if you have shown us one full screen UI.
Comment: I have multiple columns to show and this would be a compact view. We are flexible with the height but not width. It has to go on mobile as well.


What is bit time in Ethernet CSMA/CD

Question: "Propogation delay betwween two links is 200 bit times"
I am little confused here, Propogation delay should be in seconds, milliseconds etc but here it is in bit times. I also want to know what is bit time and is it same as Propogation delay.
Comment: Maybe this one https://en.wikipedia.org/wiki/Bit_time ?
Answer: Bit time only calculates the ejection of one bit, it looks at how this bit transmits out of a network interface card at a determined speed.
$$bit\_time = \frac{1}{NIC\_speed}$$
let NIC speed is $10Mbit/s$ then
bit time = $$\frac{1}{(10 * 10^6)}
                 = 10^{-7}
                 = 100 * 10^{-9}
                 = 100ns$$
Bit rate is different form propagation delay, as the propagation delay is the amount of time it takes for the head of the signal to travel from the sender to the receiver and bit rate is the number of bits that are conveyed or processed per unit of time.
Comment: Ok but i have found one questio which says "The Propogation delay betwween A and B is 200 bit times" . but Propogation delay should be in seconds or millsec etc
Comment: propagation delay is $x$ bit times that means propagation delay equals to the amount of time to transmit $x$ bits.
Answer: There are constraints on propagation delays expressed as "bit time" because the protocol, the collision detection, depends on the time each message lasts on the wires.
Because the emitter must know if there is a collision to be able to re-transmit messages, having very long propagation delays, longer than message lengths, could hide collisions from the emitter.
This is not an absolute value as it is proportional to the data rate : Faster network need a smaller propagation delay than slower networks for CSMA/CD to work.
The automotive CAN protocol has similar constraints: the maximum data rate depends on the total length of the network for correct master arbitration.
You want values ?
A 10Mbps Ethernet network has a nominal bit time of 100ns
A 1GHz Ethernet network has a nominal bit time of 1ns, but, because of encoding, there is no literally bits pushed serially at 1GHz : Sometimes it is 1.25GHz over optical fibres, sometimes it is a several bits per cycle at 125MHz over four twisted pairs, etc...
Comment: I think you've misunderstood the question. As I see it, the question is "What does the term 'bit time' mean?" but you seem to be answering "How many seconds is 'one bit time'?"

What is bit time in Ethernet CSMA/CD


How to learn object oriented Python?

Question: What would one suggest for a junior lever programming student to learn object oriented Python? For me, it is quite easy to understand objects in Java and non-object oriented Python but objects in Python seems weirder than in Java. Are there good books or lecture notes?
Comment: Weirder in what way? The object models are very similar. Python misses explicit interfaces, of course. The big difference is when a reference is checked for validity.
Answer: The object models of Java and Python are just about the same. But the reference model is a bit different. But the biggest difference is that in Java, a reference variable is defined to have a certain type (class) and in Python it is just a name. In other words, in Java, references are typed. In Python they are untyped. But in both languages the objects themselves are strongly typed at time of creation.
This difference implies that reference types (validity) are checked by the compiler - or even earlier when a good IDE is used. Thus Eclipse will highlight type errors in Java but cannot do so in Python. In Python, the validity of an expression is checked only when the program actually runs since any variable can actually refer to any object, regardless of type.
In Python if you say foo.bar(x); then this is valid or not depending on whether, at that moment the object that foo references has a method bar with appropriate parameters. The actual type of that object doesn't matter, only its available methods do. But the object itself, as distinct from the reference to it, has a specific type that doesn't change.
But in Java, this gets checked before the program runs, making error detection earlier in the process. If the declared class of foo, which is explicit in the program text has a matching bar method then the expression is guaranteed valid prior to the run.
For the programmer, this implies that you need to take more care in writing your program in Python since you get fewer hints from the IDE or the compiler if you don't use an IDE. Thus thorough testing becomes even more important in Python since untested code can harbor errors for a long time undetected.
I would hope that any decent undergraduate level programming text in either language will give you proper guidance and a mental model for that language.
Comment: The most obvious difference between Python and Java is multiple inheritance, but since Python uses MI with *linearization*, there is still a single, well-defined, linear chain of ancestors for each class, just like in Java.
Comment: @JörgWMittag, Java has multiple inheritance? Since when? Implementing multiple interfaces isn't the same thing.
Comment: Java doesn't, but Python does, hence it's (at least to me) the most obvious difference between Java and Python.
Comment: I say "Java is a Victorian convent and Python is a nudist colony," because Java has a much stronger form of encapsulation in the form of access specifiers.  There is no compiler to enforce privacy in Python.  This amounts to a very significant difference.  Python has no notion of finality either.

How to learn object oriented Python?


Whats the difference declaring variables with and without prototypes

Question: Whats the difference between (via prototypes)
var Todo = {};

Todo.prototype.name = "...";
Todo.prototype.hello = function() { ... }

Vs (variables & functions "outside" object)
var Todo = {}
Todo.name = "..."
Todo.hello = function() { ... }

Or even the below : variables & functions in object
var Todo = {
    name: "...",
    hello = function() { ... }
}

Comment: example 2 and 3 are almost identical
Comment: The first one doesn't even work since object-instances don't have a prototype property by default.
Answer: The first one doesn't make sense as you are dealing with an object instance (({}) instanceof Object === true), it won't have a prototype property (Object does).
You may be inquiring about the difference between these two patterns...
var ObjA = function() {
    this.method = function() {};
};

var ObjB = function() {};

ObjB.prototype.method = function() {};

jsFiddle.
The former will use more memory when instantiated - each object has their own method. The latter won't each have their own method, the method lives on the prototype object, which is the next in command on the prototype chain when its attempted to be accessed on the parent.
Answer: Think it like
A property or a function declared with prototype is an instance member of Todo.
A property or a function declared without prototype is a static member of Todo.
Answer: Todo.prototype is also an object, so the difference is if you declare property with prototype, then every object who created from this prototype will have the property, otherwise, the property is only for Todo the object self.
Answer: A significant difference between method #1 and #2 (which is almost identical to example #3) is on new keyword that you need to use if you extend your function via prototype, e.g.
var Todo1 = function() {};
Todo1.prototype.name = "Foobar";

var Todo2 = {name: "Foobar" }

var a = Todo1;
console.log(a.name); // no property retrieved

var b = Todo2;
console.log(b.name); // Foobar

var c = new Todo1;
console.log(c.name); // Foobar


What is this "johnson controls" device on the wall in airport?

Question: I was passing column in waiting area of Pearson airport when my sweater cought some kind of device on the wall. Device was without cover and looks like old thermostat. When my sweater caught it, it fell off and started to blow air (I think). Maybe someone knows what is this device and why it blows compressed air? Yes, I informed airport employee.
Comment: Unfortunately no. First looking for airport employee, then handling pair of overexcited kids at 5am in the morning  I somehow missed taking picture of it.
Comment: Do you have any picture of it?
Answer: It might have been a old thermostat.  In ancient times, thermostats sometimes worked pneumatically.  When the temperature got too low, the thermostat would open a valve that released the pressure on some pressurized air.  This would then directly actuate the valve on the radiator.
There was a small bleed system so that when the system was closed, it would pressurize over a minute or so.  The valve in the thermostat was significantly larger, so that when that valve was open, the pressure would be near ambient.
This kind of system was common in large buildings where a boiler would be running most of the time, since some zone somewhere always wanted heat.  The hot water or steam was always available, so the thermostat and valve combinations controlled whether it would flow thru the radiator for that zone only.
Comment: l will be returning in 4 days, maybe I'll see them in arrival area as well, so I can make picture. Thanks for answer, it looks like what it really might be.