The term `Entropy' was first introduced by [1]} who laid the foundations of information theory. Intuitively, it measures the unpredictability of a system.
The entropy \(H\)  of a discrete random variable \(X\)  with possible values \(\lbrace x_1, ..., x_n\rbrace \)  and probability mass function \(P(X)\)  (a probability distribution) is
\(H(X) = -\sum _{i=1}^{n} P(x_i) \log _bP(x_i)\) 
