I am implementing the Naive Bayes method with Gaussian distribution. The problem that I have is that the variance used on the Gaussian curve (calculated from a training set) is REALLY small. They are on the order of e-07. That means the whole equation,(1/:math.sqrt(2*:math.pi*:math.sqrt(variance))) *:math.exp(-0.5*(:math.pow(elem - mean, 2) / variance))results in really high values (such as 500, or even more). That becomes a problem later, when I multiply every probability p(x|C) together (it is a vector of 256 features). I heard that is possible to use logarithms to avoid these kind of numbers. I searched on google but didn't find anything related to the subject. Does anyone know about that?
Avoid numeric errors in Naive Bayes method
algorithms;numeric precision
Working with log-probabilities helps with the problem of intermediate values overflowing (and underflowing). Instead of calculating L = P(x1|C)P(x2|C)..., you calculate log L = log P(x1|C) + log P(x2|C) + .... You can do two things with log L:If you're trying to maximize likelihood, you can directly maximize log-likelihood instead since log is monotonic and increasing.If you want a normalized probability L/Z in [0,1], you can calculate this as exp(log L - log Z), which is possible even if L and Z are too large to fit in your floating point type.In my experience this trick is often necessary when implementing probabilistic graphical models.