<<<<<<< HEAD
# -*- coding: utf-8 -*-
import math

__all__ = ['AIC', 'AICc', 'BIC']


def AIC(L, k):
    """Return Akaike information criterion (AIC)

    :param int k: number of parameters
    :param float L: maximised value of the likelihood function
    
    
    Suppose that we have a statistical model of some data, from which we computed
    its likelihood function and let k be the number of parameters in the model 
    (i.e. degrees of freedom). Then the AIC value is ::

    :math:`\mathrm{AIC} = 2k - 2\ln(L)`

    Given a set of candidate models for the data, the preferred model is the one 
    with the minimum AIC value. Hence AIC rewards goodness of fit (as assessed 
    by the likelihood function), but it also includes a penalty that is an 
    increasing function of the number of estimated parameters. The penalty 
    discourages overfitting.

    Suppose that there are R candidate models AIC1, AIC2, AIC3, AICR. 
    Let AICmin be the minimum of those values. Then, exp((AICmin - AICi)/2)
    can be interpreted as the relative probability that the ith model 
    minimizes the (estimated) information loss.

    Suppose that there are three candidate models, whose AIC values are 100, 
    102, and 110. Then the second model is exp((100 - 102)/2) = 0.368 times 
    as probable as the first model to minimize the information loss. Similarly,
    the third model is exp((100 - 110)/2) = 0.007 times as probable as
    the first model, which can therefore be discarded.

    With the remaining two models, we can (1) gather more data, (2) conclude 
    that the data is insufficient to support selecting one model from among 
    the first two (3) take a weighted average of the first two models, 
    with weights 1 and 0.368.

    The quantity exp((AICmin - AICi)/2) is the relative likelihood of model i.

    If all the models in the candidate set have the same number of parameters, 
    then using AIC might at first appear to be very similar to using the 
    likelihood-ratio test. There are, however, important distinctions. 
    In particular, the likelihood-ratio test is valid only for nested models, 
    whereas AIC (and AICc) has no such restriction.

    Reference: Burnham, K. P.; Anderson, D. R. (2002), Model Selection and 
        Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.), 
        Springer-Verlag, ISBN 0-387-95364-7.
    """
    return 2*k -2 * math.log(L)


def AICc(L, k, n):
    """AICc criteria
    
    :param int k: number of parameters
    :param int n: sample size
    :param float L: maximised value of the likelihood function

    AIC with a correction for finite sample sizes. 
    The formula for AICc depends upon the statistical model. 
    Assuming that the model is univariate, linear, and has normally-distributed 
    residuals (conditional upon regressors), the formula for AICc is as follows:

    AICc is essentially AIC with a greater penalty for extra parameters. 
    Using AIC, instead of AICc, when n is not many times larger than k2, increases 
    the probability of selecting models that have too many parameters, i.e. of 
    overfitting. The probability of AIC overfitting can be substantial, in some cases.

    """
    res = AIC(L, k) + 2*k*(k+1.) / (n-k-1.)
    return res



def BIC(L, k, n):
    """Bayesian information criterion 
    
    
    Given any two estimated models, the model with the lower value of BIC is the one to be preferred. 
    """

    res = -2 * math.log(L) + k * (math.log(n) - math.log(2 * math.pi))
    # For large n
    #res = -2 * math.log(L) + k * math.log(n)
    return res


=======
import math


def AIC(L, k):
    return 2*k - 2 * math.log(L)


def AICc(L, k, n):
    return AIC(L, k) + 2*k*(k+1.)/(n-k-1.)


def BIC(L, k, n):
    return -2 * math.log(L) + k * (math.log(n) - math.log(2*math.pi))
>>>>>>> 514a04b5ffa7c9e3ede068c860933e9a404e6063


update full version of criteria module.