<commit_before>MADlib Release Notes
--------------------

These release notes contain the significant changes in each MADlib release,
with most recent versions listed at the top.

A complete list of changes for each release can be obtained by viewing the git
commit history located at https://github.com/madlib/madlib/commits/master.

Current list of bugs and issues can be found at http://jira.madlib.net.
--------------------------------------------------------------------------------
MADlib v1.0

Release Date: 2013-July-03

New Features:
* Cox Proportional Hazards:
    - Added Right Censoring support for Cox Prop Hazards
* Robust Variance Tests - Huber White:
    - Added a method of calculating robust variance statistic by utilizing the
    Huber-White sandwich estimator for linear regression, logistic regression,
    and multinomial logistic regression
    - Robust variance for linear and logistic regression also includes
    grouping support
* Clustered Sandwich Estimators:
    - Added clustered robust variance statistic by utilizing a clustered sandwich
    estimator for linear regression, logistic regression, and multinomial
    logistic regression
    - Grouping is currently not implemented for clustered and parameter is only
    a placeholder at present
* Marginal Effects Estimator:
    - Added a method for computing the marginal effects for logistic regression<commit_msg>Release notes: Updates for v1.1<commit_after>MADlib Release Notes
--------------------

These release notes contain the significant changes in each MADlib release,
with most recent versions listed at the top.

A complete list of changes for each release can be obtained by viewing the git
commit history located at https://github.com/madlib/madlib/commits/master.

Current list of bugs and issues can be found at http://jira.madlib.net.
--------------------------------------------------------------------------------
MADlib v1.1

Release Date: 2013-August-09

New Features:
* Singular Value Decomposition:
    - Added Singular Value Decomposition using the Lanczos bidiagonalization
    iterative method to decompose the original matrix into PBQ^t, where B is
    a bidiagonalized matrix. We assume that the original matrix is too big to
    load into memory but B can be loaded into the memory. B is then further
    decomposed into XSY^t using Eigen's JacobiSVD function. This restricts the
    number of features in the data matrix to about 5000.
    - This implementation provides SVD (for dense matrix), SVD_BLOCK (also for
    dense matrix but faster), SVD_SPARSE (convert a sparse matrix into a
    dense one, slower) and SVD_SPARSE_NATIVE (directly operate on the sparse
    matrix, much faster for really sparse matrices).

* Principal Component Analysis:
    - Added a PCA training function that generates the top-K principal
    components for an input matrix. The original data is mean-centered by the
    function with the mean matrix returned by the function as a separate table.
    - The module also includes the projection function that projects a test data
    set to the principal components returned by the train function.

* Linear Systems:
    - Added a module to solve linear system of equations (Ax = b).
    - The module utilizes various direct methods from the Eigen library for
    dense systems. Given below is a summary of the methods (more details at
    http://eigen.tuxfamily.org/dox-devel/group__TutorialLinearAlgebra.html):
        - Householder QR
        - Partial Pivoting LU
        - Full Pivoting LU
        - Column Pivoting Householder QR
        - Full Pivoting Householder QR
        - Standard Cholesky decomposition (LLT)
        - Robust Cholesky decomposition (LDLT)
    - The module also includes direct and iterative methods for sparse linear
    systems:
        Direct:
            - Standard Cholesky decomposition (LLT)
            - Robust Cholesky decomposition (LDLT)
        Iterative:
            - In-memory Conjugate gradient
            - In-memory Conjugate gradient with diagonal preconditioners
            - In-memory Bi-conjugate gradient
            - In-memory Bi-conjugate gradient with incomplete LU preconditioners

Bug fixes and other changes:
* Robust input validation:
    - Validation of input parameters to various functions has been improved to
    ensure that it does not fail if double quotes are included as part of the
    table name.
* Random Forest
    - The ID field in rf_train has been expanded from INT to BIGINT (MADLIB-764)
* Various documentation updates:
    - Documentation updated for various modules including elastic net, linear
    and logistic regression.
--------------------------------------------------------------------------------
MADlib v1.0

Release Date: 2013-July-03

New Features:
* Cox Proportional Hazards:
    - Added Right Censoring support for Cox Prop Hazards
* Robust Variance Tests - Huber White:
    - Added a method of calculating robust variance statistic by utilizing the
    Huber-White sandwich estimator for linear regression, logistic regression,
    and multinomial logistic regression
    - Robust variance for linear and logistic regression also includes
    grouping support
* Clustered Sandwich Estimators:
    - Added clustered robust variance statistic by utilizing a clustered sandwich
    estimator for linear regression, logistic regression, and multinomial
    logistic regression
    - Grouping is currently not implemented for clustered and parameter is only
    a placeholder at present
* Marginal Effects Estimator:
    - Added a method for computing the marginal effects for logistic regression
{'fraction_non_alphanumeric': 0.08481192334989354, 'fraction_numerical': 0.005855216465578425, 'mean_word_length': 2.813937753721245, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2046064', 'n_tokens_mistral': 1415, 'n_tokens_neox': 1279, 'n_words': 672}