<commit_before>=======
RLScore
=======


RLScore - regularized least-squares based machine learning algorithms
for regression, classification, ranking, clustering, and feature selection.


:Authors:         `<NAME> <http://staff.cs.utu.fi/~aatapa/>`_,
                  `<NAME> <http://tucs.fi/education/phd/alumni/student.php?student=120>`_
:Email:           <EMAIL>
:Homepage:        `http://staff.cs.utu.fi/~aatapa/software/RLScore/ <http://staff.cs.utu.fi/~aatapa/software/RLScore/>`_
:Version:         0.5.1
:License:         `The MIT License <LICENCE.TXT>`_
:Date:            2012.06.19


.. contents::


Overview
========

RLScore is a Regularized Least-Squares (RLS) based algorithm
package. It contains implementations of the RLS and RankRLS
learners allowing the optimization of performance measures for the
tasks of regression, ranking and classification. In addition,
the package contains  linear time greedy forward feature selection
with leave-one-out criterion for RLS (greedy RLS). Finally, 
the package contains an implementation of a maximum margin
clustering method based on RLS and stochastic hill climbing.
Implementations of efficient cross-validation algorithms are
integrated to the package, combined together with functionality
for fast parallel learning of multiple outputs.

Reduced set approximation for large-scale learning with kernels is
included. In this setting approximation is introduced also to the
cross-validation methods. For learning linear models from large but
sparse data sets, RLS and RankRLS can be trained using conjugate
gradient optimization techniques.

Support for different tasks
===========================


-  Regression
   
   -  Directly optimizes for the squared loss using the RLS algorithm
   -  fast n-fold cross-validation algorithm, where folds of arbitrary
      sizes can be left out of the training set
   -  fast leave-one-out (LOO) cross-validation algorithm
   -  efficient searching for the regularization parameter

-  Classification
   
   -  optimizes for accuracy using the RLS algorithm
   -  optimizes for AUC (Area Under the ROC Curve) using the RankRLS
      algorithm
   -  fast n-fold cross-validation algorithm, where folds of arbitrary
      sizes can be left out of the training set
   -  fast leave-one-out (LOO) cross-validation algorithm for accuracy
      optimization
   -  fast leave-pair-out (LPO) cross-validation algorithm for AUC
      optimization
   -  efficient searching for the regularization parameter

-  Ranking

   -  directly optimizes for the squared magnitude preserving ranking
      loss using RankRLS algorithm
   -  optimizes for disagreement error using the RankRLS algorithm
   -  fast label ranking cross-validation algorithm (leave-query-out)
   -  fast leave-pair-out (LPO) cross-validation algorithm for object
      ranking

-  Clustering

   -  given an unlabeled set of data, performs a stochastic hill
      climbing based search for cluster labels, while using RLS as a
      criterion
   -  supports user-given preliminary cluster labels in addition to
      random ones

-  Feature selection

   -  the feature selection algorithm in RLScore, greedy RLS, can be
      used to learn sparse linear RLS predictors efficiently, that is,
      the time complexity of Greedy RLS is linear in the number of
      training data, the number of features in the original data
      set, and the desired size of the set of selected features.
   -  greedy RLS starts from an empty feature set, and on each
      iteration adds the feature whose addition provides the best
      leave-one-out cross-validation performance.


Download
========

Download `RLScore.zip <RLScore.zip>`_ containing the python source code of RLScore.


Software dependencies
=====================

RLScore is written in Python and thus requires a working
installation of Python 2.8.x. The package is also dependent on
the `NumPy 1.3.x <http://numpy.scipy.org/>`_ package for matrix
operations, and `SciPy 0.13.x <http://www.scipy.org/>`_ package for sparse
matrix implementations.

Usage
=====
RLScore is designed to be used as a software library, called directly from Python code.

The easiest way to use RLScore is by modifying one of the example python codes delivered
with the distribution, to match your task. The software supports a wide variety of
different learning tasks, ranging from supervised learning to clustering and feature
selection. 


Examples
========

Examples of typical use-cases for each type of task are provided below.

The example code files, and the example data sets used by them can be found
in the 'examples' folder of the RLScore distribution. For example, to run the
example file 'rls_classification.py' included in examples/code from the command line,
go to the folder containing the RLScore distribution, and execute the command
'python examples/code/rls_classification.py'

While the examples use Unix-style paths with '/' separator,
they should also work in Windows with no modifications needed.


Binary classification, maximize accuracy
----------------------------------------

In binary classification, the data is separated into two classes. Classification
accuracy measures the fraction of correct classifications made by the learned
classifier. This is perhaps the most widely used performance measure for binary
classification. However, for very unbalanced data-sets it may be preferable to
optimize the area under ROC curve (AUC) measure, considered in a later example,
instead.

When training a classifier according to the accuracy criterion, using the
RLS module which minimizes a least squares loss on the training set class
labels is recommended. The approach is equivalent to the so-called
least-squares support vector machine. 

Requirements:
- class labels should be either 1 (positive) or -1 (negative)


Python code (rls_classification) `relative link`_.


.. _relative link: examples/code/rls_classification.py
<commit_msg>Testing.<commit_after>=======
RLScore
=======


RLScore - regularized least-squares based machine learning algorithms
for regression, classification, ranking, clustering, and feature selection.


:Authors:         `<NAME> <http://staff.cs.utu.fi/~aatapa/>`_,
                  `<NAME> <http://tucs.fi/education/phd/alumni/student.php?student=120>`_
:Email:           <EMAIL>
:Homepage:        `http://staff.cs.utu.fi/~aatapa/software/RLScore/ <http://staff.cs.utu.fi/~aatapa/software/RLScore/>`_
:Version:         0.5.1
:License:         `The MIT License <LICENCE.TXT>`_
:Date:            2012.06.19


.. contents::


Overview
========

RLScore is a Regularized Least-Squares (RLS) based algorithm
package. It contains implementations of the RLS and RankRLS
learners allowing the optimization of performance measures for the
tasks of regression, ranking and classification. In addition,
the package contains  linear time greedy forward feature selection
with leave-one-out criterion for RLS (greedy RLS). Finally, 
the package contains an implementation of a maximum margin
clustering method based on RLS and stochastic hill climbing.
Implementations of efficient cross-validation algorithms are
integrated to the package, combined together with functionality
for fast parallel learning of multiple outputs.

Reduced set approximation for large-scale learning with kernels is
included. In this setting approximation is introduced also to the
cross-validation methods. For learning linear models from large but
sparse data sets, RLS and RankRLS can be trained using conjugate
gradient optimization techniques.

Support for different tasks
===========================


-  Regression
   
   -  Directly optimizes for the squared loss using the RLS algorithm
   -  fast n-fold cross-validation algorithm, where folds of arbitrary
      sizes can be left out of the training set
   -  fast leave-one-out (LOO) cross-validation algorithm
   -  efficient searching for the regularization parameter

-  Classification
   
   -  optimizes for accuracy using the RLS algorithm
   -  optimizes for AUC (Area Under the ROC Curve) using the RankRLS
      algorithm
   -  fast n-fold cross-validation algorithm, where folds of arbitrary
      sizes can be left out of the training set
   -  fast leave-one-out (LOO) cross-validation algorithm for accuracy
      optimization
   -  fast leave-pair-out (LPO) cross-validation algorithm for AUC
      optimization
   -  efficient searching for the regularization parameter

-  Ranking

   -  directly optimizes for the squared magnitude preserving ranking
      loss using RankRLS algorithm
   -  optimizes for disagreement error using the RankRLS algorithm
   -  fast label ranking cross-validation algorithm (leave-query-out)
   -  fast leave-pair-out (LPO) cross-validation algorithm for object
      ranking

-  Clustering

   -  given an unlabeled set of data, performs a stochastic hill
      climbing based search for cluster labels, while using RLS as a
      criterion
   -  supports user-given preliminary cluster labels in addition to
      random ones

-  Feature selection

   -  the feature selection algorithm in RLScore, greedy RLS, can be
      used to learn sparse linear RLS predictors efficiently, that is,
      the time complexity of Greedy RLS is linear in the number of
      training data, the number of features in the original data
      set, and the desired size of the set of selected features.
   -  greedy RLS starts from an empty feature set, and on each
      iteration adds the feature whose addition provides the best
      leave-one-out cross-validation performance.


Download
========

Download `RLScore.zip <RLScore.zip>`_ containing the python source code of RLScore.


Software dependencies
=====================

RLScore is written in Python and thus requires a working
installation of Python 2.8.x. The package is also dependent on
the `NumPy 1.3.x <http://numpy.scipy.org/>`_ package for matrix
operations, and `SciPy 0.13.x <http://www.scipy.org/>`_ package for sparse
matrix implementations.

Usage
=====
RLScore is designed to be used as a software library, called directly from Python code.

The easiest way to use RLScore is by modifying one of the example python codes delivered
with the distribution, to match your task. The software supports a wide variety of
different learning tasks, ranging from supervised learning to clustering and feature
selection. 


Examples
========

Examples of typical use-cases for each type of task are provided below.

The example code files, and the example data sets used by them can be found
in the 'examples' folder of the RLScore distribution. For example, to run the
example file 'rls_classification.py' included in examples/code from the command line,
go to the folder containing the RLScore distribution, and execute the command
'python examples/code/rls_classification.py'

While the examples use Unix-style paths with '/' separator,
they should also work in Windows with no modifications needed.


Binary classification, maximize accuracy
----------------------------------------

In binary classification, the data is separated into two classes. Classification
accuracy measures the fraction of correct classifications made by the learned
classifier. This is perhaps the most widely used performance measure for binary
classification. However, for very unbalanced data-sets it may be preferable to
optimize the area under ROC curve (AUC) measure, considered in a later example,
instead.

When training a classifier according to the accuracy criterion, using the
RLS module which minimizes a least squares loss on the training set class
labels is recommended. The approach is equivalent to the so-called
least-squares support vector machine. 

Requirements:
- class labels should be either 1 (positive) or -1 (negative)


Python code (`rls_classification.py`_)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. _rls_classification.py: examples/code/rls_classification.py

{'fraction_non_alphanumeric': 0.073943956503555, 'fraction_numerical': 0.0038477624424926807, 'mean_word_length': 3.748212867355044, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6733479', 'n_tokens_mistral': 3123, 'n_tokens_neox': 2885, 'n_words': 1455}