<commit_before>﻿PyFactorizationMachines
=======================

A Theano-based Python implementation of factorization machines, based on the model presented in *Factorization Machines*
(Rendle 2010).

Dependencies
------------

PyFactorizationMachines requires Python 2.7 with numpy and theano.

The packages can be installed with pip.

    $ pip install numpy theano
    
Alternatively, these may be available through your operating system's package manager, or a package manager for your
Python environment (e.g., conda).

Additionally, scikit-learn is required for running example.py, which shows example usage of PyFactorizationMachines.

    $ pip install scikit-learn

Features
--------

(Rendle 2010) suggests stochastic gradient descent for optimizing the loss function. This implementation offers
various algorithms for searching for the optimal parameters.

For binary classification, this implementation uses a logit function combined with a cross entropy loss function.

This implementation offers sample weighting.

Regularizatoin and loss function optimization are both extensible.

How To Use
----------

To use PyFactorizationMachines, first import the *pyfm* module.

    >>> import pyfm
    
### Initializing a Model

A factorization machine is created with *FactorizationMachineClassifier* for a binary classification problem, or
*FactorizationMachineRegressor* for regression. The constructors require an argument specifying the number of features.

    >>> model = pyfm.FactorizationMachineRegressor(feature_count)

FactorizationMachineRegressor and FactorizationMachineClassifier both take the following arguments:

* **feature_count** The dimensionality of each data point.
* **k** (optional; defaults to 8) Dimensionality of the factorization of pairwise interactions.
* **stdev** (optional; defaults to .01) The standard deviation of the normal distribution used to initialize the
interaction parameters of the model.
    
### Training a Model

A factorization machine is trained with the *fit* method, which takes as input a training dataset X, and a vector of
target values. For binary classification, the target values must be 1 or 0.

    >>> model.fit(X, y)

*fit* takes the following arguments:

* **X** An *n-by-d* numpy.ndarray with training data. The rows correspond to observations, and the columns correspond to
dimensions.
* **y** A numpy.ndarray vector with *n* target values corresponding to the *n* data points in *X*.
* **optimizer** (optional; defaults to RMSProp()) An object of a class that extends *_Optimizer*, which specifies how
to optimize the loss function.
* **regularizer** (optional; defaults to None) An object of a class that extends *_Regularizer*, which specifies how
to regularize the loss function. For example, see L2, which implements L2 regularization.
* **sample_weight** (optional; defaults to None) A numpy.ndarray vector with *n* sample weights corresponding to the
*n* data points in *X*.
* **batch_size** (optional; defaults to 32) Number of samples per gradient update.
* **nb_epoch** (optional; defaults to 10)  The number of epochs to train the model.
* **shuffle** (optional; defaults to True) A flag indicating whether to shuffle the training samples at each epoch.
* **verbose** (optional; defaults to False) A flag specifying whether to log details to stdout when training the model.
* **memory** (optional; defaults to True) If False, the last set of weights from training will be retained. If True,
the set of weights that minimized the loss function (across epochs) will be retained.

### Predicting with a FactorizationMachine

For regression models, *FactorizationMachineRegressor.predict* can be used to predict the regression targets for a
dataset X.

    >>> model.predict(X)

For binary classification models, *FactorizationMachineClassifier.predict* can be used to predict the class targets for
a dataset X. *FactorizationMachineClassifier.predict_proba* returns the class 1 probability for a dataset X.

    >>> model.predict(X)
    >>> model.predict_proba(X)

*FactorizationMachineRegressor.predict*, *FactorizationMachineClassifier.predict*, and
*FactorizationMachineClassifier.predict_proba* take the following argument:

* **X** An *n-by-d* numpy.ndarray with data. The rows correspond to observations, and the columns correspond to
dimensions.

### Saving a FactorizationMachine

A factorization machine is saved with the *save_weights* method, which takes as input a filename.

    >>> model.save_weights("/path/to/model.fm")

*save_weights* takes the following arguments:

* **path** The file path for saving the model weights.

### Loading a Saved FactorizationMachine

A saved factorization machine can be loaded with the *load_weights* method.
    
    >>> fm = pyfm.FactorizationMachineRegressor(feature_count)
    >>> fm.load_weights("/path/to/model.fm")

*load_weights* takes the following arguments:

* **path** The file path for loading the model weights.

License
-------

PyFactorizationMachines has an [MIT License](https://en.wikipedia.org/wiki/MIT_License).

See [LICENSE](LICENSE).

Acknowledgments
---------------

RMSprop code is from [Newmu/Theano-Tutorials](https://github.com/Newmu/Theano-Tutorials/blob/master/4_modern_net.py).

References
----------
<commit_msg>Use same variable names as earlier.<commit_after>﻿PyFactorizationMachines
=======================

A Theano-based Python implementation of factorization machines, based on the model presented in *Factorization Machines*
(Rendle 2010).

Dependencies
------------

PyFactorizationMachines requires Python 2.7 with numpy and theano.

The packages can be installed with pip.

    $ pip install numpy theano
    
Alternatively, these may be available through your operating system's package manager, or a package manager for your
Python environment (e.g., conda).

Additionally, scikit-learn is required for running example.py, which shows example usage of PyFactorizationMachines.

    $ pip install scikit-learn

Features
--------

(Rendle 2010) suggests stochastic gradient descent for optimizing the loss function. This implementation offers
various algorithms for searching for the optimal parameters.

For binary classification, this implementation uses a logit function combined with a cross entropy loss function.

This implementation offers sample weighting.

Regularizatoin and loss function optimization are both extensible.

How To Use
----------

To use PyFactorizationMachines, first import the *pyfm* module.

    >>> import pyfm
    
### Initializing a Model

A factorization machine is created with *FactorizationMachineClassifier* for a binary classification problem, or
*FactorizationMachineRegressor* for regression. The constructors require an argument specifying the number of features.

    >>> model = pyfm.FactorizationMachineRegressor(feature_count)

FactorizationMachineRegressor and FactorizationMachineClassifier both take the following arguments:

* **feature_count** The dimensionality of each data point.
* **k** (optional; defaults to 8) Dimensionality of the factorization of pairwise interactions.
* **stdev** (optional; defaults to .01) The standard deviation of the normal distribution used to initialize the
interaction parameters of the model.
    
### Training a Model

A factorization machine is trained with the *fit* method, which takes as input a training dataset X, and a vector of
target values. For binary classification, the target values must be 1 or 0.

    >>> model.fit(X, y)

*fit* takes the following arguments:

* **X** An *n-by-d* numpy.ndarray with training data. The rows correspond to observations, and the columns correspond to
dimensions.
* **y** A numpy.ndarray vector with *n* target values corresponding to the *n* data points in *X*.
* **optimizer** (optional; defaults to RMSProp()) An object of a class that extends *_Optimizer*, which specifies how
to optimize the loss function.
* **regularizer** (optional; defaults to None) An object of a class that extends *_Regularizer*, which specifies how
to regularize the loss function. For example, see L2, which implements L2 regularization.
* **sample_weight** (optional; defaults to None) A numpy.ndarray vector with *n* sample weights corresponding to the
*n* data points in *X*.
* **batch_size** (optional; defaults to 32) Number of samples per gradient update.
* **nb_epoch** (optional; defaults to 10)  The number of epochs to train the model.
* **shuffle** (optional; defaults to True) A flag indicating whether to shuffle the training samples at each epoch.
* **verbose** (optional; defaults to False) A flag specifying whether to log details to stdout when training the model.
* **memory** (optional; defaults to True) If False, the last set of weights from training will be retained. If True,
the set of weights that minimized the loss function (across epochs) will be retained.

### Predicting with a FactorizationMachine

For regression models, *FactorizationMachineRegressor.predict* can be used to predict the regression targets for a
dataset X.

    >>> model.predict(X)

For binary classification models, *FactorizationMachineClassifier.predict* can be used to predict the class targets for
a dataset X. *FactorizationMachineClassifier.predict_proba* returns the class 1 probability for a dataset X.

    >>> model.predict(X)
    >>> model.predict_proba(X)

*FactorizationMachineRegressor.predict*, *FactorizationMachineClassifier.predict*, and
*FactorizationMachineClassifier.predict_proba* take the following argument:

* **X** An *n-by-d* numpy.ndarray with data. The rows correspond to observations, and the columns correspond to
dimensions.

### Saving a FactorizationMachine

A factorization machine is saved with the *save_weights* method, which takes as input a filename.

    >>> model.save_weights("/path/to/model.fm")

*save_weights* takes the following arguments:

* **path** The file path for saving the model weights.

### Loading a Saved FactorizationMachine

A saved factorization machine can be loaded with the *load_weights* method.
    
    >>> model = pyfm.FactorizationMachineRegressor(feature_count)
    >>> model.load_weights("/path/to/model.fm")

*load_weights* takes the following arguments:

* **path** The file path for loading the model weights.

License
-------

PyFactorizationMachines has an [MIT License](https://en.wikipedia.org/wiki/MIT_License).

See [LICENSE](LICENSE).

Acknowledgments
---------------

RMSprop code is from [Newmu/Theano-Tutorials](https://github.com/Newmu/Theano-Tutorials/blob/master/4_modern_net.py).

References
----------

{'fraction_non_alphanumeric': 0.0893025893025893, 'fraction_numerical': 0.004347004347004347, 'mean_word_length': 4.692845615922539, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 57, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4274644', 'n_tokens_mistral': 2790, 'n_tokens_neox': 2615, 'n_words': 1233}