<commit_before>CS 5860 - Naive Bayes Classifier
================================

    <NAME>
    University of Colorado - Colorado Springs
    CS 5910 - Computer/Network Security

## Assignment
 > Write a program in a language of your choice that classifies datasets into two classes. The two classes
here are _<NAME>_ and _<NAME>._

### [Assignment Details](./ASSIGNMENT.pdf)

## Dataset
In addition to the required Dickens and Hardy books, some additional datasets were taken from [UCI - Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets.html). The datasets used are described below.

**Datasets used, and their location in this project:**

**Dataset** | **Source** | **Path** | * **Type**
---|---|---|---
SMS | [UCI - SMS Spam Collection](https://archive.ics.uci.edu/ml/datasets/SMS+Spam+Collection) | `./data/sms` | inline
Badges | [UCI - Badges](https://archive.ics.uci.edu/ml/datasets/Badges) | `./data/badges` | inline
Main | Gutenberg - [Dickens](http://www.gutenberg.org/ebooks/author/37), [Hardy](http://www.gutenberg.org/ebooks/author/23) | gutenberg

* **Dataset Types:**

**Type** | **Description**
---|---
inline | Dataset is stored as a single file in which each line represents a training point. The first word in each line is the class/category, while the rest of the line is a list of words used as the training "text blob."
gutenberg | Dataset is stored as a list of directories representing classes/categories (e.g. "dickens", "hardy"). Each file within the class directories represent a training point. These files are actually books, but are abstractly considered to be "text blobs," just like the **inline** dataset type.<commit_msg>tweak readme<commit_after>CS 5860 - Naive Bayes Classifier
================================

    <NAME>
    University of Colorado - Colorado Springs
    CS 5910 - Computer/Network Security

## Assignment
 > Write a program in a language of your choice that classifies datasets into two classes. The two classes
here are _<NAME>_ and _<NAME>._

### [Assignment Details](./ASSIGNMENT.pdf)

## Dataset
In addition to the required Dickens and Hardy books, some additional datasets were taken from [UCI - Machine Learning Repository](https://archive.ics.uci.edu/ml/datasets.html). The datasets used are described below.

**Datasets used, and their location in this project:**

**Dataset** | **Source** | **Path** | **Type** *
---|---|---|---
SMS | [UCI - SMS Spam Collection](https://archive.ics.uci.edu/ml/datasets/SMS+Spam+Collection) | `./data/sms` | inline
Badges | [UCI - Badges](https://archive.ics.uci.edu/ml/datasets/Badges) | `./data/badges` | inline
Main | Gutenberg - [Dickens](http://www.gutenberg.org/ebooks/author/37), [Hardy](http://www.gutenberg.org/ebooks/author/23) | gutenberg

**Dataset Types:** *

**Type** | **Description**
---|---
inline | Dataset is stored as a single file in which each line represents a training point. The first word in each line is the class/category, while the rest of the line is a list of words used as the training "text blob."
gutenberg | Dataset is stored as a list of directories representing classes/categories (e.g. "dickens", "hardy"). Each file within the class directories represent a training point. These files are actually books, but are abstractly considered to be "text blobs," just like the **inline** dataset type.
{'fraction_non_alphanumeric': 0.14631233203941474, 'fraction_numerical': 0.007166318303971335, 'mean_word_length': 4.201863354037267, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 6, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6281792', 'n_tokens_mistral': 977, 'n_tokens_neox': 948, 'n_words': 370}