<commit_before># Mongoose Object Stream
Super simple package that wraps up a Mongoose Model into a stream so that you can pipe results of other operations right into a Mongo Database.

It uses a set-it-and-forget-it kind of pattern, but as is with any stream, you can end it and wait for a finish event.

This works really well if you need to throw a writable stream into a Bunyan logger, which is where I first created this.<commit_msg>Some readme stuff.<commit_after># Mongoose Object Stream

Super simple package that wraps up a Mongoose Model into a stream so that you can pipe results of other operations right into a Mongo Database.

It uses a set-it-and-forget-it kind of pattern, but as is with any stream, you can end it and wait for a finish event.

This works really well if you need to throw a writable stream into a Bunyan logger, which is where I first created this.

## Usage

There are two ways of creating a stream. 
The easiest of the two ways is to pass in a model to the stream constructor:

    // assuming you've defined/imported `someModel` already.
    var modelStream = new MongooseObjectStream(someModel);

Alternatively, you can define the model in the stream constructor. 

    var modelStream = new MongooseObjectStream(mongoose, "ModelName", {key: String}, {strict: false});

In both of these cases, you'll have a stream that you can use to write either strings or objects to directly.

    modelStream.write({key: "value"});
    modelStream.write('{"key": "another value"}');

It will parse out any strings that are passed to it, in case your data source provides strings rather objects. 

## Caveats

There's no error handling yet, so if you pass in a non-JSON string to the stream, it'll blow up.
{'fraction_non_alphanumeric': 0.055523423944476576, 'fraction_numerical': 0.0, 'mean_word_length': 4.058479532163743, 'pattern_counts': {'":': 1, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6970838', 'n_tokens_mistral': 457, 'n_tokens_neox': 440, 'n_words': 272}