<commit_before>broccoli-tree-traverser
====================

Manages src tree navigation, letting you deal with each file.

The traverser takes a path, or another tree, and a [Visitor](#visitor).

It traverses the input path, calling `visitor#visit` for each file it finds.

## Visitor <a name="visitor"></a>

A visitor can be any object with a `visit` method. the `visit` method takes a file path (string)
located somewhere along the input path. It should return a promise if it does anything asynchronously.

## Future enhancements

### Traversal
The traversal algorithm is pretty straightfoward, so it's neither depth first nor breadth first. Nodes
are visited in the same order they come back from `fs.lstat`. And since it's currently doing everything
asynchronously, it would be hard to ensure either pattern. I might enhance it with options to
ensure either algorithm if there is interest.

### Visit Directories
The traverser only calls `visit` for plain files. I may enhance it to visit directory paths as well, if there is 
interest.<commit_msg>updating the README<commit_after>broccoli-tree-traverser
====================

A broccoli plugin helper that manages src tree navigation, letting you deal with each file.

The traverser takes a path, or another tree, and a [Visitor](#visitor).

It traverses the input path, calling `visitor#visit` for each file it finds.

## Visitor <a name="visitor"></a>

A visitor can be any object with a `visit` method. the `visit` method takes a file path (string)
located somewhere along the input path. It should return a promise if it does anything asynchronously.

## Future enhancements

### Traversal
The traversal algorithm is an in-order traversal. That is, it traverses files and directories in the order returned by 
`fs.readdir`. Feel free to submit pull requests with other traversal algorithms.

### Visit Directories
The traverser only calls `visit` for plain files. Feel free to submit pull requests if you want to visit directories.

## Example use

### In a Brocfile:
```javascript
//Brocfile.js
var traverser = require('broccoli-tree-traverser');

var visitor = {
  visit:function(path){
    console.log('visiting', path);  
  }
};

module.exports = traverser('interesting/path', visitor);

```

### Within another plugin
```javascript
//index.js
var traverser = require('broccoli-tree-traverser');


function MyPlugin(inputTree){
  this.traverser = traverser(inputTree, this);
}

MyPlugin.prototype.visit = function(path){
  //do something interesting with the file
};

MyPlugin.prototype.read = function(readTree){
  return readTree(this.traverser);
};


module.exports = MyPlugin; 
```
{'fraction_non_alphanumeric': 0.1, 'fraction_numerical': 0.0, 'mean_word_length': 4.76304347826087, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '996225', 'n_tokens_mistral': 745, 'n_tokens_neox': 706, 'n_words': 330}