<commit_before># FileInfo

FileInfo extracts encoding from strings and files using the [wonderful Unix `file` command](http://en.wikipedia.org/wiki/File_\(command\)).

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'fileinfo'
```

And then execute:

```sh
$ bundle
```

Or install it yourself as:

```sh
$ gem install fileinfo
```

## Usage

Use `FileInfo.parse` with a string:

```ruby
FileInfo.parse('foo bar baz').encoding # => #<Encoding:US-ASCII>
FileInfo.parse('föø bår bàz').encoding # => #<Encoding:UTF-8>
```

Use `FileInfo.load` with a filename:

```ruby
filename = '/Users/rafbm/Downloads/some_crap_coming_from_windows.csv'
FileInfo.load(filename).encoding # => #<Encoding:ISO-8859-1>
```

## Contributing

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request<commit_msg>Add copyright notice to README<commit_after># FileInfo

FileInfo extracts encoding from strings and files using the [wonderful Unix `file` command](http://en.wikipedia.org/wiki/File_\(command\)).

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'fileinfo'
```

And then execute:

```sh
$ bundle
```

Or install it yourself as:

```sh
$ gem install fileinfo
```

## Usage

Use `FileInfo.parse` with a string:

```ruby
FileInfo.parse('foo bar baz').encoding # => #<Encoding:US-ASCII>
FileInfo.parse('föø bår bàz').encoding # => #<Encoding:UTF-8>
```

Use `FileInfo.load` with a filename:

```ruby
filename = '/Users/rafbm/Downloads/some_crap_coming_from_windows.csv'
FileInfo.load(filename).encoding # => #<Encoding:ISO-8859-1>
```

## Contributing

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request

---

© 2013 [<NAME>](http://heliom.ca). FileInfo is released under the MIT license.
{'fraction_non_alphanumeric': 0.15160349854227406, 'fraction_numerical': 0.012633624878522837, 'mean_word_length': 4.6256830601092895, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7159769', 'n_tokens_mistral': 729, 'n_tokens_neox': 677, 'n_words': 217}