<commit_before>= Ruby-Stemmer {Project Status}[http://stillmaintained.com/aurelian/ruby-stemmer.png]

Ruby-Stemmer exposes SnowBall API to Ruby.

This package includes libstemmer_c library released under BSD licence 
and available for free {here}[http://snowball.tartarus.org/dist/libstemmer_c.tgz].

Support for latin language is also included and it has been generated with the snowball compiler using 
{schinke contribution}[http://snowball.tartarus.org/otherapps/schinke/intro.html].

For more details about libstemmer_c please visit the {SnowBall website}[http://snowball.tartarus.org].

== Usage

  require 'rubygems'
  require 'lingua/stemmer'

  stemmer= Lingua::Stemmer.new(:language => "ro")
  stemmer.stem("netăgăduit") #=> netăgădu

=== Alternative

  require 'rubygems'
  require 'lingua/stemmer'

  Lingua.stemmer( %w(incontestabil neîndoielnic), :language => "ro" ) #=> ["incontest", "neîndoieln"]
  Lingua.stemmer("installation") #=> "instal"
  Lingua.stemmer("installation", :language => "fr", :encoding => "ISO_8859_1") do | word |
    puts "~> #{word}" #=> "instal"
  end # => #<Lingua::Stemmer:0x102501e48>

=== Rails
  
  # Rails2: -- config/environment.rb:
  config.gem 'ruby-stemmer', :version => '>=0.6.2', :lib => 'lingua/stemmer'
  
  # Rails3: -- Gemfile
  gem 'ruby-stemmer', '>=0.8.3', :require => 'lingua/stemmer'

=== More details 

* Complete API in {RDoc format}[http://rdoc.info/github/aurelian/ruby-stemmer/master/frames]
* More usage on the {test file}[http://github.com/aurelian/ruby-stemmer/blob/master/test/lingua/test_stemmer.rb]

== Install

=== Standard install with:

 gem install ruby-stemmer

==== Windows

There's also a Windows (Fat bin) compiled against ruby 1.9.3 and ruby 1.8.7.

 gem install ruby-stemmer --platform=x86-mingw32

As far as I know the above should work with {rubyinstaller}[http://rubyinstaller.org/]. If if fails, you could try with:

 gem install ruby-stemmer --platform=x86-mswin32

{It's known}[http://cl.ly/BX9o] to work under Windows XP.

=== Development version

  $ git clone git://github.com/aurelian/ruby-stemmer.git
  $ cd ruby-stemmer
  $ rake -T #<== see what we've got
  $ rake compile #<== builds the extension do'h
  $ rake test

==== Cross Compiling

Install {rake-compiler-dev-box}[https://github.com/tjschuck/rake-compiler-dev-box] and follow the setup.
Inside vagrant box run:

  $ cd /vagrant
  $ RUBY_CC_VERSION=1.8.7:1.9.3:2.1.5:2.2.0 package_win32_fat_binary ruby-stemmer

Idea is to have same builds as in travis.yml.

== NOT A BUG

The stemming process is an algorithm to allow one to find the stem of an word (not the root of it).
For further reference on stem vs. root, please check wikipedia articles on the topic:

* http://en.wikipedia.org/wiki/Stem_%28linguistics%29
* http://en.wikipedia.org/wiki/Root_%28linguistics%29

== TODO

* {Open issues}[http://github.com/aurelian/ruby-stemmer/issues]

== Note on Patches/Pull Requests
 
* Fork the project from {github}[http://github.com/aurelian/ruby-stemmer]
* Make your feature addition or {bug fix}[http://github.com/aurelian/ruby-stemmer/issues]
* Add tests for it. This is important so I don't break it in a
  future version unintentionally.
* Commit, do not mess with rakefile, version, or history.

  if you want to have your own version, that is fine but
  bump version in a commit by itself I can ignore when I pull
* Send me a pull request. Bonus points for topic branches.

== Alternative Stemmers for Ruby

* {stemmer4r}[http://rubyforge.org/projects/stemmer4r] (ext)
* {fast-stemmer}[http://github.com/romanbsd/fast-stemmer] (ext)
* {uea-stemmer}[http://github.com/ealdent/uea-stemmer] (ext)
* {stemmer}[http://rubyforge.org/projects/stemmer] (pure ruby)
* add yours

== Copyright

Copyright (c) 2008-2011 {<NAME>}[http://locknet.ro]. See MIT-LICENSE for details.

== Contributors

* {<NAME>}[https://github.com/aurelian]
* {<NAME>}[https://github.com/yury] - various bug fixes
* {<NAME>}[https://github.com/tenderlove] - rake compiler (windows support), code cleanup
* {<NAME>}[https://github.com/munshkr] - Ruby 1.9 encoding

== Real life usage
* http://planet33.ru is using Ruby-Stemmer together with {Classifier}[http://github.com/yury/classifier] to automatically rate places based on users comments.

# encoding: utf-8
<commit_msg>updated year and the list of ruby versions<commit_after>= Ruby-Stemmer {Project Status}[http://stillmaintained.com/aurelian/ruby-stemmer.png]

Ruby-Stemmer exposes SnowBall API to Ruby.

This package includes libstemmer_c library released under BSD licence 
and available for free {here}[http://snowball.tartarus.org/dist/libstemmer_c.tgz].

Support for latin language is also included and it has been generated with the snowball compiler using 
{schinke contribution}[http://snowball.tartarus.org/otherapps/schinke/intro.html].

For more details about libstemmer_c please visit the {SnowBall website}[http://snowball.tartarus.org].

== Usage

  require 'rubygems'
  require 'lingua/stemmer'

  stemmer= Lingua::Stemmer.new(:language => "ro")
  stemmer.stem("netăgăduit") #=> netăgădu

=== Alternative

  require 'rubygems'
  require 'lingua/stemmer'

  Lingua.stemmer( %w(incontestabil neîndoielnic), :language => "ro" ) #=> ["incontest", "neîndoieln"]
  Lingua.stemmer("installation") #=> "instal"
  Lingua.stemmer("installation", :language => "fr", :encoding => "ISO_8859_1") do | word |
    puts "~> #{word}" #=> "instal"
  end # => #<Lingua::Stemmer:0x102501e48>

=== Rails
  
  # Rails2: -- config/environment.rb:
  config.gem 'ruby-stemmer', :version => '>=0.6.2', :lib => 'lingua/stemmer'
  
  # Rails3: -- Gemfile
  gem 'ruby-stemmer', '>=0.8.3', :require => 'lingua/stemmer'

=== More details 

* Complete API in {RDoc format}[http://rdoc.info/github/aurelian/ruby-stemmer/master/frames]
* More usage on the {test file}[http://github.com/aurelian/ruby-stemmer/blob/master/test/lingua/test_stemmer.rb]

== Install

=== Standard install with:

 gem install ruby-stemmer

==== Windows

There's also a Windows (Fat bin) compiled against ruby versions: 1.8.7, 1.9.3, 2.1.5, 2.2.0

 gem install ruby-stemmer --platform=x86-mingw32

As far as I know the above should work with {rubyinstaller}[http://rubyinstaller.org/]. If if fails, you could try with:

 gem install ruby-stemmer --platform=x86-mswin32

{It's known}[http://cl.ly/BX9o] to work under Windows XP.

=== Development version

  $ git clone git://github.com/aurelian/ruby-stemmer.git
  $ cd ruby-stemmer
  $ rake -T #<== see what we've got
  $ rake compile #<== builds the extension do'h
  $ rake test

==== Cross Compiling

Install {rake-compiler-dev-box}[https://github.com/tjschuck/rake-compiler-dev-box] and follow the setup.
Inside vagrant box run:

  $ cd /vagrant
  $ RUBY_CC_VERSION=1.8.7:1.9.3:2.1.5:2.2.0 package_win32_fat_binary ruby-stemmer

Idea is to have same builds as in travis.yml.

== NOT A BUG

The stemming process is an algorithm to allow one to find the stem of an word (not the root of it).
For further reference on stem vs. root, please check wikipedia articles on the topic:

* http://en.wikipedia.org/wiki/Stem_%28linguistics%29
* http://en.wikipedia.org/wiki/Root_%28linguistics%29

== TODO

* {Open issues}[http://github.com/aurelian/ruby-stemmer/issues]

== Note on Patches/Pull Requests
 
* Fork the project from {github}[http://github.com/aurelian/ruby-stemmer]
* Make your feature addition or {bug fix}[http://github.com/aurelian/ruby-stemmer/issues]
* Add tests for it. This is important so I don't break it in a
  future version unintentionally.
* Commit, do not mess with rakefile, version, or history.

  if you want to have your own version, that is fine but
  bump version in a commit by itself I can ignore when I pull
* Send me a pull request. Bonus points for topic branches.

== Alternative Stemmers for Ruby

* {stemmer4r}[http://rubyforge.org/projects/stemmer4r] (ext)
* {fast-stemmer}[http://github.com/romanbsd/fast-stemmer] (ext)
* {uea-stemmer}[http://github.com/ealdent/uea-stemmer] (ext)
* {stemmer}[http://rubyforge.org/projects/stemmer] (pure ruby)
* add yours

== Copyright

Copyright (c) 2008-2015 {<NAME>}[http://locknet.ro]. See MIT-LICENSE for details.

== Contributors

* {<NAME>}[https://github.com/aurelian]
* {<NAME>}[https://github.com/yury] - various bug fixes
* {<NAME>}[https://github.com/tenderlove] - rake compiler (windows support), code cleanup
* {<NAME>}[https://github.com/munshkr] - Ruby 1.9 encoding

== Real life usage
* http://planet33.ru is using Ruby-Stemmer together with {Classifier}[http://github.com/yury/classifier] to automatically rate places based on users comments.

# encoding: utf-8

{'fraction_non_alphanumeric': 0.1421784472769409, 'fraction_numerical': 0.017844727694090384, 'mean_word_length': 4.269230769230769, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 45, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2788331', 'n_tokens_mistral': 3238, 'n_tokens_neox': 2957, 'n_words': 808}