<commit_before># = CodeRay
#
# $Id$
#
# CodeRay is a Ruby library for syntax highlighting.
#
# I try to make CodeRay easy to use and intuitive, but at the same time fully featured, complete,
# fast and efficient.
# 
# See README.
# 
# It consists mainly of
# * the main engine: CodeRay, CodeRay::Scanner, CodeRay::Tokens, CodeRay::TokenStream, CodeRay::Encoder
# * the scanners in CodeRay::Scanners
# * the encoders in CodeRay::Encoders
# 
# Here's a fancy graphic to light up this gray docu:
# 
# http://rd.cYcnus.de/coderay/scheme.png
# 
# == Documentation
#
# See CodeRay, Encoders, Scanners, Tokens.
#
# == Usage
#
# Remember you need RubyGems to use CodeRay. Run Ruby with -rubygems option
# if required.
#
# === Highlight Ruby code in a string as html
# 
#   require 'coderay'
#   print CodeRay.scan('puts "Hello, world!"', :ruby).html
#
#   # prints something like this:
#   puts <span class="s">&quot;Hello, world!&quot;</span>
# 
# 
# === Highlight C code from a file in a html div
# 
#   require 'coderay'
#   print CodeRay.scan(File.read('ruby.h'), :c).div
#   print CodeRay.scan_file('ruby.h').html.div
# 
# You can include this div in your page. The used CSS styles can be printed with
# 
#   % ruby -rcoderay -e "print CodeRay::Encoders[:html]::CSS"
# 
# === Highlight without typing too much
#
# If you are one of the hasty (or lazy, or extremely curious) people, just run this file:
#
#   % ruby -rubygems coderay.rb
# 
# If the output was to fast for you, try
# 
#   % ruby -rubygems coderay.rb > example.html
#
# and look at the file it created.
# 

# = CodeRay
#
# The CodeRay module provides convenience methods for the engine.<commit_msg>doc'ed on coderay.rb<commit_after># = CodeRay Library
#
# $Id$
#
# CodeRay is a Ruby library for syntax highlighting.
#
# I try to make CodeRay easy to use and intuitive, but at the same time fully featured, complete,
# fast and efficient.
# 
# See README.
# 
# It consists mainly of
# * the main engine: CodeRay (Scanners::Scanner, Tokens/TokenStream, Encoders::Encoder), PluginHost
# * the scanners in CodeRay::Scanners
# * the encoders in CodeRay::Encoders
# 
# Here's a fancy graphic to light up this gray docu:
# 
# http://rd.cYcnus.de/coderay/scheme.png
# 
# == Documentation
#
# See CodeRay, Encoders, Scanners, Tokens.
#
# == Usage
#
# Remember you need RubyGems to use CodeRay. Run Ruby with -rubygems option
# if required.
#
# === Highlight Ruby code in a string as html
# 
#   require 'coderay'
#   print CodeRay.scan('puts "Hello, world!"', :ruby).html
#
#   # prints something like this:
#   puts <span class="s">&quot;Hello, world!&quot;</span>
# 
# 
# === Highlight C code from a file in a html div
# 
#   require 'coderay'
#   print CodeRay.scan(File.read('ruby.h'), :c).div
#   print CodeRay.scan_file('ruby.h').html.div
# 
# You can include this div in your page. The used CSS styles can be printed with
# 
#   % ruby -rcoderay -e "print CodeRay::Encoders[:html]::CSS"
# 
# === Highlight without typing too much
#
# If you are one of the hasty (or lazy, or extremely curious) people, just run this file:
#
#   % ruby -rubygems coderay.rb
# 
# If the output was to fast for you, try
# 
#   % ruby -rubygems coderay.rb > example.html
#
# and look at the file it created.
# 
# = CodeRay Module
#
# The CodeRay module provides convenience methods for the engine.
{'fraction_non_alphanumeric': 0.12413793103448276, 'fraction_numerical': 0.0, 'mean_word_length': 3.7386363636363638, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1249410', 'n_tokens_mistral': 1165, 'n_tokens_neox': 1145, 'n_words': 407}