<commit_before>= Erbal: Very small, very fast Ragel/C based ERB parser

== Intro

Erbal is a lightweight ERB parser that uses the Ragel State Machine Compiler (http://www.complang.org/ragel/). It's written in C, it's very fast. Erbal also produces faster Ruby code than Ruby's ERB implementation to give you a evaluation-time performance benefit.

== Using Erbal

  >> require 'erbal'
  >> src = Erbal.new("<% a=1 -%>a is: <%= a -%>").parse
  => "@output_buffer = ''; a=1 ;\n@output_buffer.concat(%Q`a is: \#{ a }`);@output_buffer"
  >> eval(src)
  => "a is: 1"

Erbal.new takes an optional 2nd argument which is a hash of options, available options are:

* <b>:buffer</b> - The name of the output buffer. Default is '@output_buffer'.

* <b>:buffer_initial_value</b> - The initial value of the output buffer. Default is a blank string.

* <b>:unsafe_concat_method</b> - The method to call on the buffer when concatenating a string which DOES need escaping, i.e <%= some_method %> will result in the use of unsafe_concat_method. Default is 'concat'.

* <b>:safe_concat_method</b> - The method to call on the buffer when concatenating a string which doesn't need escaping, i.e text outside of ERB tags (your raw HTML) uses the safe concatenation method. Default is 'concat'.

* <b>:safe_concat_keyword</b> - By default the '<%=' tag uses the unsafe_concat_method, with this option you can specify a keyword to signify that the safe_concat_method should be used instead. For example if safe_concat_method is set to 'raw' then <%= raw some_method %> will result in a concatenation using the safe_concat_method. Be mindful that even thought the 'raw' keyword looks like a method call, it isn't. Erbal expects that the key is followed by one or more spaces, so a tag like '<%= raw(some_method) %>' will not be recognised by Erbal. Whitespace before the keyword is optional, so '<%=raw' is valid. The keyword can either be one or more a-z characters, or a single of the following: ! @ $ * = ^ & +. For example, given :safe_concat_keyword => '!' then the following is valid: '<%=! some_method %>'. Default is blank, i.e no keyword is specified.

<b>NOTE: Erbal itself does NOT perform escaping, it is the responsibility of your unsafe_concat_method to escape the string passed to it.</b>

== Rails 2.3

Create the file 'config/initializers/erbal.rb' containing:

  require 'erbal/rails'

=== Rails 2.3 + rails_xss

If you're using the rails_xss plugin, use this in your initialiser instead:

  require 'erbal'

  class ErbalTemplateHandler < ActionView::TemplateHandler
    include ActionView::TemplateHandlers::Compilable
    def compile(template)
      ::Erbal.new("<% __in_erb_template=true %>#{template.source}", {:buffer => '@output_buffer', :buffer_initial_value => 'ActiveSupport::SafeBuffer.new',
        :safe_concat_method => 'safe_concat', :unsafe_concat_method => 'concat', :safe_concat_keyword => 'raw'}).parse
    end
  end

  ActionView::Template.register_template_handler :erb, ErbalTemplateHandler

== Rails 3

I've not looked into yet.. patches are welcome! ;)

== Benchmarks

These benchmarks were run on a Mac OS X 10.6.4, 2.66 Ghx Intel Core i5 with 8 GB of 1067 MHz DDR3 RAM.

Ruby: 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI 0x6770, Ruby Enterprise Edition 2010.02

Erubis: 2.6.6

Erbal: 1.1

  => Parsing Benchmark

  => Erb
  0.851 0.853 0.848 0.847 0.849 0.847
  => Average: 0.849

  => Erubis (using FastEruby engine)
  0.442 0.442 0.444 0.442 0.441 0.442
  => Average: 0.442

  => Erubis (using default Eruby engine)
  0.451 0.424 0.446 0.425 0.445 0.446
  => Average: 0.439

  => Erbal
  0.036 0.039 0.066 0.038 0.067 0.039
  => Average: 0.047

  => eval() Benchmark

  => Erb
  0.203 0.192 0.180 0.180 0.179 0.191
  => Average: 0.187

  => Erubis (using FastEruby engine)
  0.129 0.127 0.128 0.128 0.114 0.128
  => Average: 0.125

  => Erubis (using default Eruby engine)
  0.178 0.176 0.165 0.176 0.175 0.164
  => Average: 0.172

  => Erbal
  0.111 0.125 0.124 0.110 0.133 0.124
  => Average: 0.121

== Contributing

* Fork the project.
* Make your feature addition or bug fix.
* Add tests for it, specs live in the specs/ directory.
* Commit and send me a pull request.

== Credits

Many thanks to <NAME> for writing the Ragel State Machine Compiler (http://www.complang.org/ragel/)!

== Copyrigh

Copyright (c) 2010-2011 Envato & <NAME>. See LICENSE for details.
<commit_msg>README tweaks<commit_after>= Erbal: Very small, very fast Ragel/C based ERB parser

== Intro

Erbal is a lightweight ERB parser that uses the Ragel State Machine Compiler (http://www.complang.org/ragel/). It's written in C, it's very fast. Erbal also produces faster Ruby code than Ruby's ERB implementation to give you a evaluation-time performance benefit.

== Using Erbal

  >> require 'erbal'
  >> src = Erbal.new("<% a=1 -%>a is: <%= a -%>").parse
  => "@output_buffer = ''; a=1 ;\n@output_buffer.concat(%Q`a is: \#{ a }`);@output_buffer"
  >> eval(src)
  => "a is: 1"

Erbal.new takes an optional 2nd argument which is a hash of options, available options are:

* <b>:buffer</b> - The name of the output buffer. Default is '@output_buffer'.

* <b>:buffer_initial_value</b> - The initial value of the output buffer. Default is a blank string.

* <b>:unsafe_concat_method</b> - The method to call on the buffer when concatenating a string which DOES need escaping, i.e <%= some_method %> will result in the use of unsafe_concat_method. Default is 'concat'.

* <b>:safe_concat_method</b> - The method to call on the buffer when concatenating a string which doesn't need escaping, i.e text outside of ERB tags (your raw HTML) uses the safe concatenation method. Default is 'concat'.

* <b>:safe_concat_keyword</b> - By default the '<%=' tag uses the unsafe_concat_method, with this option you can specify a keyword to signify that the safe_concat_method should be used instead. For example if safe_concat_method is set to 'raw' then <%= raw some_method %> will result in a concatenation using the safe_concat_method. Be mindful that even thought the 'raw' keyword looks like a method call, it isn't. Erbal expects that the key is followed by one or more spaces, so a tag like '<%= raw(some_method) %>' will not be recognised by Erbal. Whitespace before the keyword is optional, so '<%=raw' is valid. The keyword can either be one or more a-z characters, or a single of the following: ! @ $ * = ^ & +. For example, given :safe_concat_keyword => '!' then the following is valid: '<%=! some_method %>'. Default is blank, i.e no keyword is specified.

<b>NOTE: Erbal itself does NOT perform escaping, it is the responsibility of your unsafe_concat_method to escape the string passed to it.</b>

== Rails 2.3

Create the file 'config/initializers/erbal.rb' containing:

  require 'erbal/rails'

=== Rails 2.3 + rails_xss (https://github.com/rails/rails_xss)

If you're using the rails_xss plugin, use this in your initialiser instead:

  require 'erbal'

  class ErbalTemplateHandler < ActionView::TemplateHandler
    include ActionView::TemplateHandlers::Compilable
    def compile(template)
      ::Erbal.new("<% __in_erb_template=true %>#{template.source}",
        {:buffer => '@output_buffer',
         :buffer_initial_value => 'ActiveSupport::SafeBuffer.new',
         :safe_concat_method => 'safe_concat',
         :unsafe_concat_method => 'concat',
         :safe_concat_keyword => 'raw'}
      ).parse
    end
  end

  ActionView::Template.register_template_handler :erb, ErbalTemplateHandler

== Rails 3

I've not looked into yet.. patches are welcome! ;)

== Benchmarks

These benchmarks were run on a Mac OS X 10.6.4, 2.66 Ghx Intel Core i5 with 8 GB of 1067 MHz DDR3 RAM.

Ruby: 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI 0x6770, Ruby Enterprise Edition 2010.02

Erubis: 2.6.6

Erbal: 1.1

  => Parsing Benchmark

  => Erb
  0.851 0.853 0.848 0.847 0.849 0.847
  => Average: 0.849

  => Erubis (using FastEruby engine)
  0.442 0.442 0.444 0.442 0.441 0.442
  => Average: 0.442

  => Erubis (using default Eruby engine)
  0.451 0.424 0.446 0.425 0.445 0.446
  => Average: 0.439

  => Erbal
  0.036 0.039 0.066 0.038 0.067 0.039
  => Average: 0.047

  => eval() Benchmark

  => Erb
  0.203 0.192 0.180 0.180 0.179 0.191
  => Average: 0.187

  => Erubis (using FastEruby engine)
  0.129 0.127 0.128 0.128 0.114 0.128
  => Average: 0.125

  => Erubis (using default Eruby engine)
  0.178 0.176 0.165 0.176 0.175 0.164
  => Average: 0.172

  => Erbal
  0.111 0.125 0.124 0.110 0.133 0.124
  => Average: 0.121

== Contributing

* Fork the project.
* Make your feature addition or bug fix.
* Add tests for it, specs live in the specs/ directory.
* Commit and send me a pull request.

== Credits

Many thanks to <NAME> for writing the Ragel State Machine Compiler (http://www.complang.org/ragel/)!

== Copyrigh

Copyright (c) 2010-2011 Envato & <NAME>. See LICENSE for details.

{'fraction_non_alphanumeric': 0.11462539255271423, 'fraction_numerical': 0.0655002243158367, 'mean_word_length': 3.6857593273778244, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 109, 'https://': 1, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3333920', 'n_tokens_mistral': 3408, 'n_tokens_neox': 2868, 'n_words': 1119}