<commit_before># acts_as_xapian/lib/acts_as_xapian.rb:
# Xapian full text search in Ruby on Rails.
#
# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved.
# Email: <EMAIL>; WWW: http://www.mysociety.org/
#
# $Id: acts_as_xapian.rb,v 1.11 2008-04-25 21:01:36 francis Exp $

# TODO:
# Test :eager_load
# Test :if
# Test reverse sorting

# Documentation
# =============
#
# Xapian is a full text search engine library, which has Ruby bindings.
# acts_as_xapian adds support for it to Rails. It is an alternative to
# acts_as_lucene or acts_as_ferret.
#
# Xapian is an *offline indexing* search library - only one process can have
# the database open for writing at once, and others that try meanwhile are
# unceremoniously kicked out. For this reason, acts_as_xapian does not support
# automatic writing to the database when your models change.
#
# Instead, there is a ActsAsXapianJob model which stores which models need
# updating or deleting in the search index. A rake task 'xapian:update_index'
# then performs the updates since last change. Run it on a cron job, or
# similar.
#
# Email <EMAIL> with patches.
#
#
# Comparison to acts_as_solr (as on 24 April 2008)
# ==========================
#
# * Offline indexing only mode - which is a minus if you want changes
# immediately reflected in the search index, and a plus if you were going to
# have to implement your own offline indexing anyway.
#
# * Collapsing - the equivalent of SQL's "group by". You can specify a field
# to collapse on, and only the most relevant result from each value of that
# field is returned. Along with a count of how many there are in total.
# acts_as_solr doesn't have this.
#<commit_msg>Say version of xapian required<commit_after># acts_as_xapian/lib/acts_as_xapian.rb:
# Xapian full text search in Ruby on Rails.
#
# Copyright (c) 2008 UK Citizens Online Democracy. All rights reserved.
# Email: <EMAIL>; WWW: http://www.mysociety.org/
#
# $Id: acts_as_xapian.rb,v 1.12 2008-04-29 16:14:58 francis Exp $

# TODO:
# Test :eager_load
# Test :if
# Test reverse sorting

# Documentation
# =============
#
# Xapian is a full text search engine library, which has Ruby bindings.
# acts_as_xapian adds support for it to Rails. It is an alternative to
# acts_as_lucene or acts_as_ferret.
#
# Xapian is an *offline indexing* search library - only one process can have
# the database open for writing at once, and others that try meanwhile are
# unceremoniously kicked out. For this reason, acts_as_xapian does not support
# automatic writing to the database when your models change.
#
# Instead, there is a ActsAsXapianJob model which stores which models need
# updating or deleting in the search index. A rake task 'xapian:update_index'
# then performs the updates since last change. Run it on a cron job, or
# similar.
#
# Xapian 1.0.5 and associated Ruby bindings are required.
#
# Email <EMAIL> with patches.
#
#
# Comparison to acts_as_solr (as on 24 April 2008)
# ==========================
#
# * Offline indexing only mode - which is a minus if you want changes
# immediately reflected in the search index, and a plus if you were going to
# have to implement your own offline indexing anyway.
#
# * Collapsing - the equivalent of SQL's "group by". You can specify a field
# to collapse on, and only the most relevant result from each value of that
# field is returned. Along with a count of how many there are in total.
# acts_as_solr doesn't have this.
#
{'fraction_non_alphanumeric': 0.09222318589187627, 'fraction_numerical': 0.01647875108412836, 'mean_word_length': 4.242424242424242, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5895247', 'n_tokens_mistral': 1119, 'n_tokens_neox': 1025, 'n_words': 488}