<commit_before>SARQ - Free Text Indexing for SPARQL
------------------------------------

SARQ is a combination of ARQ and Solr. It gives ARQ the ability to perform
free text searches using a remote Solr server. Lucene indexes in Solr are 
additional information for accessing the RDF graph, not storage for the 
graph itself.

This is *experimental* (and unsupported).

See also:

 - http://openjena.org/ARQ/lucene-arq.html

 
TODO
====

 - Fix the failing test and add more tests. [DONE]
 - Double check the id as unique key, does it make sense? [DONE]
 - ...<commit_msg>Documentation.<commit_after>SARQ - Free Text Indexing for SPARQL
------------------------------------

SARQ is a combination of ARQ and Solr. It gives ARQ the ability to perform
free text searches using a remote Solr server. Lucene indexes in Solr are 
additional information for accessing the RDF graph, not storage for the 
graph itself.

This is *experimental* (and unsupported).

This is how you build an index from a Jena Model:

    IndexBuilderModel builder = new IndexBuilderString("http://127.0.0.1:8983/solr/sarq");
    builder.indexStatements(model.listStatements());
    builder.commit();

This is how you configure ARQ to use Solr:
        
    SARQ.setDefaultIndex(builder.getSolrServer());

This is an example of a SPARQL query using the sarq:search property function: 

  PREFIX sarq:     <http://openjena.org/SARQ/property#>
  SELECT * WHERE {
      ?doc ?p ?lit .
      (?lit ?score ) sarq:search "+text" .
  }
        
The design and part of the code has been taken from LARQ, see:

 - http://openjena.org/ARQ/lucene-arq.html

 
TODO
====

 - Fix the failing test and add more tests. [DONE]
 - Double check the id as unique key, does it make sense? [DONE]
 - ...
{'fraction_non_alphanumeric': 0.12371721778791334, 'fraction_numerical': 0.005701254275940707, 'mean_word_length': 2.8152173913043477, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1392100', 'n_tokens_mistral': 552, 'n_tokens_neox': 516, 'n_words': 217}