<commit_before><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
  -->
  
## Solr Index

The Solr index is mainly meant for full-text search (the 'contains' type of queries):

    //*[jcr:contains(., 'text')]

but is also able to search by path and property restrictions.
Primary type restriction support is also provided by it's not recommended as it's usually much better to use the [node type 
index](query.html#The_Node_Type_Index) for such kind of queries.

Even if it's not just a full-text index, it's recommended to use it asynchronously (see `Oak#withAsyncIndexing`)
because, in most production scenarios, it'll be a 'remote' index and therefore network latency / errors would 
have less impact on the repository performance.

The index definition node for a Solr-based index:

 * must be of type `oak:QueryIndexDefinition`
 * must have the `type` property set to __`solr`__
 * must contain the `async` property set to the value `async`, this is what sends the index update process to a background thread.

_Optionally_ one can add

 * the `reindex` flag which when set to `true`, triggers a full content re-index.

Example:

    {
      NodeBuilder index = root.child("oak:index");
      index.child("solr")
        .setProperty("jcr:primaryType", "oak:QueryIndexDefinition", Type.NAME)
        .setProperty("type", "solr")
        .setProperty("async", "async")
        .setProperty("reindex", true);
    }
        
The Oak Solr index creates one document in the Solr index for each node in the repository, each of such documents has 
usually at least a field for each property associated with the related node.
Indexing of properties can be done by name: e.g. property 'jcr:title' of a node is written into a field 'jcr:title' of 
the corresponding Solr document in the index, or by type: e.g. properties 'jcr:data' and 'binary_content' of type 
_binary_ are written into a field 'binary_data' that's responsible for the indexing of all fields having that type and 
thus properly configured for hosting such type of data.
    
### Configuring the Solr index

Besides the index definition parameters mentioned above, a number of additional parameters can be defined in 
 Oak Solr index configuration.
Such a configuration is composed by:

 - the search / indexing configuration (see [OakSolrConfiguration](http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/plugins/index/solr/configuration/OakSolrConfiguration.html))
 - the Solr server configuration (see [SolrServerConfiguration](http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/plugins/index/solr/configuration/SolrServerConfiguration.html))
 
### Search / indexing configuration options

Such options define how Oak handles search and indexing requests in order to properly delegate such operations to Solr.

#### Use for property restrictions

If set to _true_ the Solr index will be used also for filtering nodes by property restrictions.

Default is 'false'.

#### Use for path restrictions

If set to _true_ the Solr index will be used also for filtering nodes by path restrictions.

Default is 'false'.

#### Use for primary types

If set to _true_ the Solr index will be used also for filtering nodes by primary type.

Default is 'false'.

#### Path field

The name of the field to be used for searching an exact match of a certain path.

Default is 'path_exact'.

#### Catch all field

The name of the field to be used for searching when no specific field is defined in the search query (e.g. user entered 
queries like 'foo bar').

Default is 'catch_all'.

Default Solr schema.xml provided with Oak Solr index contains a copyField from everything to 'catch_all', that causing
all the properties of a certain node to be indexed into that field (as separate values) therefore a query run against 
that field would match if any of the properties of the original node would have matched such a query.

#### Descendant path field

The name of the field to be used for searching for nodes descendants of a certain node.

Default is 'path_des'.

E.g. The Solr query to find all the descendant nodes of /a/b would be 'path_des:\/a\/b'.

#### Children path field

The name of the field to be used for searching for child nodes of a certain node.

Default is 'path_child'.

E.g. The Solr query to find all the child nodes of /a/b would be 'path_child:\/a\/b'.

#### Parent path field

The name of the field to be used for searching for parent node of a certain node.

Default is 'path_anc'.

E.g. The Solr query to find the parent node of /a/b would be 'path_anc:\/a\/b'.

#### Property restriction fields

The (optional) mapping of property names into Solr fields, so that a mapping jcr:title=foo is defined each node having 
 the property jcr:title will have its correspondant Solr document having a property foo indexed with the value of the 
 jcr:title property.

Default is no mapping, therefore the default mechanism of mapping property names to field names is performed.

#### Used properties

A whitelist of properties to be used for indexing / searching by Solr index.
Such a whitelist, if not empty, would dominate whatever configuration defined for the [Ignored_properties](#Ignored_properties).

Default is an empty list.

E.g. If such a whitelist contains properties _jcr:title_ and _text_ the Solr index will only index such properties for each
node and will be possible to use it for searching only on those two properties.

#### Ignored properties
A blacklist of properties to be ignored while indexing and searching by the Solr index.

Such a blacklist makes sense (it will be taken into account by the Solr index) only if the [Used properties](#Used_properties)
 option doesn't have any value.

Default is the following array: _("rep:members", "rep:authorizableId", "jcr:uuid", "rep:principalName", "rep:password"}_.

#### Commit policy

The Solr commit policy to be used when indexing nodes as documents in Solr.

Possible values are 'SOFT', 'HARD', 'AUTO'.

SOFT: perform a Solr soft-commit for each indexed document.

HARD: perform a Solr (hard) commit for each indexed document.

AUTO: doesn't perform any commit and relies on auto commit being configured on plain Solr's configuration (solrconfig.xml).

Default is _SOFT_.

#### Rows

The number of documents per 'page' to be fetched for each query.

Default is _Integer.MAX_VALUE_ (was _50_ in Oak 1.0).

#### Collapse _jcr:content_ nodes

`@since 1.3.4, 1.2.4, 1.0.18`

Whether the [Collapsing query parser](https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results) should
be used when searching in order to collapse nodes that are descendants of 'jcr:content' nodes into the 'jcr:content' node only.
E.g. if normal query results would include '/a/jcr:content' and '/a/jcr:content/b/', with this option enabled only 
'/a/jcr:content' would be returned by Solr using the Collapsing query parser.
This feature requires an additional field to be indexed, therefore if this is turned on, reindexing should be triggered
in order to make it work properly.  

#### Collapsed path field 

`@since 1.3.4, 1.2.4, 1.0.18`

The name of the field used for collapsing descendants of jcr:content nodes, see 'Collapse jcr:content nodes' option.

##### Solr server configuration options

TBD
    
#### Setting up the Solr server
<commit_msg>no Jira - update Solr configuration defaults in docs<commit_after><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
  -->
  
## Solr Index

The Solr index is mainly meant for full-text search (the 'contains' type of queries):

    //*[jcr:contains(., 'text')]

but is also able to search by path and property restrictions.
Primary type restriction support is also provided by it's not recommended as it's usually much better to use the [node type 
index](query.html#The_Node_Type_Index) for such kind of queries.

Even if it's not just a full-text index, it's recommended to use it asynchronously (see `Oak#withAsyncIndexing`)
because, in most production scenarios, it'll be a 'remote' index and therefore network latency / errors would 
have less impact on the repository performance.

The index definition node for a Solr-based index:

 * must be of type `oak:QueryIndexDefinition`
 * must have the `type` property set to __`solr`__
 * must contain the `async` property set to the value `async`, this is what sends the index update process to a background thread.

_Optionally_ one can add

 * the `reindex` flag which when set to `true`, triggers a full content re-index.

Example:

    {
      NodeBuilder index = root.child("oak:index");
      index.child("solr")
        .setProperty("jcr:primaryType", "oak:QueryIndexDefinition", Type.NAME)
        .setProperty("type", "solr")
        .setProperty("async", "async")
        .setProperty("reindex", true);
    }
        
The Oak Solr index creates one document in the Solr index for each node in the repository, each of such documents has 
usually at least a field for each property associated with the related node.
Indexing of properties can be done by name: e.g. property 'jcr:title' of a node is written into a field 'jcr:title' of 
the corresponding Solr document in the index, or by type: e.g. properties 'jcr:data' and 'binary_content' of type 
_binary_ are written into a field 'binary_data' that's responsible for the indexing of all fields having that type and 
thus properly configured for hosting such type of data.
    
### Configuring the Solr index

Besides the index definition parameters mentioned above, a number of additional parameters can be defined in 
 Oak Solr index configuration.
Such a configuration is composed by:

 - the search / indexing configuration (see [OakSolrConfiguration](http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/plugins/index/solr/configuration/OakSolrConfiguration.html))
 - the Solr server configuration (see [SolrServerConfiguration](http://jackrabbit.apache.org/oak/docs/apidocs/org/apache/jackrabbit/oak/plugins/index/solr/configuration/SolrServerConfiguration.html))
 
### Search / indexing configuration options

Such options define how Oak handles search and indexing requests in order to properly delegate such operations to Solr.

#### Use for property restrictions

If set to _true_ the Solr index will be used also for filtering nodes by property restrictions.

Default is 'false'.

#### Use for path restrictions

If set to _true_ the Solr index will be used also for filtering nodes by path restrictions.

Default is 'false'.

#### Use for primary types

If set to _true_ the Solr index will be used also for filtering nodes by primary type.

Default is 'false'.

#### Path field

The name of the field to be used for searching an exact match of a certain path.

Default is 'path_exact'.

#### Catch all field

The name of the field to be used for searching when no specific field is defined in the search query (e.g. user entered 
queries like 'foo bar').

Default is 'catch_all'.

Default Solr schema.xml provided with Oak Solr index contains a copyField from everything to 'catch_all', that causing
all the properties of a certain node to be indexed into that field (as separate values) therefore a query run against 
that field would match if any of the properties of the original node would have matched such a query.

#### Descendant path field

The name of the field to be used for searching for nodes descendants of a certain node.

Default is 'path_des'.

E.g. The Solr query to find all the descendant nodes of /a/b would be 'path_des:\/a\/b'.

#### Children path field

The name of the field to be used for searching for child nodes of a certain node.

Default is 'path_child'.

E.g. The Solr query to find all the child nodes of /a/b would be 'path_child:\/a\/b'.

#### Parent path field

The name of the field to be used for searching for parent node of a certain node.

Default is 'path_anc'.

E.g. The Solr query to find the parent node of /a/b would be 'path_anc:\/a\/b'.

#### Property restriction fields

The (optional) mapping of property names into Solr fields, so that a mapping jcr:title=foo is defined each node having 
 the property jcr:title will have its correspondant Solr document having a property foo indexed with the value of the 
 jcr:title property.

Default is no mapping, therefore the default mechanism of mapping property names to field names is performed.

#### Used properties

A whitelist of properties to be used for indexing / searching by Solr index.
Such a whitelist, if not empty, would dominate whatever configuration defined for the [Ignored_properties](#Ignored_properties).

Default is an empty list.

E.g. If such a whitelist contains properties _jcr:title_ and _text_ the Solr index will only index such properties for each
node and will be possible to use it for searching only on those two properties.

#### Ignored properties
A blacklist of properties to be ignored while indexing and searching by the Solr index.

Such a blacklist makes sense (it will be taken into account by the Solr index) only if the [Used properties](#Used_properties)
 option doesn't have any value.

Default is the following array: _("rep:members", "rep:authorizableId", "jcr:uuid", "rep:principalName", "rep:password"}_.

#### Commit policy

The Solr commit policy to be used when indexing nodes as documents in Solr.

Possible values are 'SOFT', 'HARD', 'AUTO'.

SOFT: perform a Solr soft-commit for each indexed document.

HARD: perform a Solr (hard) commit for each indexed document.

AUTO: doesn't perform any commit and relies on auto commit being configured on plain Solr's configuration (solrconfig.xml).

Default is _SOFT_.

#### Rows

The number of documents per 'page' to be fetched for each query.

Default is _10_.

#### Collapse _jcr:content_ nodes

`@since 1.3.4, 1.2.4, 1.0.18`

Whether the [Collapsing query parser](https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results) should
be used when searching in order to collapse nodes that are descendants of 'jcr:content' nodes into the 'jcr:content' node only.
E.g. if normal query results would include '/a/jcr:content' and '/a/jcr:content/b/', with this option enabled only 
'/a/jcr:content' would be returned by Solr using the Collapsing query parser.
This feature requires an additional field to be indexed, therefore if this is turned on, reindexing should be triggered
in order to make it work properly.  

#### Collapsed path field 

`@since 1.3.4, 1.2.4, 1.0.18`

The name of the field used for collapsing descendants of jcr:content nodes, see 'Collapse jcr:content nodes' option.

##### Solr server configuration options

TBD
    
#### Setting up the Solr server

{'fraction_non_alphanumeric': 0.06967670011148272, 'fraction_numerical': 0.0033444816053511705, 'mean_word_length': 4.134181240063593, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '658637', 'n_tokens_mistral': 4420, 'n_tokens_neox': 4257, 'n_words': 2256}