<commit_before>Tutorial
--------

This tutorial use case is:

Search for the resumes (PDF or Word file which resides in One drive or local) and search for anything in the content
using Kibana. For example location worked or the previous company, etc.

Prerequisites
^^^^^^^^^^^^^

* Java 14+ must be installed
* ``JAVA_HOME`` must be defined

Install Elastic stack
^^^^^^^^^^^^^^^^^^^^^

* Download `Elasticsearch <https://www.elastic.co/downloads/elasticsearch>`_
* Download `Kibana <https://www.elastic.co/downloads/kibana>`_
* Start Elasticsearch server
* Start Kibana server
* Check that Kibana is running by opening http://localhost:5601

Start FSCrawler
^^^^^^^^^^^^^^^

* Download FSCrawler. See :ref:`installation`.
* Open a terminal and navigate to the ``fscrawler`` folder.
* Type:

.. code:: sh

   # On Linux/Mac
   bin/fscrawler resumes
   # On Windows
   .\bin\fscrawler resumes

* It will ask "Do you want to create it (Y/N)?". Answer ``Y``.
* Go to the FSCrawler configuration folder to edit the job configuration. The FSCrawler configuration folder named
  ``.fscrawler`` is by default in the user home directory, like ``C:\Users\myuser`` on Windows platform or
  ``~`` on Linux/MacOS. In this folder, you will find another folder named ``resumes``. Enter this folder:

.. code:: sh

   # On Linux/Mac
   cd ~/.fscrawler/resumes
   # On Windows
   cd C:\Users\myuser\resumes

* Edit the ``_settings.yaml`` file which is in this folder and change the ``url`` value to your folder
  which contains the resumes you would like to index:

.. code:: yaml

   ---
   name: "resumes"
   fs:
     # On Linux
     url: "/path/to/resumes"
     # On Windows
     url: "c:\path\to\resumes"

* Start again FSCrawler:

.. code:: sh

   # On Linux/Mac
   bin/fscrawler resumes
   # On Windows
   .\bin\fscrawler resumes

FSCrawler should index all the documents inside your directory.

.. note::

    If you want to start again reindexing from scratch instead of monitoring the changes, stop FSCrawler, restart it
    with the ``--restart`` option:

    .. code:: sh
<commit_msg>Fix the Windows path with double backslash<commit_after>Tutorial
--------

This tutorial use case is:

Search for the resumes (PDF or Word file which resides in One drive or local) and search for anything in the content
using Kibana. For example location worked or the previous company, etc.

Prerequisites
^^^^^^^^^^^^^

* Java 14+ must be installed
* ``JAVA_HOME`` must be defined

Install Elastic stack
^^^^^^^^^^^^^^^^^^^^^

* Download `Elasticsearch <https://www.elastic.co/downloads/elasticsearch>`_
* Download `Kibana <https://www.elastic.co/downloads/kibana>`_
* Start Elasticsearch server
* Start Kibana server
* Check that Kibana is running by opening http://localhost:5601

Start FSCrawler
^^^^^^^^^^^^^^^

* Download FSCrawler. See :ref:`installation`.
* Open a terminal and navigate to the ``fscrawler`` folder.
* Type:

.. code:: sh

   # On Linux/Mac
   bin/fscrawler resumes
   # On Windows
   .\bin\fscrawler resumes

* It will ask "Do you want to create it (Y/N)?". Answer ``Y``.
* Go to the FSCrawler configuration folder to edit the job configuration. The FSCrawler configuration folder named
  ``.fscrawler`` is by default in the user home directory, like ``C:\Users\myuser`` on Windows platform or
  ``~`` on Linux/MacOS. In this folder, you will find another folder named ``resumes``. Enter this folder:

.. code:: sh

   # On Linux/Mac
   cd ~/.fscrawler/resumes
   # On Windows
   cd C:\Users\myuser\resumes

* Edit the ``_settings.yaml`` file which is in this folder and change the ``url`` value to your folder
  which contains the resumes you would like to index:

.. code:: yaml

   ---
   name: "resumes"
   fs:
     # On Linux
     url: "/path/to/resumes"
     # On Windows
     url: "c:\\path\\to\\resumes"

* Start again FSCrawler:

.. code:: sh

   # On Linux/Mac
   bin/fscrawler resumes
   # On Windows
   .\bin\fscrawler resumes

FSCrawler should index all the documents inside your directory.

.. note::

    If you want to start again reindexing from scratch instead of monitoring the changes, stop FSCrawler, restart it
    with the ``--restart`` option:

    .. code:: sh

{'fraction_non_alphanumeric': 0.12237595419847329, 'fraction_numerical': 0.0028625954198473282, 'mean_word_length': 3.4044117647058822, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5873964', 'n_tokens_mistral': 1323, 'n_tokens_neox': 1286, 'n_words': 510}