<commit_before>Running PySparkling on Databricks Azure Cluster
-----------------------------------------------

Sparkling Water, PySparkling and RSparkling can be used on top of Databricks Azure Cluster. This tutorial is
the **PySparkling**.

For Scala Sparkling Water, please visit `Sparkling Water on Databricks Azure Cluster <sw_azure_dbc.rst>`__ and
for RSparkling, please visit `RSparkling on Databricks Azure Cluster <rsparkling_azure_dbc.rst>`__.

To start Sparkling Water ``H2OContext`` on Databricks Azure, the steps are:

1.  Login into Microsoft Azure Portal

2.  Create Databricks Azure Environment

    In order to connect to Databricks from Azure, please make sure you have created user inside Azure Active Directory and using that user for the Databricks Login.

3.  Upload PySparkling Library as a library from PiPy

    In order to create the Python library in Databricks, go to **Libraries**, select **Upload Python Egg or Pipy** and insert ``h2o_pysparkling_2.2`` for the latest PySparkling for Spark 2.0.

    The advantage of adding the library from PiPy, instead of uploading manually, is that the latest version is always selected and also, the dependencies are automatically resolved.

    .. figure:: ../images/databricks_pysparkling_pipy.png
        :alt: Uploading PySparkling Library

    You can configure each cluster manually and select which libraries should be attached or you can configure the library to be attached to all future clusters. It is advised to restart the cluster in case you attached the library to already running cluster to ensure the clean environment.

4.  Create the cluster

    - Make sure the library is attached to the cluster
    - Put ``spark.ui.enabled false`` line in the spark configuration field. This is currently required due to limitation in our code and will be fixed as soon as possible.

    - Select Spark version according to you Sparkling Water version:
        - for the latest PySparkling 2.2, select Spark 2.2.1.
        - for the latest PySparkling 2.1, select Spark 2.1.2.
        - for the latest PySparkling 2.0, select Spark 2.0.2.
<commit_msg>[SW-685] Fix typo in documentation (#542)<commit_after>Running PySparkling on Databricks Azure Cluster
-----------------------------------------------

Sparkling Water, PySparkling and RSparkling can be used on top of Databricks Azure Cluster. This tutorial is
the **PySparkling**.

For Scala Sparkling Water, please visit `Sparkling Water on Databricks Azure Cluster <sw_azure_dbc.rst>`__ and
for RSparkling, please visit `RSparkling on Databricks Azure Cluster <rsparkling_azure_dbc.rst>`__.

To start Sparkling Water ``H2OContext`` on Databricks Azure, the steps are:

1.  Login into Microsoft Azure Portal

2.  Create Databricks Azure Environment

    In order to connect to Databricks from Azure, please make sure you have created user inside Azure Active Directory and using that user for the Databricks Login.

3.  Upload PySparkling Library as a library from PiPy

    In order to create the Python library in Databricks, go to **Libraries**, select **Upload Python Egg or Pipy** and insert ``h2o_pysparkling_2.2`` for the latest PySparkling for Spark 2.2.

    The advantage of adding the library from PiPy, instead of uploading manually, is that the latest version is always selected and also, the dependencies are automatically resolved.

    .. figure:: ../images/databricks_pysparkling_pipy.png
        :alt: Uploading PySparkling Library

    You can configure each cluster manually and select which libraries should be attached or you can configure the library to be attached to all future clusters. It is advised to restart the cluster in case you attached the library to already running cluster to ensure the clean environment.

4.  Create the cluster

    - Make sure the library is attached to the cluster
    - Put ``spark.ui.enabled false`` line in the spark configuration field. This is currently required due to limitation in our code and will be fixed as soon as possible.

    - Select Spark version according to you Sparkling Water version:
        - for the latest PySparkling 2.2, select Spark 2.2.1.
        - for the latest PySparkling 2.1, select Spark 2.1.2.
        - for the latest PySparkling 2.0, select Spark 2.0.2.

{'fraction_non_alphanumeric': 0.06965871902758299, 'fraction_numerical': 0.013090229079008883, 'mean_word_length': 3.5089567966280293, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4540420', 'n_tokens_mistral': 1190, 'n_tokens_neox': 1086, 'n_words': 583}