<commit_before>Basic usage
-----------

Robot Framework test cases are executed from the command line, and the
end result is, by default, an output file in XML format and an HTML
report and log. After the execution, `output files`__ can be combined and
otherwise `post-processed`__ with the :prog:`rebot` tool.

__ `Created outputs`_
__ `Post-processing outputs`_

.. contents::
   :depth: 2
   :local:

Starting test execution
~~~~~~~~~~~~~~~~~~~~~~~

Synopsis
''''''''

::

   pybot [options] datasources
   jybot [options] datasources
   interpreter /path/to/robot/runner.py [options] datasources
   java -jar robotframework-<version>.jar [options] datasources

Different runner scripts
''''''''''''''''''''''''

Test execution is normally started with the :prog:`pybot` or
:prog:`jybot` commands. These commands are otherwise identical, but
the former executes tests using the Python interpreter and the latter uses
Jython. Which one to use depends on the needed test libraries. Some
libraries use modules or syntax available only on Python, others use
Java-based tools that require Jython, and some work on both. If you
can use either :prog:`pybot` or :prog:`jybot`, the former is
recommended, as Python is somewhat faster than Jython.

Starting from Robot Framework 2.5.2, it is also possible to get Robot Framework
as a single jar file.  In this case, Jython is bundled inside the jar, and the
above discussion about Jython applies. Normal arguments to
Robot Framework are given after the jar file name.

Another possibility for starting the test execution is running the
:prog:`runner.py` script under the installed__ :code:`robot` module
directly. This method allows selecting the interpreter and setting
command line options to it freely. The most common use case is
altering the options controlling `JVM maximum memory consumption`__.

__ `Where files are installed`_
__ `Modifying Java startup parameters`_

Regardless of the runner script, the path (or paths) to the test data to be
executed is given as an argument. Additionally, different command line
options can be used to alter the test execution or generated
outputs in some way.

Specifying test data to be executed
'''''''''''''''''''''''''''''''''''

Robot Framework test cases are created in files__ and directories__,
and they are executed by giving the path to the file or directory in
question to the selected runner script. The path can be absolute or,
more commonly, relative to the directory where tests are executed
from. The given file or directory creates the top-level test suite,
which gets its name, unless overridden with the :opt:`--name` option__,
from the `file or directory name`__. Different execution possibilities
are illustrated in the examples below. Note that in these examples, as
well as in other examples in this section, only the :prog:`pybot`
command is used, but :prog:`jybot` or a custom runner script
could be used similarly.

__ `Test case files`_
__ `Test suite directories`_
__ `Setting the name`_
__ `Test suite name and documentation`_

::

   pybot test_cases.html
   pybot path/to/my_tests/
   pybot c:\robot\tests.txt

It is also possible to give paths to several test case files or
directories at once, separated with spaces. In this case, Robot
Framework creates the top-level test suite automatically, and
the specified files and directories become its child test suites. The name
of the created test suite is got from child suite names by
catenating them together with an ampersand (&) and spaces. For example,
the name of the top-level suite in the first example below is
:name:`My Tests & Your Tests`. These automatically created names are
often quite long and complicated. In most cases, it is thus better to
use the :opt:`--name` option for overriding it, as in the second
example below::

   pybot my_tests.html your_tests.html
   pybot --name Example path/to/tests/pattern_*.html

Using command line options
~~~~~~~~~~~~~~~~~~~~~~~~~~

Robot Framework provides a number of command line options that can be
used to control how test cases are executed and what outputs are
generated. This section explains the option syntax, and what
options actually exist. How they can be used is discussed elsewhere
in this chapter.

Using options
'''''''''''''

When options are used, they must always be given between the runner
script and the data sources. For example::

   pybot -L debug my_tests.txt
   jybot --include smoke --variable HOST:10.0.0.42 path/to/tests/

Short and long options
''''''''''''''''''''''

Options always have a long name, such as :opt:`--name`, and the
most frequently needed options also have a short name, such as
:opt:`-N`. In addition to that, long options can be shortened as
long as they are unique. For example, :cli:`--logle DEBUG` works,
while :cli:`--lo log.html` does not, because the former matches only
:opt:`--loglevel`, but the latter matches several options. Short
and shortened options are practical when executing test cases
manually, but long options are recommended in `start-up scripts`__,
because they are easier to understand.

__ `Creating start-up scripts`_

The long option format is case-insensitive, which facilitates writing option
names in an easy-to-read format. For example, :opt:`--SuiteStatLevel`
is equivalent to, but easier to read than :opt:`--suitestatlevel`.

Setting option values
'''''''''''''''''''''

Most of the options require a value, which is given after the option
name. Both short and long options accept the value separated
from the option name with a space, as in :cli:`--include tag`
or :cli:`-i tag`. With long options, the separator can also be the
equals sign, for example :cli:`--include=tag`, and with short options the
separator can be omitted, as in :cli:`-itag`.

Some options can be specified several times. For example,
:cli:`--variable VAR1:value --variable VAR2:another` sets two
variables. If the options that take only one value are used several
times, the value given last is effective.

Option value as simple pattern
''''''''''''''''''''''''''''''

Many of the options take arguments as *simple patterns*. This means
that :code:`*` and :code:`?` can be used as special characters, so
that the former matches any string (even an empty string) and the
latter matches any single character. For example, :cli:`--include
prefix-\*` matches all tags starting with :code:`prefix-`, and
:cli:`--include a???` matches any tag that is four characters long and
starts with a character :code:`a`.

Test results
~~~~~~~~~~~~

Command line output
'''''''''''''''''''

The most visible output from test execution is the output displayed in
the command line. All executed test suites and test cases, as well as
their statuses, are shown there in real time. The example below shows the
output from executing a simple test suite with only two test cases::

   ==============================================================================
   Example test suite
   ==============================================================================
   First test :: Possible test documentation                             | PASS |
   ------------------------------------------------------------------------------
   Second test                                                           | FAIL |
   Error message is displayed here
   ==============================================================================
   Example test suite                                                    | FAIL |
   2 critical tests, 1 passed, 1 failed
   2 tests total, 1 passed, 1 failed
   ==============================================================================
   Output:  /path/to/output.xml
   Report:  /path/to/report.html
   Log:     /path/to/log.html

Generated output files
''''''''''''''''''''''

The command line output is very limited, and separate output files are
normally needed for investigating the test results. As the example
above shows, three output files are generated by default. The first
one is in XML format and contains all the information about test
execution. The second is a higher-level report and the third is a more
detailed log file. These files and other possible output files are
discussed in more detail in the section `Different output files`_.

Return codes
''''''''''''

Runner scripts communicate the overall test execution status to the<commit_msg>Documented notifications on console while test is running in UG.<commit_after>Basic usage
-----------

Robot Framework test cases are executed from the command line, and the
end result is, by default, an output file in XML format and an HTML
report and log. After the execution, `output files`__ can be combined and
otherwise `post-processed`__ with the :prog:`rebot` tool.

__ `Created outputs`_
__ `Post-processing outputs`_

.. contents::
   :depth: 2
   :local:

Starting test execution
~~~~~~~~~~~~~~~~~~~~~~~

Synopsis
''''''''

::

   pybot [options] datasources
   jybot [options] datasources
   interpreter /path/to/robot/runner.py [options] datasources
   java -jar robotframework-<version>.jar [options] datasources

Different runner scripts
''''''''''''''''''''''''

Test execution is normally started with the :prog:`pybot` or
:prog:`jybot` commands. These commands are otherwise identical, but
the former executes tests using the Python interpreter and the latter uses
Jython. Which one to use depends on the needed test libraries. Some
libraries use modules or syntax available only on Python, others use
Java-based tools that require Jython, and some work on both. If you
can use either :prog:`pybot` or :prog:`jybot`, the former is
recommended, as Python is somewhat faster than Jython.

Starting from Robot Framework 2.5.2, it is also possible to get Robot Framework
as a single jar file.  In this case, Jython is bundled inside the jar, and the
above discussion about Jython applies. Normal arguments to
Robot Framework are given after the jar file name.

Another possibility for starting the test execution is running the
:prog:`runner.py` script under the installed__ :code:`robot` module
directly. This method allows selecting the interpreter and setting
command line options to it freely. The most common use case is
altering the options controlling `JVM maximum memory consumption`__.

__ `Where files are installed`_
__ `Modifying Java startup parameters`_

Regardless of the runner script, the path (or paths) to the test data to be
executed is given as an argument. Additionally, different command line
options can be used to alter the test execution or generated
outputs in some way.

Specifying test data to be executed
'''''''''''''''''''''''''''''''''''

Robot Framework test cases are created in files__ and directories__,
and they are executed by giving the path to the file or directory in
question to the selected runner script. The path can be absolute or,
more commonly, relative to the directory where tests are executed
from. The given file or directory creates the top-level test suite,
which gets its name, unless overridden with the :opt:`--name` option__,
from the `file or directory name`__. Different execution possibilities
are illustrated in the examples below. Note that in these examples, as
well as in other examples in this section, only the :prog:`pybot`
command is used, but :prog:`jybot` or a custom runner script
could be used similarly.

__ `Test case files`_
__ `Test suite directories`_
__ `Setting the name`_
__ `Test suite name and documentation`_

::

   pybot test_cases.html
   pybot path/to/my_tests/
   pybot c:\robot\tests.txt

It is also possible to give paths to several test case files or
directories at once, separated with spaces. In this case, Robot
Framework creates the top-level test suite automatically, and
the specified files and directories become its child test suites. The name
of the created test suite is got from child suite names by
catenating them together with an ampersand (&) and spaces. For example,
the name of the top-level suite in the first example below is
:name:`My Tests & Your Tests`. These automatically created names are
often quite long and complicated. In most cases, it is thus better to
use the :opt:`--name` option for overriding it, as in the second
example below::

   pybot my_tests.html your_tests.html
   pybot --name Example path/to/tests/pattern_*.html

Using command line options
~~~~~~~~~~~~~~~~~~~~~~~~~~

Robot Framework provides a number of command line options that can be
used to control how test cases are executed and what outputs are
generated. This section explains the option syntax, and what
options actually exist. How they can be used is discussed elsewhere
in this chapter.

Using options
'''''''''''''

When options are used, they must always be given between the runner
script and the data sources. For example::

   pybot -L debug my_tests.txt
   jybot --include smoke --variable HOST:10.0.0.42 path/to/tests/

Short and long options
''''''''''''''''''''''

Options always have a long name, such as :opt:`--name`, and the
most frequently needed options also have a short name, such as
:opt:`-N`. In addition to that, long options can be shortened as
long as they are unique. For example, :cli:`--logle DEBUG` works,
while :cli:`--lo log.html` does not, because the former matches only
:opt:`--loglevel`, but the latter matches several options. Short
and shortened options are practical when executing test cases
manually, but long options are recommended in `start-up scripts`__,
because they are easier to understand.

__ `Creating start-up scripts`_

The long option format is case-insensitive, which facilitates writing option
names in an easy-to-read format. For example, :opt:`--SuiteStatLevel`
is equivalent to, but easier to read than :opt:`--suitestatlevel`.

Setting option values
'''''''''''''''''''''

Most of the options require a value, which is given after the option
name. Both short and long options accept the value separated
from the option name with a space, as in :cli:`--include tag`
or :cli:`-i tag`. With long options, the separator can also be the
equals sign, for example :cli:`--include=tag`, and with short options the
separator can be omitted, as in :cli:`-itag`.

Some options can be specified several times. For example,
:cli:`--variable VAR1:value --variable VAR2:another` sets two
variables. If the options that take only one value are used several
times, the value given last is effective.

Option value as simple pattern
''''''''''''''''''''''''''''''

Many of the options take arguments as *simple patterns*. This means
that :code:`*` and :code:`?` can be used as special characters, so
that the former matches any string (even an empty string) and the
latter matches any single character. For example, :cli:`--include
prefix-\*` matches all tags starting with :code:`prefix-`, and
:cli:`--include a???` matches any tag that is four characters long and
starts with a character :code:`a`.

Test results
~~~~~~~~~~~~

Command line output
'''''''''''''''''''

The most visible output from test execution is the output displayed in
the command line. All executed test suites and test cases, as well as
their statuses, are shown there in real time. The example below shows the
output from executing a simple test suite with only two test cases::

   ==============================================================================
   Example test suite
   ==============================================================================
   First test :: Possible test documentation                             | PASS |
   ------------------------------------------------------------------------------
   Second test                                                           | FAIL |
   Error message is displayed here
   ==============================================================================
   Example test suite                                                    | FAIL |
   2 critical tests, 1 passed, 1 failed
   2 tests total, 1 passed, 1 failed
   ==============================================================================
   Output:  /path/to/output.xml
   Report:  /path/to/report.html
   Log:     /path/to/log.html

Starting from Robot Framework 2.7, there is also a notification on the console
whenever a top-level keyword in a test case ends. A green dot is used if
a keyword passes and a red F if it fails. These markers are written to the end
of line and they are overwritten by the test status when the test itself ends.
Writing the markers is disabled if console output is redirected to a file.

Generated output files
''''''''''''''''''''''

The command line output is very limited, and separate output files are
normally needed for investigating the test results. As the example
above shows, three output files are generated by default. The first
one is in XML format and contains all the information about test
execution. The second is a higher-level report and the third is a more
detailed log file. These files and other possible output files are
discussed in more detail in the section `Different output files`_.

Return codes
''''''''''''

Runner scripts communicate the overall test execution status to the
{'fraction_non_alphanumeric': 0.12786292907512092, 'fraction_numerical': 0.002214581269304738, 'mean_word_length': 4.016077170418006, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7552652', 'n_tokens_mistral': 4349, 'n_tokens_neox': 4145, 'n_words': 2315}