<commit_before>#!/usr/bin/env python

#  Copyright 2008-2009 Nokia Siemens Networks Oyj
#
#  Licensed under the Apache License, Version 2.0 (the "License");
#  you may not use this file except in compliance with the License.
#  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


"""Tool for cleaning up Robot Framework test data.

Usage:  robotidy.py [options] inputfile outputfile
  or:   robotidy.py --inplace [options] inputfile [more input files]
  or:   robotidy.py --recursive [options] directory

Options:
 -I --inplace      Tidy given file(s) so that original file(s) are overwritten
                   When this option is used, it is possible to give multiple
                   files at once. Examples:
                   robotidy.py --inplace tests.html
                   robotidy.py --inplace --fixcomments *.html
 -R --recursive    Process given directory recursively. Files in the directory
                   are processed in place similarly as when '--inplace'
                   option is used.
 -X --fixcomments  Fix comments in the test data.
 -F --format html|tsv|txt
                   Format to use for output. Possible values are HTML, TSV and
                   TXT. If this option is not used, the format is got from the
                   extension of the output file.
 -T --title text   Title to use in the test data. By default the title is got 
                   from the name of the output file. If the output file is
                   HTML the title is used with 'h1' and 'title' tags, and with
                   TSV it is simply printed before the first table. Possible
                   underscores in the given title are converted to spaces.
 -S --style path   Read styles from an external style sheet file and replace
                   default styles with them. If the path has a special value
                   'NONE', no styles are used. This setting is applicable only
                   when the output format is HTML.
 -h --help         Print this help.


This multipurpose tool has three main usages listed below.

1) Clean up the test data.

Source code created by most HTML editors is quite ugly. With this tool the 
source can be formatted nicely and it is even possible to specify a custom
style sheet. Additionally test data itself is cleaned up so that settings and
metadata is always in same predefined order.

Examples:
  robotidy.py messed_up_tests.html cleaned_tests.html 
  robotidy.py --style new_styles.css my_tests.html my_tests.html

2) Change format between HTML, TSV and TXT.

Robot Framework supports test data in HTML, TSV and TXT formats and this tools
makes changing between formats trivial. Input format is always determined from
the extension of the input file. Output format is also got from the output file
extension by default but it can also be set explicitly with '--format' option.

Examples:
  robotidy.py tests_in_html.html tests_in_tsv.tsv
  robotidy.py --format html tests.tsv tests.xxx

3) Fix comments.

Robot IDE is a great tool for editing the test data but at least currently it
totally ignores comments. This tool can be used to convert comments so that 
they are not lost when the test data is opened to the IDE. Comments in test 
case and user keyword tables are changed so that instead of '#' a built-in 
'Comment' keyword is used. In variable tables comments are converted to '@{#}'
or '${#}'. Comments in setting tables are not fixed. 

Examples:
  robotidy.py --fixcomments orig.html fixed.html
  robotidy.py --fixcomments --inplace *.html


Outputs are always written using UTF-8 encoding.
"""
<commit_msg>rm unnecessary empty line in doc<commit_after>#!/usr/bin/env python

#  Copyright 2008-2009 Nokia Siemens Networks Oyj
#
#  Licensed under the Apache License, Version 2.0 (the "License");
#  you may not use this file except in compliance with the License.
#  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#  See the License for the specific language governing permissions and
#  limitations under the License.


"""Tool for cleaning up Robot Framework test data.

Usage:  robotidy.py [options] inputfile outputfile
  or:   robotidy.py --inplace [options] inputfile [more input files]
  or:   robotidy.py --recursive [options] directory

Options:
 -I --inplace      Tidy given file(s) so that original file(s) are overwritten
                   When this option is used, it is possible to give multiple
                   files at once. Examples:
                   robotidy.py --inplace tests.html
                   robotidy.py --inplace --fixcomments *.html
 -R --recursive    Process given directory recursively. Files in the directory
                   are processed in place similarly as when '--inplace'
                   option is used.
 -X --fixcomments  Fix comments in the test data.
 -F --format html|tsv|txt
                   Format to use for output. Possible values are HTML, TSV and
                   TXT. If this option is not used, the format is got from the
                   extension of the output file.
 -T --title text   Title to use in the test data. By default the title is got 
                   from the name of the output file. If the output file is
                   HTML the title is used with 'h1' and 'title' tags, and with
                   TSV it is simply printed before the first table. Possible
                   underscores in the given title are converted to spaces.
 -S --style path   Read styles from an external style sheet file and replace
                   default styles with them. If the path has a special value
                   'NONE', no styles are used. This setting is applicable only
                   when the output format is HTML.
 -h --help         Print this help.


This multipurpose tool has three main usages listed below.

1) Clean up the test data.

Source code created by most HTML editors is quite ugly. With this tool the 
source can be formatted nicely and it is even possible to specify a custom
style sheet. Additionally test data itself is cleaned up so that settings and
metadata is always in same predefined order.

Examples:
  robotidy.py messed_up_tests.html cleaned_tests.html 
  robotidy.py --style new_styles.css my_tests.html my_tests.html

2) Change format between HTML, TSV and TXT.

Robot Framework supports test data in HTML, TSV and TXT formats and this tools
makes changing between formats trivial. Input format is always determined from
the extension of the input file. Output format is also got from the output file
extension by default but it can also be set explicitly with '--format' option.

Examples:
  robotidy.py tests_in_html.html tests_in_tsv.tsv
  robotidy.py --format html tests.tsv tests.xxx

3) Fix comments.

Robot IDE is a great tool for editing the test data but at least currently it
totally ignores comments. This tool can be used to convert comments so that 
they are not lost when the test data is opened to the IDE. Comments in test 
case and user keyword tables are changed so that instead of '#' a built-in 
'Comment' keyword is used. In variable tables comments are converted to '@{#}'
or '${#}'. Comments in setting tables are not fixed. 

Examples:
  robotidy.py --fixcomments orig.html fixed.html
  robotidy.py --fixcomments --inplace *.html

Outputs are always written using UTF-8 encoding.
"""

{'fraction_non_alphanumeric': 0.05459119496855346, 'fraction_numerical': 0.004276729559748427, 'mean_word_length': 2.679315131883387, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7430933', 'n_tokens_mistral': 2079, 'n_tokens_neox': 1995, 'n_words': 1103}