<commit_before>## This is the home of the new parser Parrot, which is based on antlr4.

The new parser(Parrot) can parse Groovy source code and construct the related AST, which is almost identical to the one generated by the old parser(except the corrected node position, e.g. line, column of node). Currently all features of Groovy are available. In addition, **the following new features have been added:**

* do-while loop, standard loop(e.g. `for(int i = 0, j = 10; i < j; i++, j--) {..}`)
* lambda expression
* method reference and constructor reference
* try-with-resources(i.e. ARM)
* code block(i.e. `{..}`)
* array initializer of Java style(e.g. `new int[] {1, 2, 3}`)
* default method of interface
* new operators: identity operators(`===`, `!==`), elvis assignment(`?=`), `!in`, `!instanceof`
* safe index(e.g. `nullableVar?[1, 2]`)
* groovydoc attached to AST node as metadata

**How to enable the new parser**
* In the gradle build the property useAntlr4 has to be set to enable the build of the parser and the execution of all tests with it. Command line example:
```
./gradlew -PuseAntlr4=true bootstrapJar
```
* To enable the new parser automatically at runtime the system property groovy.antlr4 has to be set. Command line example:
```
export JAVA_OPTS="-Dgroovy.antlr4=true"
groovy foo.groovy
```
* This system property also controls groovyc and has to be used in case it is used outside of this build, for example with:
```
groovyOptions.forkOptions.jvmArgs += ["-Dgroovy.antlr4=true"]
```

**JVM options to control parsing:**
* `groovy.antlr4.cache.threshold`: how frequently to clear DFA cache(default: 50). **Notice:** The more frequently the DFA cache is cleared, the poorer parsing performance will be(you can not set the value that is less than the default value). But the DFA cache has to be cleared to avoid OutOfMemoryError's occurring. 
* `groovy.extract.doc.comment`: whether to collect groovydoc while parsing groovy source code(default: false)

*P.S. Parrot is based on the highly optimized version of antlr4(com.tunnelvisionlabs:antlr4), which is licensed under BSD.*
 <commit_msg>Update the readme of the Parrot sub-project<commit_after>## This is the home of the new parser Parrot, which is based on antlr4.

The new parser(Parrot) can parse Groovy source code and construct the related AST, which is almost identical to the one generated by the old parser(except the corrected node position, e.g. line, column of node). Currently all features of Groovy are available. In addition, **the following new features have been added:**

* do-while loop, standard loop(e.g. `for(int i = 0, j = 10; i < j; i++, j--) {..}`)
* lambda expression
* method reference and constructor reference
* try-with-resources(i.e. ARM)
* code block(i.e. `{..}`)
* array initializer of Java style(e.g. `new int[] {1, 2, 3}`)
* default method of interface
* new operators: identity operators(`===`, `!==`), elvis assignment(`?=`), `!in`, `!instanceof`
* safe index(e.g. `nullableVar?[1, 2]`)
* groovydoc attached to AST node as metadata

**How to enable the new parser**
* In the gradle build the property useAntlr4 has to be set to enable the build of the parser and the execution of all tests with it. Command line example:
```
./gradlew -PuseAntlr4=true bootstrapJar
```
* To enable the new parser automatically at runtime the system property groovy.antlr4 has to be set. Command line example:
```
export JAVA_OPTS="-Dgroovy.antlr4=true"
groovy foo.groovy
```
* This system property also controls groovyc and has to be used in case it is used outside of this build, for example with:
```
groovyOptions.forkOptions.jvmArgs += ["-Dgroovy.antlr4=true"]
```

**JVM system properties to control parsing:**
* `groovy.antlr4.cache.threshold`: how frequently to clear DFA cache(default: 50). **Notice:** The more frequently the DFA cache is cleared, the poorer parsing performance will be(you can not set the value that is less than the default value). But the DFA cache has to be cleared to avoid OutOfMemoryError's occurring. 
* `groovy.extract.doc.comment`: whether to collect groovydoc while parsing groovy source code(default: false)

*P.S. Parrot is based on the highly optimized version of antlr4(com.tunnelvisionlabs:antlr4), which is licensed under BSD.*
 
{'fraction_non_alphanumeric': 0.10660731021555764, 'fraction_numerical': 0.008903467666354265, 'mean_word_length': 4.669322709163347, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7309464', 'n_tokens_mistral': 1315, 'n_tokens_neox': 1233, 'n_words': 581}