<commit_before>
Internal documentation on how to make a LanguageTool release

-make a SVN update
-download latest translations from http://crowdin.net/download/project/languagetool.zip
 and put them into SVN using ./i18n_update.sh
-set version number in build.properties
-set JLanguageTool.VERSION in JLanguageTool.java
-set the local IDE settings back to Java 1.5 and make sure that any compile problems
 are compile-time only issues, i.e. that the code will run under Java 1.5
 (e.g. using @Override is okay, using methods introduced in Java 1.6 is not)
-update CHANGES files
-update README file (e.g. list of supported languages)
-make sure there are no other local changes left that are not committed
-run "ant test"
-run "ant" 
-test command line application, OOo integration, and stand-alone GUI version
 with at least these test sentences: 
    I've got allot of questions to you. (en)
    Seit 10 Jahre. (de)
    El agua es de gratis. (es)
    Je voudrais ajouter en plus quelque chose. (fr)
    Ovviamente... è tutto a gratis! (it)
    Dit is goed, en niet foutief. (nl)
    Kupiłem wino chciałem też pić wodę. (pl)
    Стоимость работ составила 10 тыс рублей. (ru)
-upload to the server:
    -dist/*.oxt, dist/README.txt and dist/CHANGES.txt to "download"
    -run "ant javadoc" and upload dist/docs/api/* to "development"
-run "ant jnlp" (note you need file myalias.p12 with the certificate to do this;
	for security reasons it is not in the repository.)
-upload to the server:
    -dist/web/ to www/webstart/web/ (resulting from "ant jnlp")
-commit local version number changes etc. to SVN
-tag release as V_x_y, e.g. V_1_3
 (Eclipse: 'Team -> Tag as Version' on the JLanguageTool project)

-update website
    -index.php (remember to also set $lastmod)
    -languages/index.php (run de.danielnaber.languagetool.dev.RuleOverview,
     remember to also set $lastmod)
    -languagetool.update.xml (active only if it's linked in description.xml)
    -...

-build.properties, JLanguageTool.VERSION in JLanguageTool.java:
    -set to next version: x.y-dev

-write announcements:
    -LanguageTool mailing list
    -http://extensions.services.openoffice.org/project/languagetool
    -freshmeat
    -http://twitter.com/languagetoolorg
    -Daniel's blog
	-Polish blog: morfologik.blogspot.com
	-pl.openoffice.org
	
	
Community update of the Wikipedia data on community.languagetool.org (needs server access):

 -unzip the current release in /home/languagetool/ltcommunity/corpus/code
 -if the CheckWikipediaDump.java itself has changed, upload the *.class files built by Eclipse
  to /home/languagetool/ltcommunity/corpus/de/danielnaber/languagetool/dev/wikipedia/ 
 -in /home/languagetool/ltcommunity/corpus, call this for each language ("de" in this example):
    ./update.sh de
    (this will run ./clean.sh de, ./download.sh de and ./check.sh de - it will download only part of the dump)
  Language codes are: en de pl fr es it nl lt uk ru sl sv ro is gl ca

Community update of the rules at community.languagetool.org (needs server access):

 -create a new JAR of the rules and resources of the current release:
 	zip -r LanguageTool-Rules.jar resource/ rules/
 -make a backup of /home/languagetool/tomcat/webapps/ROOT/WEB-INF/lib/LanguageTool-Rules.jar and LanguageTool.jar
  to a different directory
 -copy the newly created LanguageTool-Rules.jar and LanguageTool.jar from the current release to
  /home/languagetool/tomcat/webapps/ROOT/WEB-INF/lib
 -maybe update the other JARs, too
 -restart tomcat
 <commit_msg>small update for community website<commit_after>
Internal documentation on how to make a LanguageTool release

-make a SVN update
-download latest translations from http://crowdin.net/download/project/languagetool.zip
 and put them into SVN using ./i18n_update.sh
-set version number in build.properties
-set JLanguageTool.VERSION in JLanguageTool.java
-set the local IDE settings back to Java 1.5 and make sure that any compile problems
 are compile-time only issues, i.e. that the code will run under Java 1.5
 (e.g. using @Override is okay, using methods introduced in Java 1.6 is not)
-update CHANGES files
-update README file (e.g. list of supported languages)
-make sure there are no other local changes left that are not committed
-run "ant test"
-run "ant" 
-test command line application, OOo integration, and stand-alone GUI version
 with at least these test sentences: 
    I've got allot of questions to you. (en)
    Seit 10 Jahre. (de)
    El agua es de gratis. (es)
    Je voudrais ajouter en plus quelque chose. (fr)
    Ovviamente... è tutto a gratis! (it)
    Dit is goed, en niet foutief. (nl)
    Kupiłem wino chciałem też pić wodę. (pl)
    Стоимость работ составила 10 тыс рублей. (ru)
-upload to the server:
    -dist/*.oxt, dist/README.txt and dist/CHANGES.txt to "download"
    -run "ant javadoc" and upload dist/docs/api/* to "development"
-run "ant jnlp" (note you need file myalias.p12 with the certificate to do this;
	for security reasons it is not in the repository.)
-upload to the server:
    -dist/web/ to www/webstart/web/ (resulting from "ant jnlp")
-commit local version number changes etc. to SVN
-tag release as V_x_y, e.g. V_1_3
 (Eclipse: 'Team -> Tag as Version' on the JLanguageTool project)

-update website
    -index.php (remember to also set $lastmod)
    -languages/index.php (run de.danielnaber.languagetool.dev.RuleOverview,
     remember to also set $lastmod)
    -languagetool.update.xml (active only if it's linked in description.xml)
    -...

-build.properties, JLanguageTool.VERSION in JLanguageTool.java:
    -set to next version: x.y-dev

-write announcements:
    -LanguageTool mailing list
    -http://extensions.services.openoffice.org/project/languagetool
    -freshmeat
    -http://twitter.com/languagetoolorg
    -Daniel's blog
	-Polish blog: morfologik.blogspot.com
	-pl.openoffice.org
	
	
Community update of the Wikipedia data on community.languagetool.org (needs server access):

 -unzip the current release in /home/languagetool/ltcommunity/corpus/code
 -if the CheckWikipediaDump.java itself has changed, upload the *.class files built by Eclipse
  to /home/languagetool/ltcommunity/corpus/de/danielnaber/languagetool/dev/wikipedia/ 
 -in /home/languagetool/ltcommunity/corpus, call this for each language ("de" in this example):
    ./update.sh de
    (this will run ./clean.sh de, ./download.sh de and ./check.sh de - it will download only part of the dump)
  Language codes are: en de pl fr es it nl lt uk ru sl sv ro is gl ca

Community update of the rules at community.languagetool.org (needs server access):

 -create a new JAR of the rules and resources of the current release:
 	zip -r LanguageTool-Rules.jar resource/ rules/
 -make a backup of the libs:
  cp /home/languagetool/tomcat/webapps/ROOT/WEB-INF/lib/*.jar /home/languagetool/tomcat/backup/
 -copy the newly created LanguageTool-Rules.jar and LanguageTool.jar from the current release to
  /home/languagetool/tomcat/webapps/ROOT/WEB-INF/lib
 -maybe update the other JARs, too
 -restart tomcat
 
{'fraction_non_alphanumeric': 0.08485621192803514, 'fraction_numerical': 0.004533220002833262, 'mean_word_length': 3.537275064267352, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2632175', 'n_tokens_mistral': 2332, 'n_tokens_neox': 2244, 'n_words': 880}