<commit_before>Carrot2 Document Clustering Server
----------------------------------

Requirements
------------

All applications in this package require Java Runtime Environment (JRE)
version 1.4.2 or newer. JRE can be downloaded free of charge from:

http://java.sun.com/javase/downloads/index.jsp


Applications
------------

This package provides the following applications:


1) batch

Clusters XML files and saves the results to an output folder. 
Format for the input data is the following:

<searchresult>
    <query>new york hotels</query>
    <document>
    	<title>NewYork.com</title>
        <snippet>guide to New York City with links to hotels, attractions, shopping, restaurants, and more. NewYork.com also features local New York news, arts, and sports.</snippet>
    </document>

    <document>
        ...
    </document>
</searchresult>
<commit_msg>carrot2-demo-dcs: brief information about building the applications added.<commit_after>Carrot2 Document Clustering Server
----------------------------------

Requirements
------------

All applications in this package require Java Runtime Environment (JRE)
version 1.4.2 or newer. JRE can be downloaded free of charge from:

http://java.sun.com/javase/downloads/index.jsp


Building
--------

To build the DCS, run: ant dist. You will find the assembled applications
in the tmp/dist/web directory.


Applications
------------

This package provides the following applications:


1) batch

Clusters XML files and saves the results to an output folder. 
Format for the input data is the following:

<searchresult>
    <query>new york hotels</query>
    <document>
    	<title>NewYork.com</title>
        <snippet>guide to New York City with links to hotels, attractions, shopping, restaurants, and more. NewYork.com also features local New York news, arts, and sports.</snippet>
    </document>

    <document>
        ...
    </document>
</searchresult>

For a description of the batch application switches, run the script
(batch.cmd, batch.sh) without parameters.


{'fraction_non_alphanumeric': 0.13330054107230693, 'fraction_numerical': 0.0054107230693556324, 'mean_word_length': 2.874285714285714, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2266944', 'n_tokens_mistral': 555, 'n_tokens_neox': 526, 'n_words': 220}