<commit_before>#!/usr/bin/perl
use strict;
use warnings;
use FindBin;
use lib "$FindBin::RealBin/lib";
use Data::Dumper;
$Data::Dumper::Sortkeys = 1;
use Getopt::Long;
use Pod::Usage;
use File::Path qw(make_path);
use File::Basename;

my %options = ();

=head1 NAME 

classify_reads.pl

=head1 DESCRIPTION

This script wraps the preprocessing and classification of multiple fastq files.
Input are multiple unjoined ITS2 read fastq files and output is a folder with a tax_table and otu_table for import into phyloseq.
The intermediate files are also preserved.
The steps are join (with fastq-join), filter (with usearch), classify (with usearch or RDPclassifier), count, aggregate and convert.
Requirements (tested versions): fastq-join (Version 1.01.759), usearch (Version v8.0.1477)[, RDPclassifier (Version 2.10.2)]
The readfiles have to be ordered (alternating first read file and second read file of the same sample).
With most naming schemes this should be automatically the case when using wildcards but be aware of this requirement if you face problems.
The file name of the first read file will be used for the joined and filtered files.

=head1 USAGE

  $ perl classify_reads.pl --out=<dir> [options] <sample1_1>.fq <sample1_2>.fq [<sample2_1>.fq <sample2_2>.fq ...]

=cut

=head1 OPTIONS

=over 25

=item --out=<dir>

Output directory will contain a joined and filtered subfolder with the respective reads

=cut

$options{'out=s'} = \( my $opt_out );

=item --utax-db=<file>

Path to the utax database file in fasta or udb format

=cut

$options{'utax-db=s'} = \( my $opt_utax_db );

=item --utax-taxtree=<file>

Path to the utax taxtree file

=cut

$options{'utax-taxtree=s'} = \( my $opt_utax_tt );

=item [--utax-rawscore-cutoff=<int>]

This value is passed to the counting script and takes the cuts the classification at the first level with rawscore below the cutoff
Default = 20

=cut<commit_msg>added --[no]utax option to specify if utax should be used for classification<commit_after>#!/usr/bin/perl
use strict;
use warnings;
use FindBin;
use lib "$FindBin::RealBin/lib";
use Data::Dumper;
$Data::Dumper::Sortkeys = 1;
use Getopt::Long;
use Pod::Usage;
use File::Path qw(make_path);
use File::Basename;

my %options = ();

=head1 NAME 

classify_reads.pl

=head1 DESCRIPTION

This script wraps the preprocessing and classification of multiple fastq files.
Input are multiple unjoined ITS2 read fastq files and output is a folder with a tax_table and otu_table for import into phyloseq.
The intermediate files are also preserved.
The steps are join (with fastq-join), filter (with usearch), classify (with usearch or RDPclassifier), count, aggregate and convert.
Requirements (tested versions): fastq-join (Version 1.01.759), usearch (Version v8.0.1477)[, RDPclassifier (Version 2.10.2)]
The readfiles have to be ordered (alternating first read file and second read file of the same sample).
With most naming schemes this should be automatically the case when using wildcards but be aware of this requirement if you face problems.
The file name of the first read file will be used for the joined and filtered files.

=head1 USAGE

  $ perl classify_reads.pl --out=<dir> [options] <sample1_1>.fq <sample1_2>.fq [<sample2_1>.fq <sample2_2>.fq ...]

=cut

=head1 OPTIONS

=over 25

=item --out=<dir>

Output directory will contain a joined and filtered subfolder with the respective reads

=cut

$options{'out=s'} = \( my $opt_out );

=item [--[no]utax]

Should utax be used for classification (default: yes)

=cut

$options{'utax!'} = \( my $opt_utax );

=item --utax-db=<file>

Path to the utax database file in fasta or udb format (required if not --noutax)

=cut

$options{'utax-db=s'} = \( my $opt_utax_db );

=item --utax-taxtree=<file>

Path to the utax taxtree file (required if not --noutax)

=cut

$options{'utax-taxtree=s'} = \( my $opt_utax_tt );

=item [--utax-rawscore-cutoff=<int>]

This value is passed to the counting script and takes the cuts the classification at the first level with rawscore below the cutoff
Default = 20

=cut
{'fraction_non_alphanumeric': 0.10364352535696701, 'fraction_numerical': 0.01674052191038897, 'mean_word_length': 4.611878453038674, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4785777', 'n_tokens_mistral': 1372, 'n_tokens_neox': 1304, 'n_words': 528}