<commit_before># $Id$

# Ensembl module for Bio::EnsEMBL::Collection
#
# You may distribute this module under the same terms as perl itself.

=head1 NAME

Bio::EnsEMBL::Collection - Abstract base class for feature collection
classes.

=head1 SYNOPSIS

  use Bio::EnsEMBL::Collection::RepeatFeature;

  # Pick a slice.
  my $slice =
    $slice_adaptor->fetch_by_region( 'Chromosome', '2', 1, 1e9 );

  # Create a feature collection on the slice.
  my $collection =
    Bio::EnsEMBL::Collection::RepeatFeature->new( -slice => $slice,
                                                  -light => 0 );

  # Populate the feature collection from the slice.
  $collection->populate();

  # Populate the feature collection from the slice.  Sort the
  # entries on feature start position.
  $collection->populate( -sorted => 1 );

  # Populate the feature collection from the slice.  Sort the
  # entries on feature length and don't bother about feature
  # type-specific data.
  $collection->populate(
    -light   => 1,
    -sorted  => 1,
    -sortsub => sub {
      $a->[3] - $a->[2] <=> $b->[3] - $b->[2];
    } );

  # Retrieve the entries from the collection as an array of arrays.
  my @entries = @{ $collection->entries() };

  # Retrieve a binned representation of the entries using 100 bins.
  my @binned_entries = @{ $collection->get_bin_entries(100) };

  # Retrieve a binned representation of the entries using 100 bins,
  # where each entry is represented by its index in the feature
  # collection array (@entries above).
  my @binned_entry_indicies =
    @{ $collection->get_bin_entry_indices(100) };

  # Retrieve only the bin counts.
  my @bin_counts = @{ $collection->get_bin_counts(100) };

=head1 DESCRIPTION

This is the abstract base class for feature collections.

A feature collection provides a compact representation of features of a
particular type on a slice.  Each entry in a collection is a short array
of data representing a feature.  This data is divided into two halfs:

=over 4

=item 1.

Basic feature representation.

=item 2.

Extended feature representation.

=back

=head2 Basic feature representation

The basic feature representation is common to all entries in any type
of feature collection and consists of a minimal set of values.  Each
collection entry is an array that contains at least the following data
(in this order)

=over 4

=item 1.

Ensembl internal database ID.

=item 2.

Ensembl internal sequence region ID.

=item 3.

Feature start position.

=item 4.

Feature end position.

=item 5.

Feature strand.

=back

The module defines a number of constants that may be used
as symbolic constants in place of the index numbers 0 to
4: ENTRY_DBID, ENTRY_SEQREGIONID, ENTRY_SEQREGIONSTART,
ENTRY_SEQREGIONEND, ENTRY_SEQREGIONSTRAND.  For an entry $entry,
$entry->[Bio::EnsEMBL::Collection::ENTRY_SEQREGIONEND] will thus be the
end position for the feature that the entry represents.

The position of the feature is in the same coordinate system as the
slice associated with the collection object.

=head2 Extended feature representation

A sub-class of this abstract base class will specify further data to be
added to the entries in order to account of the particular feature type.
An entry from a gene feature collection (Bio::EnsEMBL::Collection::Gene)
might, for example, contain the Ensembl Stable ID of the gene.

=head2 Light-weight collections/entries

A light-weight collection is a feature collection whose collection
entries are light-weight, whose entries does not contain the extended
feature representation.

A collection which is light-weight by default may be created by using
the argument '-light=>1' in the constructor, new(), but one may also use
the same argument with populate().

=head2 Binning methods

This module allows for various ways of binning the result of the
populate() method by using the get_bins() method.

Each binning method bins the collection entries and gives an array
with the specified length (number of bins).  An entry, which basically
consists of a start and a end position, is allocated to one or several
bins depending on its span and the size of the individual bins.

 Features:     |----|     |----------------|  |--|         |------|
               |-------------|                |-----|             |--|
               |-------------------|                 |----|       |--|

 Finer bins:   3 3 3 2 2 2 3 3 2 2 2 1 1 1 1 0 2 2 1 1 1 1 1 1 1 1 2 2
 Coarser bins: 3  3  2  2  3  2  2  1  1  1  0  2  1  1  1  1  1  3  2

The example above shows the arrays that might be returned from
get_bins() when using the 'count' binning method with 28 and 19 bins
respectively.

Here follows a brief description of each implemented binning method.

=over 4

=item  'count' and 'density'

Returns an array of bins, each bin containing the number of entries
allocated to (i.e. overlapping) that bin.  The 'density' binning method
is equivalent to 'count' and this is also the default binning method.

=item 'indices' and 'index'

Returns an array of bins, each bin containing an array of indices into
the collection entry array (available from the entries() method) for the
entries allocated to that bin.  The 'index' binning method is equivalent
to 'indicies'.

=item 'entries' and 'entry'

Returns an array of bins, each bin containing an array of entries
(references into the array of entries retrieved from the entries()
method) allocated to that bin.  The 'entry' binning method is equivalent
to 'entries'.

=back

=head1 CONTACT

This modules is part of the Ensembl project.  See
http://www.ensembl.org/ for further information.

Questions may be posted to the ensembl-dev mailing list:
<EMAIL>

=cut

package Bio::EnsEMBL::Collection;

use strict;
use warnings;

use Bio::EnsEMBL::DBSQL::BaseAdaptor;

use Bio::EnsEMBL::Utils::Argument qw( rearrange );
use Bio::EnsEMBL::Utils::Exception qw( throw );

use base qw( Bio::EnsEMBL::DBSQL::BaseAdaptor );

# Symbolic constants that acts as indices into the individual feature
# collection entries.  These must be in sync with the columns returned
# by the _columns() method.
use constant { ENTRY_DBID            => 0,
               ENTRY_SEQREGIONID     => 1,
               ENTRY_SEQREGIONSTART  => 2,
               ENTRY_SEQREGIONEND    => 3,
               ENTRY_SEQREGIONSTRAND => 4 };

=head1 METHODS (constructor)

=head2 new

  Arg [SLICE]   : Bio::EnsEMBL::Slice
                  The slice to be associated with this feature
                  collection.

  Arg [LIGHT]   : Boolean (optional, default false)
                  If true, the collection will be 'light-weight',
                  i.e. no type-specific data will be stored in its
                  entries when populate() is called.

  Example       : my $collection =
                    Bio::EnsEMBL::Collection::<feature_type>->new(
                                                       -slice => $slice,
                                                       -light => 1 );

  Description   : When called for a sub-class, creates a feature
                  collection object and associates a slice with it.

  Return type   : Bio::EnsEMBL::Collection::<feature_type>

  Exceptions    : Throws if no slice is specified.

  Caller        : General (through a sub-class).

  Status        : At Risk (under development)

=cut

sub new {
  my $proto = shift;
  my ( $slice, $light ) = rearrange( [ 'SLICE', 'LIGHT' ], @_ );

  if ( !defined($slice) ) {
    throw(   'Unable to create a feature collection '
           . 'without a slice object, I am.' );
  }

  my $this = $proto->SUPER::new( $slice->adaptor()->db() );

  my $sql = qq(
    SELECT  cs.name, mc.max_length, m.meta_value
    FROM    coord_system cs,
            meta_coord mc
    LEFT JOIN meta m ON m.meta_key = ?
    WHERE   mc.table_name = ?
    AND     mc.coord_system_id = cs.coord_system_id
  );

  my $sth           = $this->prepare($sql);
  my $feature_table = $this->_feature_table()->[0];
  $sth->execute( $feature_table . 'build.level', $feature_table );

  my ( $cs_name, $max_length, $build_level );
  $sth->bind_columns( \( $cs_name, $max_length, $build_level ) );

  my %coordinate_systems;

  while ( $sth->fetch() ) {
    $coordinate_systems{$cs_name} = {
            'name'               => $cs_name,
            'max_feature_length' => $max_length,
            'build_level'        => $build_level    # Not currently used
    };
  }
  $sth->finish();

  $this->__attrib( 'coordinate_systems', \%coordinate_systems );

  $this->lightweight($light);
  $this->slice($slice);

  return $this;
} ## end sub new

#-----------------------------------------------------------------------

=head1 METHODS (public)

=head2 slice

  Arg [1]       : Bio::EnsEMBL::Slice (optional)
                  The new slice to be associated with this
                  collection.

  Example       : my $slice = $collection->slice();

  Description   : Getter/setter for the main slice associated with
                  this collection.  Associating a new slice with
                  the collection will empty all entries from the
                  collection.

  Return type   : Bio::EnsEMBL::Slice

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

sub slice {
  my ( $this, $slice ) = @_;

  if ( defined($slice) ) {
    my %seqreg_map;
    my @all_segments;

    foreach
      my $cs_name ( keys( %{ $this->__attrib('coordinate_systems') } ) )
    {
      my @segments = @{ $slice->project($cs_name) };

      foreach my $segment (@segments) {
        $seqreg_map{ $segment->to_Slice()->get_seq_region_id() } =
          $segment;
      }

      @segments = ( undef, [@segments] );
      $segments[0] = shift( @{ $segments[1] } );
      if ( scalar( @{ $segments[1] } ) > 0 ) {
        $segments[2] = pop( @{ $segments[1] } );
      }
      if ( scalar( @{ $segments[1] } ) == 0 ) {
        if ( exists( $segments[2] ) ) {
          $segments[1] = $segments[2];
        }
        pop(@segments);
      }

      push( @all_segments, @segments );
    }

    # The 'segments' list is a list of ProjectionSegment objects and of
    # arrays of ProjectionSegment objects.  For segments in arrays, no
    # constraint on seq_region_start or seq_region_end is needed.
    $this->__attrib( 'segments', \@all_segments );

    # This is a simple map between seq_region_id and a
    # ProjectionSegment, used in the mapping done by the
    # _objs_from_sth() method.
    $this->__attrib( 'seqreg_map', \%seqreg_map );

    $this->entries( [] );
    $this->__attrib( 'is_populated', 0 );

    $this->__attrib( 'slice', $slice );
  } ## end if ( defined($slice) )

  return $this->__attrib('slice');
} ## end sub slice

=head2 entries

  Arg [1]       : List reference
                  The new list of feature collection entries,
                  typically produced within the populate() method.

  Example       : my @entries = @{ $collection->entries() };

  Description   : Getter/setter for the list of collection entries.

  Return type   : List reference to list of list references

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

sub entries {
  my ( $this, $entries ) = @_;
  return $this->__attrib( 'entries', $entries );
}

=head2 is_populated

  Args          : None

  Example       : if ( !$collection->is_populated() ) {
                    $collection->populate();
                  }

  Description   : Returns a true value if the collection has been
                  populated, false of not.

  Return type   : Boolean

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

# Returns true if the collection has been populated, false of not.
sub is_populated {
  my ($this) = @_;
  return $this->__attrib('is_populated');
}

=head2 populate

  Arg [SORTED]  : Boolean (optional, default false)
                  If true, will sort the entries after popluating
                  the collection.

      [SORTSUB] : Subroutine reference (optional, default undef)
                  If defined, will be used to sort the entries after
                  popluating the collection.

      [LIGHT]   : Boolean (optional, default undef)
                  If true, will populate the collection with
                  light-weight entries.  If false, will populate the
                  collection with entries that are not light-weight.
                  If unset, will use the 'lightweight' boolean set
                  when calling new().

  Example       : $collection->populate( -sorted => 1, -light => 0 );

  Description   : Populates the collection with a compact
                  representation of the features overlapping the
                  current slice, and optionally sorts the results.
<commit_msg>Correct some of the examples in the SYNOPSIS, and make some POD code lines shorter.<commit_after># $Id$

# Ensembl module for Bio::EnsEMBL::Collection
#
# You may distribute this module under the same terms as perl itself.

=head1 NAME

Bio::EnsEMBL::Collection - Abstract base class for feature collection
classes.

=head1 SYNOPSIS

  use Bio::EnsEMBL::Collection::RepeatFeature;

  # Pick a slice.
  my $slice =
    $slice_adaptor->fetch_by_region( 'Chromosome', '2', 1, 1e9 );

  # Create a feature collection on the slice.
  my $collection =
    Bio::EnsEMBL::Collection::RepeatFeature->new( -slice => $slice,
                                                  -light => 0 );

  # Populate the feature collection from the slice.
  $collection->populate();

  # Populate the feature collection from the slice.  Sort the
  # entries on feature start position.
  $collection->populate( -sorted => 1 );

  # Populate the feature collection from the slice.  Sort the
  # entries on feature length and don't bother about feature
  # type-specific data.
  $collection->populate(
    -light   => 1,
    -sorted  => 1,
    -sortsub => sub {
      $a->[3] - $a->[2] <=> $b->[3] - $b->[2];
    } );

  # Retrieve the entries from the collection as an array of arrays.
  my @entries = @{ $collection->entries() };

  # Retrieve a binned representation of the entries using 100 bins.
  my @binned_entries = @{
    $collection->get_bins( -nbins  => 100,
                           -method => 'entries'
    ) };

  # Retrieve a binned representation of the entries using 100 bins,
  # where each entry is represented by its index in the feature
  # collection array (@entries above).
  my @binned_entry_indicies = @{
    $collection->get_bins( -nbins  => 100,
                           -method => 'indices'
    ) };

  # Retrieve only the bin counts.
  my @bin_counts = @{ $collection->get_bins( -nbins => 100 ) };

=head1 DESCRIPTION

This is the abstract base class for feature collections.

A feature collection provides a compact representation of features of a
particular type on a slice.  Each entry in a collection is a short array
of data representing a feature.  This data is divided into two halfs:

=over 4

=item 1.

Basic feature representation.

=item 2.

Extended feature representation.

=back

=head2 Basic feature representation

The basic feature representation is common to all entries in any type
of feature collection and consists of a minimal set of values.  Each
collection entry is an array that contains at least the following data
(in this order)

=over 4

=item 1.

Ensembl internal database ID.

=item 2.

Ensembl internal sequence region ID.

=item 3.

Feature start position.

=item 4.

Feature end position.

=item 5.

Feature strand.

=back

The module defines a number of constants that may be used
as symbolic constants in place of the index numbers 0 to
4: ENTRY_DBID, ENTRY_SEQREGIONID, ENTRY_SEQREGIONSTART,
ENTRY_SEQREGIONEND, ENTRY_SEQREGIONSTRAND.  For an entry $entry,
$entry->[Bio::EnsEMBL::Collection::ENTRY_SEQREGIONEND] will thus be the
end position for the feature that the entry represents.

The position of the feature is in the same coordinate system as the
slice associated with the collection object.

=head2 Extended feature representation

A sub-class of this abstract base class will specify further data to be
added to the entries in order to account of the particular feature type.
An entry from a gene feature collection (Bio::EnsEMBL::Collection::Gene)
might, for example, contain the Ensembl Stable ID of the gene.

=head2 Light-weight collections/entries

A light-weight collection is a feature collection whose collection
entries are light-weight, whose entries does not contain the extended
feature representation.

A collection which is light-weight by default may be created by using
the argument '-light=>1' in the constructor, new(), but one may also use
the same argument with populate().

=head2 Binning methods

This module allows for various ways of binning the result of the
populate() method by using the get_bins() method.

Each binning method bins the collection entries and gives an array
with the specified length (number of bins).  An entry, which basically
consists of a start and a end position, is allocated to one or several
bins depending on its span and the size of the individual bins.

 Features:     |----|     |----------------|  |--|         |------|
               |-------------|                |-----|             |--|
               |-------------------|                 |----|       |--|

 Finer bins:   3 3 3 2 2 2 3 3 2 2 2 1 1 1 1 0 2 2 1 1 1 1 1 1 1 1 2 2
 Coarser bins: 3  3  2  2  3  2  2  1  1  1  0  2  1  1  1  1  1  3  2

The example above shows the arrays that might be returned from
get_bins() when using the 'count' binning method with 28 and 19 bins
respectively.

Here follows a brief description of each implemented binning method.

=over 4

=item  'count' and 'density'

Returns an array of bins, each bin containing the number of entries
allocated to (i.e. overlapping) that bin.  The 'density' binning method
is equivalent to 'count' and this is also the default binning method.

=item 'indices' and 'index'

Returns an array of bins, each bin containing an array of indices into
the collection entry array (available from the entries() method) for the
entries allocated to that bin.  The 'index' binning method is equivalent
to 'indicies'.

=item 'entries' and 'entry'

Returns an array of bins, each bin containing an array of entries
(references into the array of entries retrieved from the entries()
method) allocated to that bin.  The 'entry' binning method is equivalent
to 'entries'.

=back

=head1 CONTACT

This modules is part of the Ensembl project.  See
http://www.ensembl.org/ for further information.

Questions may be posted to the ensembl-dev mailing list:
<EMAIL>

=cut

package Bio::EnsEMBL::Collection;

use strict;
use warnings;

use Bio::EnsEMBL::DBSQL::BaseAdaptor;

use Bio::EnsEMBL::Utils::Argument qw( rearrange );
use Bio::EnsEMBL::Utils::Exception qw( throw );

use base qw( Bio::EnsEMBL::DBSQL::BaseAdaptor );

# Symbolic constants that acts as indices into the individual feature
# collection entries.  These must be in sync with the columns returned
# by the _columns() method.
use constant { ENTRY_DBID            => 0,
               ENTRY_SEQREGIONID     => 1,
               ENTRY_SEQREGIONSTART  => 2,
               ENTRY_SEQREGIONEND    => 3,
               ENTRY_SEQREGIONSTRAND => 4 };

=head1 METHODS (constructor)

=head2 new

  Arg [SLICE]   : Bio::EnsEMBL::Slice
                  The slice to be associated with this feature
                  collection.

  Arg [LIGHT]   : Boolean (optional, default false)
                  If true, the collection will be 'light-weight',
                  i.e. no type-specific data will be stored in its
                  entries when populate() is called.

  Example       : my $collection =
                    Bio::EnsEMBL::Collection::<feature_type>->new(
                                                   -slice => $slice,
                                                   -light => 1 );

  Description   : When called for a sub-class, creates a feature
                  collection object and associates a slice with it.

  Return type   : Bio::EnsEMBL::Collection::<feature_type>

  Exceptions    : Throws if no slice is specified.

  Caller        : General (through a sub-class).

  Status        : At Risk (under development)

=cut

sub new {
  my $proto = shift;
  my ( $slice, $light ) = rearrange( [ 'SLICE', 'LIGHT' ], @_ );

  if ( !defined($slice) ) {
    throw(   'Unable to create a feature collection '
           . 'without a slice object, I am.' );
  }

  my $this = $proto->SUPER::new( $slice->adaptor()->db() );

  my $sql = qq(
    SELECT  cs.name, mc.max_length, m.meta_value
    FROM    coord_system cs,
            meta_coord mc
    LEFT JOIN meta m ON m.meta_key = ?
    WHERE   mc.table_name = ?
    AND     mc.coord_system_id = cs.coord_system_id
  );

  my $sth           = $this->prepare($sql);
  my $feature_table = $this->_feature_table()->[0];
  $sth->execute( $feature_table . 'build.level', $feature_table );

  my ( $cs_name, $max_length, $build_level );
  $sth->bind_columns( \( $cs_name, $max_length, $build_level ) );

  my %coordinate_systems;

  while ( $sth->fetch() ) {
    $coordinate_systems{$cs_name} = {
            'name'               => $cs_name,
            'max_feature_length' => $max_length,
            'build_level'        => $build_level    # Not currently used
    };
  }
  $sth->finish();

  $this->__attrib( 'coordinate_systems', \%coordinate_systems );

  $this->lightweight($light);
  $this->slice($slice);

  return $this;
} ## end sub new

#-----------------------------------------------------------------------

=head1 METHODS (public)

=head2 slice

  Arg [1]       : Bio::EnsEMBL::Slice (optional)
                  The new slice to be associated with this
                  collection.

  Example       : my $slice = $collection->slice();

  Description   : Getter/setter for the main slice associated with
                  this collection.  Associating a new slice with
                  the collection will empty all entries from the
                  collection.

  Return type   : Bio::EnsEMBL::Slice

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

sub slice {
  my ( $this, $slice ) = @_;

  if ( defined($slice) ) {
    my %seqreg_map;
    my @all_segments;

    foreach
      my $cs_name ( keys( %{ $this->__attrib('coordinate_systems') } ) )
    {
      my @segments = @{ $slice->project($cs_name) };

      foreach my $segment (@segments) {
        $seqreg_map{ $segment->to_Slice()->get_seq_region_id() } =
          $segment;
      }

      @segments = ( undef, [@segments] );
      $segments[0] = shift( @{ $segments[1] } );
      if ( scalar( @{ $segments[1] } ) > 0 ) {
        $segments[2] = pop( @{ $segments[1] } );
      }
      if ( scalar( @{ $segments[1] } ) == 0 ) {
        if ( exists( $segments[2] ) ) {
          $segments[1] = $segments[2];
        }
        pop(@segments);
      }

      push( @all_segments, @segments );
    }

    # The 'segments' list is a list of ProjectionSegment objects and of
    # arrays of ProjectionSegment objects.  For segments in arrays, no
    # constraint on seq_region_start or seq_region_end is needed.
    $this->__attrib( 'segments', \@all_segments );

    # This is a simple map between seq_region_id and a
    # ProjectionSegment, used in the mapping done by the
    # _objs_from_sth() method.
    $this->__attrib( 'seqreg_map', \%seqreg_map );

    $this->entries( [] );
    $this->__attrib( 'is_populated', 0 );

    $this->__attrib( 'slice', $slice );
  } ## end if ( defined($slice) )

  return $this->__attrib('slice');
} ## end sub slice

=head2 entries

  Arg [1]       : List reference
                  The new list of feature collection entries,
                  typically produced within the populate() method.

  Example       : my @entries = @{ $collection->entries() };

  Description   : Getter/setter for the list of collection entries.

  Return type   : List reference to list of list references

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

sub entries {
  my ( $this, $entries ) = @_;
  return $this->__attrib( 'entries', $entries );
}

=head2 is_populated

  Args          : None

  Example       : if ( !$collection->is_populated() ) {
                    $collection->populate();
                  }

  Description   : Returns a true value if the collection has been
                  populated, false of not.

  Return type   : Boolean

  Exceptions    : None

  Caller        : General

  Status        : At Risk (under development)

=cut

# Returns true if the collection has been populated, false of not.
sub is_populated {
  my ($this) = @_;
  return $this->__attrib('is_populated');
}

=head2 populate

  Arg [SORTED]  : Boolean (optional, default false)
                  If true, will sort the entries after popluating
                  the collection.

      [SORTSUB] : Subroutine reference (optional, default undef)
                  If defined, will be used to sort the entries after
                  popluating the collection.

      [LIGHT]   : Boolean (optional, default undef)
                  If true, will populate the collection with
                  light-weight entries.  If false, will populate the
                  collection with entries that are not light-weight.
                  If unset, will use the 'lightweight' boolean set
                  when calling new().

  Example       : $collection->populate( -sorted => 1,
                                         -light  => 0 );

  Description   : Populates the collection with a compact
                  representation of the features overlapping the
                  current slice, and optionally sorts the results.

{'fraction_non_alphanumeric': 0.10264031498494557, 'fraction_numerical': 0.009959082837952597, 'mean_word_length': 2.443705968363685, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 146, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5543503', 'n_tokens_mistral': 7798, 'n_tokens_neox': 7265, 'n_words': 2846}