<commit_before># Copyright 2012 <NAME>
# This file is part of Marpa::R2.  Marpa::R2 is free software: you can
# redistribute it and/or modify it under the terms of the GNU Lesser
# General Public License as published by the Free Software Foundation,
# either version 3 of the License, or (at your option) any later version.
#
# Marpa::R2 is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser
# General Public License along with Marpa::R2.  If not, see
# http://www.gnu.org/licenses/.

=head1 NAME

Marpa::R2::Changes - Differences between Marpa::R2 and Marpa::XS

=head1 ABOUT THIS DOCUMENT

This document describes the incompatible
differences between Marpa::XS
and Marpa::R2.
(Differences that do not give rise to incompatibility
are outside of its scope.)
It is intended for readers already familiar with Marpa::XS,
who are writing new applications for Marpa::R2,
and for readers migrating Marpa::XS applications
and tools to Marpa::R2.

=head1 CHANGES

=head2 Different rules with the same rank now appear in arbitrary order

In ranking parse trees,
if two rule instances are for different
rules
but have the same rule rank,
they will now appear in arbitrary order.
This is probably the behavior that programmers
have always expected.

In Marpa::XS, when the C<null_ranking> named
argument of rules was in use for one of the rules,
specific guarantees were made for the order
in some of the cases.
The intent was to be orthogonal with the guarantees
made for the ranking of null variants within the
same rule.
These additional guarantees
proved useless in practice,<commit_msg>Documentation<commit_after># Copyright 2012 <NAME>
# This file is part of Marpa::R2.  Marpa::R2 is free software: you can
# redistribute it and/or modify it under the terms of the GNU Lesser
# General Public License as published by the Free Software Foundation,
# either version 3 of the License, or (at your option) any later version.
#
# Marpa::R2 is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser
# General Public License along with Marpa::R2.  If not, see
# http://www.gnu.org/licenses/.

=head1 NAME

Marpa::R2::Changes - Differences between Marpa::R2 and Marpa::XS

=head1 ABOUT THIS DOCUMENT

This document describes the incompatible
differences between Marpa::XS
and Marpa::R2.
(Differences that do not give rise to incompatibility
are outside of its scope.)
It is intended for readers already familiar with Marpa::XS,
who are writing new applications for Marpa::R2,
and for readers migrating Marpa::XS applications
and tools to Marpa::R2.

=head1 CHANGES

=head2 Rule LHS's are no longer a source of action names

In Marpa::XS, if there was no explicit action name for a rule, Marpa
would try to find a closure that had the same name as the rule's
LHS.  The use of rule LHS's as
action names had a potential for
unpleasant surprises.
A surprise could occur if the rule's LHS coincided with
a function name without the prorgrammer realizing or intending it.
This kind of 'action at a distance' bug can be very hard to detect
and trace.

It was originally thought that implicitly using the LHS as the name
of an action would be convenient enough to outweigh the dangers.
But in fact,
this feature wound up being little used.  And accidental
resolution via a rule LHS was a danger for all users, whether they
used the feature or not.  For these reasons, as well as
potential optimization and efficiency considerations, Marpa::R2 no longer
does implicit action resolution using a rule LHS.

=head2 Different rules with the same rank now appear in arbitrary order

In ranking parse trees,
if two rule instances are for different
rules
but have the same rule rank,
they will now appear in arbitrary order.
This is probably the behavior that programmers
have always expected.

In Marpa::XS, when the C<null_ranking> named
argument of rules was in use for one of the rules,
specific guarantees were made for the order
in some of the cases.
The intent was to be orthogonal with the guarantees
made for the ranking of null variants within the
same rule.
These additional guarantees
proved useless in practice,
{'fraction_non_alphanumeric': 0.04670028502521377, 'fraction_numerical': 0.008331506248629687, 'mean_word_length': 4.57020757020757, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2179844', 'n_tokens_mistral': 1237, 'n_tokens_neox': 1162, 'n_words': 713}