<commit_before>=begin pod

=TITLE Testing

=SUBTITLE Writing and running tests in Perl 6

Testing code is an integral part of software development.  For instance, how
can one be sure the code works if there isn't some way to verify it behaves
as desired?  This is the role of tests: to provide automated, repeatable
verifications of code behaviour.

In Perl 6, the C<Test> module provides a testing framework similar to the
traditional Perl 5 L<Test::More|http://perldoc.perl.org/Test/More.html>
module.  Thus, anyone familiar with C<Test::More> (and friends) should be
comfortable with Perl 6's C<Test> module.  The C<Test> module is the testing
module used by the official spectest suite.

The testing functions emit output conforming to the L<Test Anything
Protocol|http://testanything.org>.

=head1 Writing tests

As with any Perl project, the tests live under the C<t> directory in the
project's base directory.

A typical test file looks something like this:

    use v6;
    use Test;
    use lib 'lib';

    plan $num-tests;

    # .... tests

We ensure that we're using Perl 6, via the C<use v6> pragma, then we load
the C<Test> module and specify where our libraries are.  We then specify how
many tests we I<plan> to run (such that the testing framework can tell us if
more or fewer tests were run than we expected) and when finished with the
tests, we tell the framework that we are I<done> testing.

=head1 Running tests

One can run tests individually by specifying the test filename on the
command line:

    $ perl6 t/test-filename.t

Or via the L<prove|http://perldoc.perl.org/prove.html> command from Perl 5,
where one specifies C<perl6> as the executable to run the tests:

    $ prove --exec perl6 -r t

=head1 Test plans

=item plan

Specify the total number of tests, usually specified at the beginning of a
test file.

    plan 15;   # expect to have run 15 tests

In C<subtest>s, C<plan> is used to specify the total number of tests within
the subtest.

If a C<plan> is used, it is not necessary to denote the end of testing with
C<done>.

=item done

Specify that testing has finished.  Use this function when you do not as yet
have a C<plan> for the number of tests to run.  A C<plan> is thus not
required when using C<done>.

=head1 Test functions

The C<Test> module exports the following functions to aid in structuring and
organising tests:

=head2 Unconditional passing/failing/notification

=item pass($description?), flunk($description?)

The C<pass> function marks a test as passed.  C<flunk> marks a test as
B<not> passed.  Both functions accept an optional test C<$description>.

    pass "Actually, this test has passed";

    flunk "But this one hasn't passed";

As the Perl 5 documentation for C<Test::More> notes, such constructs are
used when one has a complicated test condition.  As such, they should be
used very, very sparingly.

=item diag($message);

Display diagnostic information in a TAP-compatible manner on the standard
error stream.  It is usually used when a particular test has failed to
provide information that the test itself did not provide.  Or it can be used
to provide visual markers on how the testing of a test-file is progressing
(which can be important when doing stress testing).

    diag "Yay!  The tests got to here!";

=head2 Skipping tests

Sometimes tests just aren't ready to be run, for instance a feature might
not yet be implemented, in which case tests can be marked as C<todo>.  It
could be the case that a given feature only works on a particular platform.
In which case one can C<skip> tests.

=item todo($reason, $count = 1)

Mark C<$count> tests as TODO, giving a C<$reason> as to why.  By default
only one test will be marked TODO.

    sub my-custom-pi { 3 };
    todo 'not yet precise enough';         # mark the test as TODO
    # run test, but don't report failure in test harness
    is my-custom-pi(), pi, 'my-custom-pi';

=item skip($reason, $count = 1)

Skip C<$count> tests, giving a C<$reason> as to why.  By default only one
test will be skipped.  Use such functionality when a test (or tests) would
die if run.

    if $*KERNEL ~~ 'linux' {
	is(num_forward_slashes("/a/b"), 2);
	is(num_forward_slashes(normalize("/a//b")), 2);
    }
    else {
	skip "Can't use forward slashes on Windows", 2;
    }

=item skip_rest($reason?)

Skip the remaining tests.  If the remainder of the tests in the test file
would all fail due to some condition, use this function to skip them,
providing an optional C<$reason> as to why.

    unless $location ~~ "Wimbledon Common" {
	skip_rest "We can't womble, the remaining tests will fail";
	exit;
    }

    # tests requiring functional wombling
    ok(womble());
    # ...

Note that C<skip_rest> requires a C<plan> to be set, otherwise the
C<skip_rest> call will throw an error. If no plan was set, you can use
C<done> to indicate the end of the test run (though of course the test
summary will not count skipped tests).

=head2 Evaluation of test conditions in boolean context

=item ok($condition, $description?), nok($condition, $description?)

The C<ok> function marks a test as passed if the given C<$condition>
evaluates to C<True> in a boolean context. The C<nok> function marks
a test as passed if the given condition evaluates to C<False>.

    plan 2;

    my $pi = 3;
    ok($pi == 3, 'The variable $pi equals 3');
    nok(pi == 3, 'The constant pi does not equal 3');

=head2 Comparison of expected and actual output

=item is($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are equal
when coerced to strings (i.e., they compare as equal when using the
L<eq operator|/routine/eq>).  The function
accepts an optional C<$description> of the test.

    my $pi = 3;
    is($pi, 3, 'The variable $pi equals 3');

=item isnt($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are
B<not> equal (i.e., they compare as B<not> equal when using the C<'eq'>
operator).  The function accepts an optional C<$description> of the test.

    isnt(pi, 3, 'The constant π is not equal to 3');

=item cmp_ok($obtained, $comparison, $expected, $description?)

The C<cmp_ok()> function compares the C<$obtained> and C<$expected> values
with the given C<$comparison> operator and passes the test if the comparison
yields a C<True> value.  For ease of use, operators may be passed as
strings, such as C<'=='> or C<'~~'>.

    cmp_ok('my spelling is apperling', '~~', /perl/, "bad speller");

=head2 Structural comparison with C<< infix:<eqv> >>

=item is_deeply($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are
C<eqv>.  This is the best way to check for equality of (deep) data
structures.  The function accepts an optional C<$description> of the test.

=begin code
use v6;
use Test;
plan 1;

sub count-chars(Str() $x) {
    my %chars;
    %chars{$_}++ for $x.comb;
    return %chars;
}
my %expected =
    f   => 1,
    l   => 1,
    e   => 3,
    c   => 1,
    ;
is_deeply count-chars('fleece'), %expected, 'count-chars works on "fleece"';
=end code

Note that string comparison here would have been fragile, because hashes are
not sorted, and converting a L<Hash|/type/Hash> to a string loses information.

=head2 Numeric comparison within a tolerance of 1e-5

=item is_approx($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> numerical values
are within C<1e-5> of each other.  The function accepts an optional
C<$description> of the test.

    my $eulers_constant_approx = 2.71828;
    is_approx($eulers_constant_approx, e, "approximate Euler's constant");

=head2 Class membership testing

=item isa_ok($object, $type, $description?)

Marks a test as passed if the given C<$object> is, or inherits from, the
given C<$type>.  For convenience, types may also be specified as a string.
The function accepts an optional C<$description> of the test.

    class Womble {}
    class GreatUncleBulgaria is Womble {}

    my $womble = GreatUncleBulgaria.new;
    isa_ok($womble, Womble, "Great Uncle Bulgaria is a womble");
    isa_ok($womble, 'Womble');     # equivalent

=head2 Grouping tests

The result of a group of subtests is only C<ok> if all subtests are C<ok>.

=item subtest(&subtests, $description?)

The C<subtest> function executes the given block, consisting of usually more
than one test, possibly including a C<plan> or C<done>.  It will pass the
test only if B<all> tests in the block, pass.  The function accepts an
optional C<$description> of the test.

    class Womble {}

    class GreatUncleBulgaria is Womble {
	has $.location = "Wimbledon Common";
	has $.spectacles = True;
    }

    subtest {
	my $womble = GreatUncleBulgaria.new;
	isa_ok($womble, Womble, "Great Uncle Bulgaria is a womble");
	is($womble->location, "Wimbledon Common", "Correct location");
	ok($womble->spectacles, "Wears spectacles");
    }, "Check Great Uncle Bulgaria";

=head2 Exception testing

=item dies_ok($code, $description?)

Marks the test as passed if the given C<$code> throws an exception.

The function accepts an optional C<$description> of the test.

    sub saruman(Bool :$ents-destroy-isengard) {
	die "Killed by Wormtongue" if $ents-destroy-isengard;
    }

    dies_ok { saruman(ents-destroy-isengard => True) }, "Saruman dies";

=item lives_ok($code, $description?)

Marks the test as passed if the given C<$code> B<does not> throw an
exception.

The function accepts an optional C<$description> of the test.

    sub frodo(Bool :$destroys-ring) {
	die "Oops, that wasn't supposed to happen" unless $destroys-ring;
    }

    lives_ok { frodo(destroys-ring => True) }, "Frodo survivies";

=item eval_dies_ok($code, $description?)

Marks the test as passed if the given C<eval>ed C<$code> throws an
exception.

The function accepts an optional C<$description> of the test.

    eval_dies_ok q[my $joffrey = "nasty";
                   die "bye bye Ned" if $joffrey ~~ /nasty/],
	"Ned Stark dies";

=item eval_lives_ok

Marks the test as passed if the given C<eval>ed C<$code> throws an
exception.

The function accepts an optional C<$description> of the test.

    eval_lives_ok q[my $daenerys_burns = False;
		    die "Oops, Khaleesi now ashes" if $daenerys_burns],
	"Dany is blood of the dragon";

=item throws_like($code, $ex_type, $description?)

The C<throws_like> function checks whether the given C<$code> (specified as
either something C<Callable>, or as a something to be C<EVAL>led) throws a
specific exception (either specified as a C<Type> object, or as a string).
If an exception was thrown, it will also try to match the matcher hash, in
which the key is the name of the method to be called on the exception, and
the value is the value it should have to pass.

The function accepts an optional C<$description> of the test.

Please note that you can only use the C<EVAL> form if you are not referencing
any symbols in the surrounding scope.  If you are, you should encapsulate
your string with a block and an EVAL.  For instance:

    throws_like { EVAL q[ fac("foo") ] }, X::TypeCheck::Argument;

=end pod
<commit_msg>Document new features of is()/isnt()<commit_after>=begin pod

=TITLE Testing

=SUBTITLE Writing and running tests in Perl 6

Testing code is an integral part of software development.  For instance, how
can one be sure the code works if there isn't some way to verify it behaves
as desired?  This is the role of tests: to provide automated, repeatable
verifications of code behaviour.

In Perl 6, the C<Test> module provides a testing framework similar to the
traditional Perl 5 L<Test::More|http://perldoc.perl.org/Test/More.html>
module.  Thus, anyone familiar with C<Test::More> (and friends) should be
comfortable with Perl 6's C<Test> module.  The C<Test> module is the testing
module used by the official spectest suite.

The testing functions emit output conforming to the L<Test Anything
Protocol|http://testanything.org>.

=head1 Writing tests

As with any Perl project, the tests live under the C<t> directory in the
project's base directory.

A typical test file looks something like this:

    use v6;
    use Test;
    use lib 'lib';

    plan $num-tests;

    # .... tests

We ensure that we're using Perl 6, via the C<use v6> pragma, then we load
the C<Test> module and specify where our libraries are.  We then specify how
many tests we I<plan> to run (such that the testing framework can tell us if
more or fewer tests were run than we expected) and when finished with the
tests, we tell the framework that we are I<done> testing.

=head1 Running tests

One can run tests individually by specifying the test filename on the
command line:

    $ perl6 t/test-filename.t

Or via the L<prove|http://perldoc.perl.org/prove.html> command from Perl 5,
where one specifies C<perl6> as the executable to run the tests:

    $ prove --exec perl6 -r t

=head1 Test plans

=item plan

Specify the total number of tests, usually specified at the beginning of a
test file.

    plan 15;   # expect to have run 15 tests

In C<subtest>s, C<plan> is used to specify the total number of tests within
the subtest.

If a C<plan> is used, it is not necessary to denote the end of testing with
C<done>.

=item done

Specify that testing has finished.  Use this function when you do not as yet
have a C<plan> for the number of tests to run.  A C<plan> is thus not
required when using C<done>.

=head1 Test functions

The C<Test> module exports the following functions to aid in structuring and
organising tests:

=head2 Unconditional passing/failing/notification

=item pass($description?), flunk($description?)

The C<pass> function marks a test as passed.  C<flunk> marks a test as
B<not> passed.  Both functions accept an optional test C<$description>.

    pass "Actually, this test has passed";

    flunk "But this one hasn't passed";

As the Perl 5 documentation for C<Test::More> notes, such constructs are
used when one has a complicated test condition.  As such, they should be
used very, very sparingly.

=item diag($message);

Display diagnostic information in a TAP-compatible manner on the standard
error stream.  It is usually used when a particular test has failed to
provide information that the test itself did not provide.  Or it can be used
to provide visual markers on how the testing of a test-file is progressing
(which can be important when doing stress testing).

    diag "Yay!  The tests got to here!";

=head2 Skipping tests

Sometimes tests just aren't ready to be run, for instance a feature might
not yet be implemented, in which case tests can be marked as C<todo>.  It
could be the case that a given feature only works on a particular platform.
In which case one can C<skip> tests.

=item todo($reason, $count = 1)

Mark C<$count> tests as TODO, giving a C<$reason> as to why.  By default
only one test will be marked TODO.

    sub my-custom-pi { 3 };
    todo 'not yet precise enough';         # mark the test as TODO
    # run test, but don't report failure in test harness
    is my-custom-pi(), pi, 'my-custom-pi';

=item skip($reason, $count = 1)

Skip C<$count> tests, giving a C<$reason> as to why.  By default only one
test will be skipped.  Use such functionality when a test (or tests) would
die if run.

    if $*KERNEL ~~ 'linux' {
	is(num_forward_slashes("/a/b"), 2);
	is(num_forward_slashes(normalize("/a//b")), 2);
    }
    else {
	skip "Can't use forward slashes on Windows", 2;
    }

=item skip_rest($reason?)

Skip the remaining tests.  If the remainder of the tests in the test file
would all fail due to some condition, use this function to skip them,
providing an optional C<$reason> as to why.

    unless $location ~~ "Wimbledon Common" {
	skip_rest "We can't womble, the remaining tests will fail";
	exit;
    }

    # tests requiring functional wombling
    ok(womble());
    # ...

Note that C<skip_rest> requires a C<plan> to be set, otherwise the
C<skip_rest> call will throw an error. If no plan was set, you can use
C<done> to indicate the end of the test run (though of course the test
summary will not count skipped tests).

=head2 Evaluation of test conditions in boolean context

=item ok($condition, $description?), nok($condition, $description?)

The C<ok> function marks a test as passed if the given C<$condition>
evaluates to C<True> in a boolean context. The C<nok> function marks
a test as passed if the given condition evaluates to C<False>.

    plan 2;

    my $pi = 3;
    ok($pi == 3, 'The variable $pi equals 3');
    nok(pi == 3, 'The constant pi does not equal 3');

=head2 Comparison of expected and actual output

=item is($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are equal.
If C<$expected> is a type object, then the comparison operator is C<===>.
If C<$expected> is defined, then C<$obtained> and C<$expected> are coerced
to strings and compared using the L<eq operator|/routine/eq>).  The function
accepts an optional C<$description> of the test.

    my $pi = 3;
    is($pi, 3, 'The variable $pi equals 3');

    my Int $a;
    is $a, Int, 'The variable $a is an unassigned Int';

=item isnt($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are
B<not> equal using the same rules as C<is()>.  The function accepts an
optional C<$description> of the test.

    isnt(pi, 3, 'The constant π is not equal to 3');

    my Int $a = 23;
    $a = Nil;
    isnt $a, Nil, 'Nil should not survive being put in a container';

=item cmp_ok($obtained, $comparison, $expected, $description?)

The C<cmp_ok()> function compares the C<$obtained> and C<$expected> values
with the given C<$comparison> operator and passes the test if the comparison
yields a C<True> value.  For ease of use, operators may be passed as
strings, such as C<'=='> or C<'~~'>.

    cmp_ok('my spelling is apperling', '~~', /perl/, "bad speller");

=head2 Structural comparison with C<< infix:<eqv> >>

=item is_deeply($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> values are
C<eqv>.  This is the best way to check for equality of (deep) data
structures.  The function accepts an optional C<$description> of the test.

=begin code
use v6;
use Test;
plan 1;

sub count-chars(Str() $x) {
    my %chars;
    %chars{$_}++ for $x.comb;
    return %chars;
}
my %expected =
    f   => 1,
    l   => 1,
    e   => 3,
    c   => 1,
    ;
is_deeply count-chars('fleece'), %expected, 'count-chars works on "fleece"';
=end code

Note that string comparison here would have been fragile, because hashes are
not sorted, and converting a L<Hash|/type/Hash> to a string loses information.

=head2 Numeric comparison within a tolerance of 1e-5

=item is_approx($obtained, $expected, $description?)

Marks a test as passed if the C<$obtained> and C<$expected> numerical values
are within C<1e-5> of each other.  The function accepts an optional
C<$description> of the test.

    my $eulers_constant_approx = 2.71828;
    is_approx($eulers_constant_approx, e, "approximate Euler's constant");

=head2 Class membership testing

=item isa_ok($object, $type, $description?)

Marks a test as passed if the given C<$object> is, or inherits from, the
given C<$type>.  For convenience, types may also be specified as a string.
The function accepts an optional C<$description> of the test.

    class Womble {}
    class GreatUncleBulgaria is Womble {}

    my $womble = GreatUncleBulgaria.new;
    isa_ok($womble, Womble, "Great Uncle Bulgaria is a womble");
    isa_ok($womble, 'Womble');     # equivalent

=head2 Grouping tests

The result of a group of subtests is only C<ok> if all subtests are C<ok>.

=item subtest(&subtests, $description?)

The C<subtest> function executes the given block, consisting of usually more
than one test, possibly including a C<plan> or C<done>.  It will pass the
test only if B<all> tests in the block, pass.  The function accepts an
optional C<$description> of the test.

    class Womble {}

    class GreatUncleBulgaria is Womble {
	has $.location = "Wimbledon Common";
	has $.spectacles = True;
    }

    subtest {
	my $womble = GreatUncleBulgaria.new;
	isa_ok($womble, Womble, "Great Uncle Bulgaria is a womble");
	is($womble->location, "Wimbledon Common", "Correct location");
	ok($womble->spectacles, "Wears spectacles");
    }, "Check Great Uncle Bulgaria";

=head2 Exception testing

=item dies_ok($code, $description?)

Marks the test as passed if the given C<$code> throws an exception.

The function accepts an optional C<$description> of the test.

    sub saruman(Bool :$ents-destroy-isengard) {
	die "Killed by Wormtongue" if $ents-destroy-isengard;
    }

    dies_ok { saruman(ents-destroy-isengard => True) }, "Saruman dies";

=item lives_ok($code, $description?)

Marks the test as passed if the given C<$code> B<does not> throw an
exception.

The function accepts an optional C<$description> of the test.

    sub frodo(Bool :$destroys-ring) {
	die "Oops, that wasn't supposed to happen" unless $destroys-ring;
    }

    lives_ok { frodo(destroys-ring => True) }, "Frodo survivies";

=item eval_dies_ok($code, $description?)

Marks the test as passed if the given C<eval>ed C<$code> throws an
exception.

The function accepts an optional C<$description> of the test.

    eval_dies_ok q[my $joffrey = "nasty";
                   die "bye bye Ned" if $joffrey ~~ /nasty/],
	"<NAME> dies";

=item eval_lives_ok

Marks the test as passed if the given C<eval>ed C<$code> throws an
exception.

The function accepts an optional C<$description> of the test.

    eval_lives_ok q[my $daenerys_burns = False;
		    die "Oops, Khaleesi now ashes" if $daenerys_burns],
	"Dany is blood of the dragon";

=item throws_like($code, $ex_type, $description?)

The C<throws_like> function checks whether the given C<$code> (specified as
either something C<Callable>, or as a something to be C<EVAL>led) throws a
specific exception (either specified as a C<Type> object, or as a string).
If an exception was thrown, it will also try to match the matcher hash, in
which the key is the name of the method to be called on the exception, and
the value is the value it should have to pass.

The function accepts an optional C<$description> of the test.

Please note that you can only use the C<EVAL> form if you are not referencing
any symbols in the surrounding scope.  If you are, you should encapsulate
your string with a block and an EVAL.  For instance:

    throws_like { EVAL q[ fac("foo") ] }, X::TypeCheck::Argument;

=end pod

{'fraction_non_alphanumeric': 0.0913467909359954, 'fraction_numerical': 0.005565616855868192, 'mean_word_length': 3.9486338797814207, 'pattern_counts': {'":': 0, '<': 206, '<?xml version=': 0, '>': 222, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4116934', 'n_tokens_mistral': 7212, 'n_tokens_neox': 6883, 'n_words': 3233}