<commit_before>This document assumes you found a bug with fuzzer-combined.js and want to make a reduced testcase.  A reduced testcase easier for a Gecko developer to work with to understand the bug, and avoids giving away hints about how the fuzzers work to people who look at the testcase.


=== Get the interesting bits into a single file ===

The fast way:

1. grep FRC w1234-out | pbcopy
2. Paste into a copy of a-template.html (call it b0.html), replacing the "var fuzzSettings = null;" and "var fuzzCommands = null;" lines.
3. Add a "]" on a separate line if it's missing from the end.  (It will be missing from the output if Firefox crashed, for example.)
4. Load b0.html in Firefox.

If you can't reproduce the bug:

5. Grab a copy of the original page.
6. If needed, add <base fuzzskip="skip" href="[pageurl]"> to the <head>.
7. Copy the three <script> elements from b0.html to the original page.
     For HTML or XHTML, paste it in <head> (below meta charset, if present -- bug 322181) (above base href, if present).
     For things like SVG that don't have <head>, paste it so that it's the first child of the root, with no whitespace before it.

Using the original page ensures that the Document type (e.g. XMLDocument, HTMLDocument) is the same and the layout mode (e.g. quirks, almost-standards, standards) is the same.  It also makes the next step possible.  If you still can't reproduce the bug:

8. Remove the FRC2 lines.

The fuzz commands will now act on the DOM as it results from markup, rather than the DOM reconstruction script.  This should be very similar what happened the first time.  If you *still* can't reproduce, then as a last resort, you can try:

9. Load the URL you get from "grep running w1234-out", including the #squarefree-af!... part.

This ensures that makeCommand (which isn't *supposed* to have side effects) gets called, but it makes it possible for new randomness to sneak in.



=== Reduce the fuzzCommands array ===

1. Reduce the fuzzCommands array.

You'll usually want to do this part using an automated reduction program such as Lithium -- it's not only automatic (letting you focus on other things for a while) but also usually faster than doing the same thing by hand.  To prepare the testcase for Lithium, make sure there are comment lines containing "DDBEGIN" and "DDEND" around the meat of the fuzzCommands array (DDEND might be missing due to a crash), and add a to goQuitApplication after the DDEND line.  It should look like this:

var fuzzCommands = [
  ...
  // DDBEGIN
  ... recorded commands ...
  // DDEND
  { origCount: 10000, rest: true },
  { origCount: 10001, fun: goQuitApplication },
];

Now the testcase is ready for you to use Lithium: it will always cause Firefox to exit without human intervention (either by crashing or by hitting the goQuitApplication line), and Lithium just needs to find a minimal subsequence of the recorded commands that triggers the bug.

For a crash, use one of the following:
./lithium L foo.html ~/.../firefox-bin crashes 120
./lithium L foo.html ~/.../firefox outputs1 "running" "Quitting"

For an assertion, use:
./lithium L foo.html ~/.../firefox outputs "index out of range"


=== Dealing with bugs that are not 100% reproducible ===

- Use fuzzRetry to make it try several times.  This will make it a bit slower, and there's a risk that it will become harder and harder to reproduce as it gets smaller.
- Use the "fuzzCount = 0" trick.
- Force GC
- Ensure that the page doesn't depend on images or stylesheets that must be loaded from the Web.
- Increase timeouts (e.g. to let remote images load).
- Decrease timeouts -- use offsetHeight (for HTML) or boxObject.height (for XUL) instead.
- Move XBL to markup, since onload knows when it's done, and otherwise it's tricky to tell when it's done.


=== Dealing with messy testcases ===

Sometimes, Lithium gives you something that has 20 or more lines.  Common culprits are wrapping bugs (where widths have to add up to some large amount in order to wrap) and -moz-column bugs (where the total height might be split into two equal parts, and the break point matters).

When -moz-column is involved, use the "fuzzCount = 0" trick and make it use a different width or height each time.  This ensures that the "correct" column-break point is tested even as content is removed from either side.

When wrapping is involved, do the same with varying widths, or just use a small width (like 1px).


=== Reduce the rest and convert script to markup ===

Now that you have a short sequence of JavaScript commands, you'll have to reduce the rest of the script (and the markup, if you're not using FRC2) by hand.  You want a final reduced testcase that include very little or no script.  Often, it's only necessary for the last appendChild or removeChild call to be dynamic; the rest of the testcase can be markup.

Here's an example of a reduced testcase where exactly one modification was left dynamic:
view-source:https://bugzilla.mozilla.org/attachment.cgi?id=250556<commit_msg>Make serializeDOMAsScript include disconnected trees.  Add fuzzReset (like fuzzRetry but without reloading).<commit_after>This document assumes you found a bug with fuzzer-combined.js and want to make a reduced testcase.  A reduced testcase easier for a Gecko developer to work with to understand the bug, and avoids giving away hints about how the fuzzers work to people who look at the testcase.


=== Get the interesting bits into a single file ===

The fast way:

1. grep FRC w1234-out | pbcopy
2. Paste into a copy of a-template.html (call it b0.html), replacing the "var fuzzSettings = null;" and "var fuzzCommands = null;" lines.
3. Add a "]" on a separate line if it's missing from the end.  (It will be missing from the output if Firefox crashed, for example.)
4. Load b0.html in Firefox.

If you can't reproduce the bug:

5. Grab a copy of the original page.
6. If needed, add <base fuzzskip="skip" href="[pageurl]"> to the <head>.
7. Copy the three <script> elements from b0.html to the original page.
     For HTML or XHTML, paste it in <head> (below meta charset, if present -- bug 322181) (above base href, if present).
     For things like SVG that don't have <head>, paste it so that it's the first child of the root, with no whitespace before it.

Using the original page ensures that the Document type (e.g. XMLDocument, HTMLDocument) is the same and the layout mode (e.g. quirks, almost-standards, standards) is the same.  It also makes the next step possible.  If you still can't reproduce the bug:

8. Remove the FRC2 lines.

The fuzz commands will now act on the DOM as it results from markup, rather than the DOM reconstruction script.  This should be very similar what happened the first time.  If you *still* can't reproduce, then as a last resort, you can try:

9. Load the URL you get from "grep running w1234-out", including the #squarefree-af!... part.

This ensures that makeCommand (which isn't *supposed* to have side effects) gets called, but it makes it possible for new randomness to sneak in.



=== Reduce the fuzzCommands array ===

1. Reduce the fuzzCommands array.

You'll usually want to do this part using an automated reduction program such as Lithium -- it's not only automatic (letting you focus on other things for a while) but also usually faster than doing the same thing by hand.  To prepare the testcase for Lithium, make sure there are comment lines containing "DDBEGIN" and "DDEND" around the meat of the fuzzCommands array (DDEND might be missing due to a crash), and add a to goQuitApplication after the DDEND line.  It should look like this:

var fuzzCommands = [
  ...
  // DDBEGIN
  ... recorded commands ...
  // DDEND
  { origCount: 10000, rest: true },
  { origCount: 10001, fun: goQuitApplication },
];

Now the testcase is ready for you to use Lithium: it will always cause Firefox to exit without human intervention (either by crashing or by hitting the goQuitApplication line), and Lithium just needs to find a minimal subsequence of the recorded commands that triggers the bug.

For a crash, use one of the following:
./lithium L foo.html ~/.../firefox-bin crashes 120
./lithium L foo.html ~/.../firefox outputs1 "running" "Quitting"

For an assertion, use:
./lithium L foo.html ~/.../firefox outputs "index out of range"


=== Dealing with bugs that are not 100% reproducible ===

- Use fuzzReset to make it restart from the beginning of fuzzCommands several times.  This will make it a bit slower, and there's a risk that it will become harder and harder to reproduce as it gets smaller.
- Use fuzzRetry to make it reload the entire page several times.  This has the same risks as fuzzReset, and is considerably slower.
- Ensure that the page doesn't depend on images or stylesheets that must be loaded from the Web.
- Increase timeouts (e.g. to let remote images load).
- Decrease timeouts -- use offsetHeight (for HTML) or boxObject.height (for XUL) instead.
- Force GC
- Move XBL to markup, since onload knows when it's done, and otherwise it's tricky to tell when it's done.


=== Dealing with messy testcases ===

Sometimes, Lithium gives you something that has 20 or more lines.  Common culprits are wrapping bugs (where widths have to add up to some large amount in order to wrap) and -moz-column bugs (where the total height might be split into two equal parts, and the break point matters).  In both cases, you can use fuzzReset, and have one of the heights or widths depend on fuzzResetCount.  (Or you can just try forcing a small height or width to encourage maximal wrapping.)


=== Reduce the rest and convert script to markup ===

Now that you have a short sequence of JavaScript commands, you'll have to reduce the rest of the script (and the markup, if you're not using FRC2) by hand.  You want a final reduced testcase that include very little or no script.  Often, it's only necessary for the last appendChild or removeChild call to be dynamic; the rest of the testcase can be markup.

Here's an example of a reduced testcase where exactly one modification was left dynamic:
view-source:https://bugzilla.mozilla.org/attachment.cgi?id=250556
{'fraction_non_alphanumeric': 0.06610837438423646, 'fraction_numerical': 0.010935960591133005, 'mean_word_length': 3.8615900383141764, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '75885', 'n_tokens_mistral': 2903, 'n_tokens_neox': 2709, 'n_words': 1618}