<commit_before>// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// Benchmany runs Go benchmarks across many git commits.
//
// Usage:
//
//	benchmany run [-C git-dir] [-n iterations] <commit or range>...
//
//      benchmany plot [-C git-dir] <commit or range>...
//
//      benchmany list [-C git-dir] <commit or range>...
//
// run
//
// "benchmany run" runs the benchmarks in the current directory
// <iterations> times for each commit in <commit or range> and writes
// the benchmark results to log.<commit hash>. Benchmarks may be Go
// testing framework benchmarks or benchmarks from
// golang.org/x/benchmarks.
//
// <commit or range>... can be either a list of individual commits or
// a revision range. For the spelling of a revision range, see
// "SPECIFYING RANGES" in gitrevisions(7). For exact details, see the
// --no-walk option to git-rev-list(1).
//
// Benchmany will check out each revision in git-dir. The current
// directory may or may not be in the same git repository as git-dir.
// If git-dir refers to a Go installation, benchmany will run
// make.bash at each revision; otherwise, it assumes go test can
// rebuild the necessary dependencies.
//
// Benchmany is safe to interrupt. If it is restarted, it will parse
// the benchmark log files to recover its state.
//
// Benchmany supports multiple ways of prioritizing the order in which
// individual iterations are run. By default, it runs in "sequential"
// mode: it runs the first iteration of all benchmarks, then the
// second, and so forth. It also supports a "spread" mode designed to
// quickly get coverage for large sets of revisions. This mode
// randomizes the order to run iterations in, but biases this order
// toward covering an evenly distributed set of revisions early and
// finishing all of the iterations of the revisions it has started on
// before moving on to new revisions. This way, if benchmany is
// interrupted, the revisions benchmarked cover the space more-or-less
// evenly. Finally, it supports a "metric" mode, which zeroes in on
// changes in a benchmark metric by selecting the commit half way
// between the pair of commits with the biggest difference in the
// metric. This is like "git bisect", but for performance.
//
// plot
//
// "benchmany plot" reads the benchmark logs generated by "benchmany
// run" for the given <commit or range> and outputs tables that can be
// used to plot these benchmarks over time.
//
// list
//
// "benchmany list" prints a list of log file names generated by
// "benchmany run" for the given <commit or range>. By default, logs
// are listed from oldest to newest revision. This is useful as input
// to comparison tools like http://rsc.io/benchstat.
package main

import (
	"flag"
	"fmt"
	"os"
	"os/exec"
	"strings"
)

var gitDir string
var dryRun bool<commit_msg>benchmany: improve command doc<commit_after>// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// Benchmany runs Go benchmarks across many git commits.
//
// Usage:
//
//      benchmany run [-C git-dir] [-n iterations] <commit or range>...
//
//      benchmany plot [-C git-dir] <commit or range>...
//
//      benchmany list [-C git-dir] <commit or range>...
//
// Run subcommand
//
// "benchmany run" runs the benchmarks in the current directory
// <iterations> times for each commit in <commit or range> and writes
// the benchmark results to log.<commit hash>. Benchmarks may be Go
// testing framework benchmarks or benchmarks from
// golang.org/x/benchmarks.
//
// <commit or range>... can be either a list of individual commits or
// a revision range. For the spelling of a revision range, see
// "SPECIFYING RANGES" in gitrevisions(7). For exact details, see the
// --no-walk option to git-rev-list(1).
//
// Benchmany will check out each revision in git-dir. The current
// directory may or may not be in the same git repository as git-dir.
// If git-dir refers to a Go installation, benchmany will run
// make.bash at each revision; otherwise, it assumes go test can
// rebuild the necessary dependencies. Benchmany also supports using
// gover (https://godoc.org/github.com/aclements/go-misc/gover) to
// save and reuse Go build trees. This is useful for saving time
// across multiple benchmark runs and for benchmarks that depend on
// the Go tree itself (such as compiler benchmarks).
//
// Benchmany supports multiple ways of prioritizing the order in which
// individual iterations are run. By default, it runs in "sequential"
// mode: it runs the first iteration of all benchmarks, then the
// second, and so forth. It also supports a "spread" mode designed to
// quickly get coverage for large sets of revisions. This mode
// randomizes the order to run iterations in, but biases this order
// toward covering an evenly distributed set of revisions early and
// finishing all of the iterations of the revisions it has started on
// before moving on to new revisions. This way, if benchmany is
// interrupted, the revisions benchmarked cover the space more-or-less
// evenly. Finally, it supports a "metric" mode, which zeroes in on
// changes in a benchmark metric by selecting the commit half way
// between the pair of commits with the biggest difference in the
// metric. This is like "git bisect", but for performance.
//
// Benchmany is safe to interrupt. If it is restarted, it will parse
// the benchmark log files to recover its state.
//
// Plot subcommand
//
// "benchmany plot" reads the benchmark logs generated by "benchmany
// run" for the given <commit or range> and plots the results of
// benchmarks over time.
//
// Plot supports various output formats. By default, it outputs a text
// table that can be used as input to most plotting programs. The
// -html option produces a Google Charts page that can be used to
// interactively view benchmark results.
//
// Plot supports a -filter flag that reduces noise in benchmark
// results using an adaptive Kolmogorov-Zurbenko (KZA) low-pass
// filter. This filter retains sharp changes in performance, so you
// can see exactly when significant changes happened, while smoothing
// out transient noise.
//
// Plot supports normalizing results to a baseline using the -baseline
// flag. This is useful when looking for relative changes, or when
// plotting many different benchmarks that may have wildly different
// expected results.
//
// List subcommand
//
// "benchmany list" prints a list of log file names generated by
// "benchmany run" for the given <commit or range>. By default, logs
// are listed from oldest to newest revision. This is useful as input
// to comparison tools like http://rsc.io/benchstat.
package main

import (
	"flag"
	"fmt"
	"os"
	"os/exec"
	"strings"
)

var gitDir string
var dryRun bool
{'fraction_non_alphanumeric': 0.08953287197231834, 'fraction_numerical': 0.0017301038062283738, 'mean_word_length': 3.232458816351434, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2448462', 'n_tokens_mistral': 1899, 'n_tokens_neox': 1778, 'n_words': 1011}