Edit on GitHub

Comparing commits

You can use judge to compare benchmark results of two versions of the package.

BenchmarkTools.judge — Function
judge(pkg::Union{Module, String},
      [target]::Union{String, BenchmarkConfig},
      baseline::Union{String, BenchmarkConfig};
      kwargs...)

Arguments:

  • pkg - Package to benchmark. Either a package module, name, or directory.
  • target - What do judge, given as a git id or a BenchmarkConfig. If skipped, use the current state of the package repo.
  • baseline - The commit / BenchmarkConfig to compare target against.

Keyword arguments:

  • f - Estimator function to use in the judging.
  • judgekwargs::Dict{Symbol, Any} - keyword arguments to pass to the judge function in BenchmarkTools

The remaining keyword arguments are passed to benchmarkpkg

Return value:

Returns a BenchmarkJudgement

source
judge(target::BenchmarkResults, baseline::BenchmarkResults, f;
      judgekwargs = Dict())

Judges the two BenchmarkResults in target and baseline using the function f.

Return value

Returns a BenchmarkJudgement

source

which returns a BenchmarkJudgement

PkgBenchmark.BenchmarkJudgement — Type

Stores the results from running a judgement, see judge.

The following (unexported) methods are defined on a BenchmarkJudgement (written below as judgement):

  • target_result(judgement)::BenchmarkResults - the BenchmarkResults of the target.
  • baseline_result(judgement)::BenchmarkResults - the BenchmarkResults of the baseline.
  • benchmarkgroup(judgement)::BenchmarkGroup - a BenchmarkGroup containing the estimated results

A BenchmarkJudgement can be exported to markdown using the function export_markdown.

See also BenchmarkResults

source