Package gomega/gmeasure provides support for benchmarking and measuring code. It is intended as a more robust replacement for Ginkgo V1's Measure nodes.

gmeasure is organized around the metaphor of an Experiment that can record multiple Measurements. A Measurement is a named collection of data points and gmeasure supports measuring Values (of type float64) and Durations (of type time.Duration).

Experiments allows the user to record Measurements directly by passing in Values (i.e. float64) or Durations (i.e. time.Duration) or to measure measurements by passing in functions to measure. When measuring functions Experiments take care of timing the duration of functions (for Duration measurements) and/or recording returned values (for Value measurements). Experiments also support sampling functions - when told to sample Experiments will run functions repeatedly and measure and record results. The sampling behavior is configured by passing in a SamplingConfig that can control the maximum number of samples, the maximum duration for sampling (or both) and the number of concurrent samples to take.

Measurements can be decorated with additional information. This is supported by passing in special typed decorators when recording measurements. These include:

- Units("any string") - to attach units to a Value Measurement (Duration Measurements always have units of "duration") - Style("any Ginkgo color style string") - to attach styling to a Measurement. This styling is used when rendering console information about the measurement in reports. Color style strings are documented at TODO. - Precision(integer or time.Duration) - to attach precision to a Measurement. This controls how many decimal places to show for Value Measurements and how to round Duration Measurements when rendering them to screen.

In addition, individual data points in a Measurement can be annotated with an Annotation("any string"). The annotation is associated with the individual data point and is intended to convey additional context about the data point.

Once measurements are complete, an Experiment can generate a comprehensive report by calling its String() or ColorableString() method.

Users can also access and analyze the resulting Measurements directly. Use Experiment.Get(NAME) to fetch the Measurement named NAME. This returned struct will have fields containing all the data points and annotations recorded by the experiment. You can subsequently fetch the Measurement.Stats() to get a Stats struct that contains basic statistical information about the Measurement (min, max, median, mean, standard deviation). You can order these Stats objects using RankStats() to identify best/worst performers across multiple experiments or measurements.

gmeasure also supports caching Experiments via an ExperimentCache. The cache supports storing and retrieving experiments by name and version. This allows you to rerun code without repeating expensive experiments that may not have changed (which can be controlled by the cache version number). It also enables you to compare new experiment runs with older runs to detect variations in performance/behavior.

When used with Ginkgo, you can emit experiment reports and encode them in test reports easily using Ginkgo V2's support for Report Entries. Simply pass your experiment to AddReportEntry to get a report every time the tests run. You can also use AddReportEntry with Measurements to emit all the captured data and Rankings to emit measurement summaries in rank order.

Finally, Experiments provide an additional mechanism to measure durations called a Stopwatch. The Stopwatch makes it easy to pepper code with statements that measure elapsed time across different sections of code and can be useful when debugging or evaluating bottlenecks in a given codepath.

DefaultPrecisionBundle captures the default precisions for Vale and Duration measurements.

This section is empty.

The Annotation decorator allows you to attach an annotation to a given recorded data-point:

For example:

e := gmeasure.NewExperiment("My Experiment")
e.RecordValue("length", 3.141, gmeasure.Annotation("bob"))
e.RecordValue("length", 2.71, gmeasure.Annotation("jane"))

...will result in a Measurement named "length" that records two values )[3.141, 2.71]) annotation with (["bob", "jane"])

type CachedExperimentHeader struct {
}

CachedExperimentHeader captures the name of the Cached Experiment and its Version

type Experiment struct {
	Name string


	Measurements Measurements
}

Experiment is gmeasure's core data type. You use experiments to record Measurements and generate reports. Experiments are thread-safe and all methods can be called from multiple goroutines.

NexExperiment creates a new experiment with the passed-in name.

When using Ginkgo we recommend immediately registering the experiment as a ReportEntry:

experiment = NewExperiment("My Experiment")
AddReportEntry(experiment.Name, experiment)

this will ensure an experiment report is emitted as part of the test output and exported with any test reports.

func (e *Experiment) ColorableString() string

ColorableString returns a Ginkgo formatted summary of the experiment and all its Measurements. It is called automatically by Ginkgo's reporting infrastructure when the Experiment is registered as a ReportEntry via AddReportEntry.

Get returns the Measurement with the associated name. If no Measurement is found a zero Measurement{} is returned.

GetStats returns the Stats for the Measurement with the associated name. If no Measurement is found a zero Stats{} is returned.

experiment.GetStats(name) is equivalent to experiment.Get(name).Stats()

MeasureDuration runs the passed-in callback and times how long it takes to complete. The resulting duration is recorded on a Duration Measurement with the passed-in name. If the Measurement does not exist it is created.

MeasureDuration supports the Style(), Precision(), and Annotation() decorations.

MeasureValue runs the passed-in callback and records the return value on a Value Measurement with the passed-in name. If the Measurement does not exist it is created.

MeasureValue supports the Style(), Units(), Precision(), and Annotation() decorations.

func (e *Experiment) NewStopwatch() *Stopwatch

NewStopwatch() returns a stopwatch configured to record duration measurements with this experiment.

RecordDuration records the passed-in duration on a Duration Measurement with the passed-in name. If the Measurement does not exist it is created.

RecordDuration supports the Style(), Precision(), and Annotation() decorations.

RecordNote records a Measurement of type MeasurementTypeNote - this is simply a textual note to annotate the experiment. It will be emitted in any experiment reports.

RecordNote supports the Style() decoration.

RecordValue records the passed-in value on a Value Measurement with the passed-in name. If the Measurement does not exist it is created.

RecordValue supports the Style(), Units(), Precision(), and Annotation() decorations.

func (e *Experiment) Sample(callback func(idx int), samplingConfig SamplingConfig)

Sample samples the passed-in callback repeatedly. The sampling is governed by the passed in SamplingConfig.

The SamplingConfig can limit the total number of samples and/or the total time spent sampling the callback. The SamplingConfig can also instruct Sample to run with multiple concurrent workers.

The callback is called with a zero-based index that incerements by one between samples.

func (e *Experiment) SampleAnnotatedDuration(name string, callback func(idx int) Annotation, samplingConfig SamplingConfig, args ...any)

SampleDuration samples the passed-in callback and times how long it takes to complete each sample. The resulting durations are recorded on a Duration Measurement with the passed-in name. If the Measurement does not exist it is created.

The callback is given a zero-based index that increments by one between samples. The callback must return an Annotation - this annotation is attached to the measured duration.

The Sampling is configured via the passed-in SamplingConfig ¶

SampleAnnotatedDuration supports the Style() and Precision() decorations.

SampleAnnotatedValue samples the passed-in callback and records the return value on a Value Measurement with the passed-in name. If the Measurement does not exist it is created.

The callback is given a zero-based index that increments by one between samples. The callback must return a float64 and an Annotation - the annotation is attached to the recorded value.

The Sampling is configured via the passed-in SamplingConfig ¶

SampleValue supports the Style(), Units(), and Precision() decorations.

func (e *Experiment) SampleDuration(name string, callback func(idx int), samplingConfig SamplingConfig, args ...any)

The callback is given a zero-based index that increments by one between samples. The Sampling is configured via the passed-in SamplingConfig

SampleDuration supports the Style(), Precision(), and Annotation() decorations. When passed an Annotation() the same annotation is applied to all sample measurements.

SampleValue samples the passed-in callback and records the return value on a Value Measurement with the passed-in name. If the Measurement does not exist it is created.

The callback is given a zero-based index that increments by one between samples. The callback must return a float64. The Sampling is configured via the passed-in SamplingConfig

SampleValue supports the Style(), Units(), Precision(), and Annotation() decorations. When passed an Annotation() the same annotation is applied to all sample measurements.

ColorableString returns an unformatted summary of the experiment and all its Measurements.

type ExperimentCache struct {
	Path string
}

ExperimentCache provides a director-and-file based cache of experiments

NewExperimentCache creates and initializes a new cache. Path must point to a directory (if path does not exist, NewExperimentCache will create a directory at path).

Cached Experiments are stored as separate files in the cache directory - the filename is a hash of the Experiment name. Each file contains two JSON-encoded objects - a CachedExperimentHeader that includes the experiment's name and cache version number, and then the Experiment itself.

Clear empties out the cache - this will delete any and all detected cache files in the cache directory. Use with caution!

Delete removes the experiment with the passed-in name from the cache

List returns a list of all Cached Experiments found in the cache.

Load fetches an experiment from the cache. Lookup occurs by name. Load requires that the version number in the cache is equal to or greater than the passed-in version.

If an experiment with corresponding name and version >= the passed-in version is found, it is unmarshaled and returned.

If no experiment is found, or the cached version is smaller than the passed-in version, Load will return nil.

When paired with Ginkgo you can cache experiments and prevent potentially expensive recomputation with this pattern:

	const EXPERIMENT_VERSION = 1 //bump this to bust the cache and recompute _all_ experiments
    Describe("some experiments", func() {
    	var cache gmeasure.ExperimentCache
    	var experiment *gmeasure.Experiment
    	BeforeEach(func() {
    		cache = gmeasure.NewExperimentCache("./gmeasure-cache")
    		name := CurrentSpecReport().LeafNodeText
    		experiment = cache.Load(name, EXPERIMENT_VERSION)
    		if experiment != nil {
    			AddReportEntry(experiment)
    			Skip("cached")
    		}
    		experiment = gmeasure.NewExperiment(name)
			AddReportEntry(experiment)
    	})
    	It("foo runtime", func() {
    		experiment.SampleDuration("runtime", func() {
    			//do stuff
    		}, gmeasure.SamplingConfig{N:100})
    	})
    	It("bar runtime", func() {
    		experiment.SampleDuration("runtime", func() {
    			//do stuff
    		}, gmeasure.SamplingConfig{N:100})
    	})
    	AfterEach(func() {
    		if !CurrentSpecReport().State.Is(types.SpecStateSkipped) {
	    		cache.Save(experiment.Name, EXPERIMENT_VERSION, experiment)
	    	}
    	})
    })

Save stores the passed-in experiment to the cache with the passed-in name and version.

Measurement records all captured data for a given measurement. You generally don't make Measurements directly - but you can fetch them from Experiments using Get().

When using Ginkgo, you can register Measurements as Report Entries via AddReportEntry. This will emit all the captured data points when Ginkgo generates the report.

ColorableString generates a styled report that includes all the data points for this Measurement. It is called automatically by Ginkgo's reporting infrastructure when the Measurement is registered as a ReportEntry via AddReportEntry.

func (m Measurement) Stats() Stats

Stats returns a Stats struct summarizing the statistic of this measurement

String generates an unstyled report that includes all the data points for this Measurement.

type MeasurementType uint

const (
	MeasurementTypeInvalid MeasurementType = iota
	MeasurementTypeNote
	MeasurementTypeDuration
	MeasurementTypeValue
)

type Measurements []Measurement

The PrecisionBundle decorator controls the rounding of value and duration measurements. See Precision().

Precision() allows you to specify the precision of a value or duration measurement - this precision is used when rendering the measurement to screen.

To control the precision of Value measurements, pass Precision an integer. This will denote the number of decimal places to render (equivalen to the format string "%.Nf") To control the precision of Duration measurements, pass Precision a time.Duration. Duration measurements will be rounded oo the nearest time.Duration when rendered.

For example:

e := gmeasure.NewExperiment("My Experiment")
e.RecordValue("length", 3.141, gmeasure.Precision(2))
e.RecordValue("length", 2.71)
e.RecordDuration("cooking time", 3214 * time.Millisecond, gmeasure.Precision(100*time.Millisecond))
e.RecordDuration("cooking time", 2623 * time.Millisecond)

type Ranking struct {
	Criteria RankingCriteria
	Stats    []Stats
}

Ranking ranks a set of Stats by a specified RankingCriteria. Use RankStats to create a Ranking.

When using Ginkgo, you can register Rankings as Report Entries via AddReportEntry. This will emit a formatted table representing the Stats in rank-order when Ginkgo generates the report.

func RankStats(criteria RankingCriteria, stats ...Stats) Ranking

RankStats creates a new ranking of the passed-in stats according to the passed-in criteria.

func (c Ranking) ColorableString() string

ColorableString generates a styled report that includes a table of the rank-ordered Stats It is called automatically by Ginkgo's reporting infrastructure when the Ranking is registered as a ReportEntry via AddReportEntry.

String generates an unstyled report that includes a table of the rank-ordered Stats

func (c Ranking) Winner() Stats

Winner returns the Stats with the most optimal rank based on the specified ranking criteria. For example, if the RankingCriteria is LowerMaxIsBetter then the Stats with the lowest value or duration for StatMax will be returned as the "winner"

type RankingCriteria uint

RankingCriteria is an enum representing the criteria by which Stats should be ranked. The enum names should be self explanatory. e.g. LowerMeanIsBetter means that Stats with lower mean values are considered more beneficial, with the lowest mean being declared the "winner" .

const (
	LowerMeanIsBetter RankingCriteria = iota
	HigherMeanIsBetter
	LowerMedianIsBetter
	HigherMedianIsBetter
	LowerMinIsBetter
	HigherMinIsBetter
	LowerMaxIsBetter
	HigherMaxIsBetter
)

SamplingConfig configures the Sample family of experiment methods. These methods invoke passed-in functions repeatedly to sample and record a given measurement. SamplingConfig is used to control the maximum number of samples or time spent sampling (or both). When both are specified sampling ends as soon as one of the conditions is met. SamplingConfig can also ensure a minimum interval between samples and can enable concurrent sampling.

Stat is an enum representing the statistics you can request of a Stats struct

const (
	StatInvalid Stat = iota
	StatMin
	StatMax
	StatMean
	StatMedian
	StatStdDev
)

Stats records the key statistics for a given measurement. You generally don't make Stats directly - but you can fetch them from Experiments using GetStats() and from Measurements using Stats().

When using Ginkgo, you can register Measurements as Report Entries via AddReportEntry. This will emit all the captured data points when Ginkgo generates the report.

DurationFor returns the time.Duration for a particular Stat. You should only use this if the Stats has Type StatsTypeDuration For example:

mean := experiment.GetStats("runtime").ValueFor(gmeasure.StatMean)

will return the mean duration for the "runtime" Measurement.

FloatFor returns a float64 representation of the passed-in Stat. When Type is StatsTypeValue this is equivalent to s.ValueFor(stat). When Type is StatsTypeDuration this is equivalent to float64(s.DurationFor(stat))

String returns a minimal summary of the stats of the form "MIN < [MEDIAN] | <MEAN> ±STDDEV < MAX"

StringFor returns a formatted string representation of the passed-in Stat. The formatting honors the precision directives provided in stats.PrecisionBundle

ValueFor returns the float64 value for a particular Stat. You should only use this if the Stats has Type StatsTypeValue For example:

median := experiment.GetStats("length").ValueFor(gmeasure.StatMedian)

will return the median data point for the "length" Measurement.

const (
	StatsTypeInvalid StatsType = iota
	StatsTypeValue
	StatsTypeDuration
)

type Stopwatch struct {
	Experiment *Experiment

}

Stopwatch provides a convenient abstraction for recording durations. There are two ways to make a Stopwatch:

You can make a Stopwatch from an Experiment via experiment.NewStopwatch(). This is how you first get a hold of a Stopwatch.

You can subsequently call stopwatch.NewStopwatch() to get a fresh Stopwatch. This is only necessary if you need to record durations on a different goroutine as a single Stopwatch is not considered thread-safe.

The Stopwatch starts as soon as it is created. You can Pause() the stopwatch and Reset() it as needed.

Stopwatches refer back to their parent Experiment. They use this reference to record any measured durations back with the Experiment.

func (s *Stopwatch) NewStopwatch() *Stopwatch

NewStopwatch returns a new Stopwatch pointing to the same Experiment as this Stopwatch

func (s *Stopwatch) Pause() *Stopwatch

Pause pauses the Stopwatch. While pasued the Stopwatch does not accumulate elapsed time. This is useful for ignoring expensive operations that are incidental to the behavior you are attempting to characterize. Note: You must call Resume() before you can Record() subsequent measurements.

For example:

stopwatch := experiment.NewStopwatch()
// first expensive operation
stopwatch.Record("first operation").Reset()
// second expensive operation - part 1
stopwatch.Pause()
// something expensive that we don't care about
stopwatch.Resume()
// second expensive operation - part 2
stopwatch.Record("second operation").Reset() // the recorded duration captures the time elapsed during parts 1 and 2 of the second expensive operation, but not the bit in between

The Stopwatch must be running when Pause is called.

Record captures the amount of time that has passed since the Stopwatch was created or most recently Reset(). It records the duration on it's associated Experiment in a Measurement with the passed-in name.

Record takes all the decorators that experiment.RecordDuration takes (e.g. Annotation("...") can be used to annotate this duration)

Note that Record does not Reset the Stopwatch. It does, however, return the Stopwatch so the following pattern is common:

stopwatch := experiment.NewStopwatch()
// first expensive operation
stopwatch.Record("first operation").Reset() //records the duration of the first operation and resets the stopwatch.
// second expensive operation
stopwatch.Record("second operation").Reset() //records the duration of the second operation and resets the stopwatch.

omitting the Reset() after the first operation would cause the duration recorded for the second operation to include the time elapsed by both the first _and_ second operations.

The Stopwatch must be running (i.e. not paused) when Record is called.

func (s *Stopwatch) Reset() *Stopwatch

Reset resets the Stopwatch. Subsequent recorded durations will measure the time elapsed from the moment Reset was called. If the Stopwatch was Paused it is unpaused after calling Reset.

func (s *Stopwatch) Resume() *Stopwatch

Resume resumes a paused Stopwatch. Any time that elapses after Resume is called will be accumulated as elapsed time when a subsequent duration is Recorded.

The Stopwatch must be Paused when Resume is called

The Style decorator allows you to associate a style with a measurement. This is used to generate colorful console reports using Ginkgo V2's console formatter. Styles are strings in curly brackets that correspond to a color or style.

For example:

e := gmeasure.NewExperiment("My Experiment")
e.RecordValue("length", 3.141, gmeasure.Style("{{blue}}{{bold}}"))
e.RecordValue("length", 2.71)
e.RecordDuration("cooking time", 3 * time.Second, gmeasure.Style("{{red}}{{underline}}"))
e.RecordDuration("cooking time", 2 * time.Second)

will emit a report with blue bold entries for the length measurement and red underlined entries for the cooking time measurement.

Units are only set the first time a value or duration of a given name is recorded. In the example above any subsequent calls to e.RecordValue("length", X) will maintain the "{{blue}}{{bold}}" style even if a new Style is passed in later.

The Units decorator allows you to specify units (an arbitrary string) when recording values. It is ignored when recording durations.

e := gmeasure.NewExperiment("My Experiment")
e.RecordValue("length", 3.141, gmeasure.Units("inches"))

Units are only set the first time a value of a given name is recorded. In the example above any subsequent calls to e.RecordValue("length", X) will maintain the "inches" units even if a new set of Units("UNIT") are passed in later.