criterion performance measurements

overview

want to understand this report?

PThread//counter/getTLS/readIORef_1io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.503509340931406e-8 3.504403620932745e-8 3.505357817144495e-8
Standard deviation 2.6395241375343026e-11 3.187700134321433e-11 3.942875085545667e-11

Outlying measurements have no (3.5586734693877544e-3%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_2io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.77420539322357e-8 1.861642921773286e-8 2.0815311319121057e-8
Standard deviation 1.0681655844646623e-10 3.962602833669687e-9 7.01538844759738e-9

Outlying measurements have severe (0.9817979615313311%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_3io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.2265202287498294e-8 1.289183610161753e-8 1.4377787067842252e-8
Standard deviation 1.7944502481756505e-9 3.3416285515270777e-9 5.0688110363597494e-9

Outlying measurements have severe (0.9892759255283491%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_4io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.675465886189709e-9 1.0355264041485497e-8 1.1477585360209155e-8
Standard deviation 1.9148126402513563e-9 2.9893393152019205e-9 3.931997339249421e-9

Outlying measurements have severe (0.9928359177869138%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_5io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7981312687133506e-8 1.8001656697249445e-8 1.8009315156880382e-8
Standard deviation 2.123679505077979e-11 4.096158996451708e-11 7.70422071038673e-11

Outlying measurements have no (3.389791290666007e-3%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_6io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8047660411365566e-8 1.8260237235612748e-8 1.888091404383871e-8
Standard deviation 5.590059455964554e-11 1.3534881414773318e-9 2.6102219146434426e-9

Outlying measurements have severe (0.8595998156836705%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_7io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8269910297114798e-8 1.870225945828542e-8 1.9803382250257302e-8
Standard deviation 2.69627498583088e-11 1.9657122702018877e-9 3.4583298613563994e-9

Outlying measurements have severe (0.9241675502142864%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_8io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8463103292193413e-8 1.84690986481367e-8 1.849047060598295e-8
Standard deviation 9.618384453388882e-12 3.488834273835308e-11 7.076727020889514e-11

Outlying measurements have no (3.401320923948122e-3%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_9io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.724952710041194e-8 2.7261573045686935e-8 2.727368196839553e-8
Standard deviation 3.0857051802923374e-11 3.73790235399059e-11 4.84425129680312e-11

Outlying measurements have no (3.4964604493690366e-3%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_10io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.7267580562280437e-8 2.76469578053614e-8 2.8785212792273525e-8
Standard deviation 6.678610638682547e-11 1.8147625413979735e-9 3.75444986198065e-9

Outlying measurements have severe (0.8227798682116929%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_11io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.750045967259563e-8 2.8007042154486025e-8 2.9017746578364546e-8
Standard deviation 5.4060534399262585e-11 2.3848103025618816e-9 4.353333567217634e-9

Outlying measurements have severe (0.8886835362428941%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_12io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.8341955377491333e-8 2.9240184539176822e-8 3.037060070882872e-8
Standard deviation 2.2356435800897633e-9 3.428094450477271e-9 4.209053028676222e-9

Outlying measurements have severe (0.9366029724775375%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_13io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.6457935873963033e-8 3.6480570519902414e-8 3.652029591881528e-8
Standard deviation 4.399753430951736e-11 9.859647051358279e-11 1.8360778499226078e-10

Outlying measurements have no (3.5713826903559895e-3%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_14io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.64888005425562e-8 3.6918459943672476e-8 3.799354669275369e-8
Standard deviation 4.9169927695020266e-11 1.9484533635099316e-9 3.3926336760625053e-9

Outlying measurements have severe (0.7447728884403282%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_15io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.6701827377195327e-8 3.691826721541994e-8 3.799062930667824e-8
Standard deviation 2.2312249209182094e-11 1.408934740523305e-9 3.2411395093609897e-9

Outlying measurements have severe (0.6018735874658285%) effect on estimated standard deviation.

PThread//counter/getTLS/readIORef_16io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.7139707844990886e-8 3.758156382451902e-8 3.88888904956564e-8
Standard deviation 1.825771052701032e-11 2.3844988204924052e-9 3.773811721297635e-9

Outlying measurements have severe (0.8072379813503198%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_1io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.414388686242594e-8 5.432142146431234e-8 5.455298405626335e-8
Standard deviation 5.514092098932056e-10 6.362887305524976e-10 6.755248745030763e-10

Outlying measurements have moderate (0.11884665172054729%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_2io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.785188641147839e-8 3.792489324889052e-8 3.8052087474755747e-8
Standard deviation 2.0427751320768716e-10 3.047092543316221e-10 4.782861745646763e-10

Outlying measurements have slight (6.1098073443395094e-2%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_3io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.5497548334381965e-8 2.5558761684471523e-8 2.5613539181413542e-8
Standard deviation 1.356044986215106e-10 1.9924113175424144e-10 3.016176774859041e-10

Outlying measurements have slight (5.9375926984726816e-2%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_4io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.4172734196610636e-8 2.421775236515616e-8 2.4264325645873517e-8
Standard deviation 1.4686914865207499e-10 1.6413888203896287e-10 1.963483793030851e-10

Outlying measurements have no (3.472180067744028e-3%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_5io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.3279125006033426e-8 4.351323083135997e-8 4.403578643122183e-8
Standard deviation 2.391457850332965e-10 1.13069283492967e-9 1.8516367027866691e-9

Outlying measurements have moderate (0.4054796136082915%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_6io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.669034323916898e-8 4.75478450511412e-8 4.807448529382007e-8
Standard deviation 1.6707366893486779e-9 2.2396510388256917e-9 2.8368206627404893e-9

Outlying measurements have severe (0.6940668944082075%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_7io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.8080871944216664e-8 4.840719987670336e-8 4.863260315707911e-8
Standard deviation 7.762423039059437e-10 9.212132955156266e-10 1.1332858931538455e-9

Outlying measurements have moderate (0.262797645236622%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_8io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.189173393587421e-8 5.265241300865464e-8 5.327168770443962e-8
Standard deviation 1.8446391161886352e-9 2.2687585135816392e-9 2.584550227912216e-9

Outlying measurements have severe (0.6517408638247328%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_9io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.433725692980347e-8 7.547443072208422e-8 7.87904521927773e-8
Standard deviation 3.1817175054626152e-9 5.745584914939696e-9 1.0518531886055681e-8

Outlying measurements have severe (0.8522977163263364%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_10io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.989272816562347e-8 8.069950356716127e-8 8.162788979223247e-8
Standard deviation 2.649606830534674e-9 3.035108688472655e-9 3.6947671220490575e-9

Outlying measurements have severe (0.5783785895420837%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_11io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.106457628556393e-8 8.200237707515936e-8 8.274417346294154e-8
Standard deviation 2.327628230971531e-9 2.8015130667484804e-9 3.379685656880055e-9

Outlying measurements have severe (0.5305800844151504%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_12io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.210995468956149e-8 8.305115070142184e-8 8.401949178129479e-8
Standard deviation 2.640540449724113e-9 3.2319786696535597e-9 4.254342815732334e-9

Outlying measurements have severe (0.5960007885550759%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_13io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1066991539000197e-7 1.1240258459966463e-7 1.140931734271576e-7
Standard deviation 5.371312068190004e-9 6.119149448956534e-9 7.092441224030365e-9

Outlying measurements have severe (0.7384156128744372%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_14io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0896829043091199e-7 1.1067695360980437e-7 1.1246342559543471e-7
Standard deviation 4.663211557387353e-9 5.760230746237189e-9 7.2967002933659324e-9

Outlying measurements have severe (0.7222229244752575%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_15io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1478488357260036e-7 1.1607375248796952e-7 1.174844021308874e-7
Standard deviation 3.830601716932955e-9 4.502669032824265e-9 5.383337342438183e-9

Outlying measurements have severe (0.5853407974745893%) effect on estimated standard deviation.

GHC//counter/getTLS/readIORef_16io_4os

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1721845341727552e-7 1.1851933632402174e-7 1.202339646076615e-7
Standard deviation 3.853002538578325e-9 4.8208788484006025e-9 6.656411608560277e-9

Outlying measurements have severe (0.6091308298972559%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.