diff --git a/.gitignore b/.gitignore index e443807..527a78f 100644 --- a/.gitignore +++ b/.gitignore @@ -16,4 +16,5 @@ testdata vault-key .idea .DS_Store -bin \ No newline at end of file +bin +benchmark/results.html \ No newline at end of file diff --git a/README.md b/README.md index d3dd4ba..59973da 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ As compared to `ansible-vault` (v2.9.11 on python v3.8.5), typical actions take | encrypt_string | 429 ms | **64 ms** | | encrypt + decrypt | 1,087 ms | **168 ms** | -See [`./benchmark/results.html`](https://raw.githubusercontent.com/GoodwayGroup/gwvault/master/benchmark/results.html) for a detailed breakdown of the results. +See [`./benchmark/results.html`](./benchmark/results.html) for a detailed breakdown of the results after running the benchmark. ## Built With diff --git a/benchmark/results.html b/benchmark/results.html deleted file mode 100644 index 3962e5c..0000000 --- a/benchmark/results.html +++ /dev/null @@ -1,1111 +0,0 @@ - - - - - criterion report - - - - - - - - - -
-
-

criterion performance measurements

- -

overview

- -

want to understand this report?

- -
- -

bench/./ansible-vault-encrypt.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.48471282415706140.49089480866179510.5109107636187348
Standard deviation4.176360362081773e-31.3673801077204834e-22.0399166016711115e-2
- - -

Outlying measurements have moderate - (0.12244897959183673%) - effect on estimated standard deviation.

-
-

bench/./gwvault-encrypt.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time8.799614949608082e-28.8629525776332e-28.900730617232591e-2
Standard deviation5.807767007341921e-49.972547118101713e-41.5795410789369389e-3
- - -

Outlying measurements have slight - (5.536332179930795e-2%) - effect on estimated standard deviation.

-
-

bench/./ansible-vault-decrypt.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.476826918781080130.5067539762470420.5443583530510778
Standard deviation3.818451484421398e-25.0367134962777595e-25.789626276543197e-2
- - -

Outlying measurements have moderate - (0.2672689853230438%) - effect on estimated standard deviation.

-
-

bench/./gwvault-decrypt.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time9.506872849859133e-29.602444657299489e-29.696212148375674e-2
Standard deviation1.615210336380169e-32.114096929002399e-32.8168711478317784e-3
- - -

Outlying measurements have slight - (5.53633217993079e-2%) - effect on estimated standard deviation.

-
-

bench/./ansible-vault-string.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.42471204794087220.42803772441690490.43268214948319816
Standard deviation3.241674649906193e-35.614496247321413e-38.471046026295666e-3
- - -

Outlying measurements have moderate - (0.12244897959183673%) - effect on estimated standard deviation.

-
-

bench/./gwvault-string.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time5.923397785509686e-25.948915604019876e-25.979412212483465e-2
Standard deviation4.0861921805163213e-46.414232614480645e-41.0769747057569558e-3
- - -

Outlying measurements have slight - (4.5351473922902494e-2%) - effect on estimated standard deviation.

-
-

bench/./ansible-vault-file.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.90631110165007470.91374645161004690.9240750379100428
Standard deviation5.393903981006103e-31.1205636220453591e-21.542422323897618e-2
- - -

Outlying measurements have moderate - (0.16000000000000003%) - effect on estimated standard deviation.

-
-

bench/./gwvault-file.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.154302689110351780.15561373293071440.15695866184990012
Standard deviation1.6966469325334782e-32.5261353593158393e-33.811514561890576e-3
- - -

Outlying measurements have slight - (7.100591715976311e-2%) - effect on estimated standard deviation.

-
-

bench/./ansible-vault-rekey.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.66287892468333980.72165681471120580.8090466152946546
Standard deviation4.873506144831662e-29.838676547648781e-20.1460107560009224
- - -

Outlying measurements have moderate - (0.31776599608044276%) - effect on estimated standard deviation.

-
-

bench/./gwvault-rekey.sh

- - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lower boundestimateupper bound
OLS regressionxxxxxxxxx
R² goodness-of-fitxxxxxxxxx
Mean execution time0.16005204254692520.1626616280863180.16561311435429607
Standard deviation3.8545745739126222e-35.242671865187316e-37.0081533232793924e-3
- - -

Outlying measurements have slight - (7.100591715976326e-2%) - effect on estimated standard deviation.

-
- -

understanding this report

- -

In this report, each function benchmarked by criterion is assigned - a section of its own. The charts in each section are active; if - you hover your mouse over data points and annotations, you will see - more details.

- - - -

Under the charts is a small table. - The first two rows are the results of a linear regression run - on the measurements displayed in the right-hand chart.

- - - -

We use a statistical technique called - the bootstrap - to provide confidence intervals on our estimates. The - bootstrap-derived upper and lower bounds on estimates let you see - how accurate we believe those estimates to be. (Hover the mouse - over the table headers to see the confidence levels.)

- -

A noisy benchmarking environment can cause some or many - measurements to fall far from the mean. These outlying - measurements can have a significant inflationary effect on the - estimate of the standard deviation. We calculate and display an - estimate of the extent to which the standard deviation has been - inflated by outliers.

- - - -
-
- - -