This repository contains a set of benchmarks for BuildKit that run on GitHub Actions public runners. Results are published on GitHub Pages.
To run locally, you can use the following command:
make testThis runs all tests and benchmarks from ./test package with BuildKit changes from default branch and latest Buildx stable. You can also specify a commit to test or multiple references and tweak the benchmark settings:
# run only tests
TEST_TYPES=test make test
# run only benchmarks
TEST_TYPES=benchmark make test
# or
make bench
# run a specific benchmark
TEST_BENCH_REGEXP=/BenchmarkBuildLocal$ make bench
# run all benchmarks 3 times (default 1)
TEST_BENCH_RUN=3 make bench
# run 5 iterations of each benchmark (default 1x)
TEST_BENCH_TIME=5x make bench
# run all with master, v0.9.3 and v0.16.0 buildkit references (with latest buildx stable)
BUILDKIT_REFS=master,v0.9.3,v0.16.0 make bench
# run all with master, v0.17.0 buildx git references (with latest buildkit stable)
BUILDX_REFS=master,v0.17.0 make benchNote
Set TEST_KEEP_CACHE=1 for the test framework to keep external dependant
images in a docker volume if you are repeatedly calling make test or
make bench. This helps to avoid rate limiting on the remote registry side.
After running the tests, you can generate the HTML report and serve the website with:
make genThen open http://localhost:8080 in your browser.
This project is licensed under the Apache License, Version 2.0 - see the LICENSE file for details.