Reduction workflow scripts #129
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| test_name,file,line,covered-points | ||
| test-a,/build/llvm/llvm/lib/Target/AMDGPU/FooA.cpp,100,0x1000 | ||
| test-b,/build/llvm/llvm/lib/Target/AMDGPU/FooB.cpp,200,0x3000;0x2000 | ||
| test-c,/build/llvm/llvm/lib/Target/AMDGPU/FooC.cpp,300,0x4000 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| #!/usr/bin/env bash | ||
| # Build a minimal gap-fill output tree under DEST for reduction-batch lit tests. | ||
| set -euo pipefail | ||
|
|
||
| if [[ $# -ne 1 ]]; then | ||
| echo "usage: $(basename "$0") <dest-gap-fill-dir>" >&2 | ||
| exit 2 | ||
| fi | ||
|
|
||
| dest="$1" | ||
| fixture_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
|
|
||
| mkdir -p "${dest}/incremental" "${dest}/candidate_tests" "${dest}/fake-llvm/bin" | ||
| cp "${fixture_dir}/new_coverage.csv" "${dest}/incremental/" | ||
|
|
||
| fake_llc="${dest}/fake-llvm/bin/llc" | ||
| printf '#!/bin/sh\n' >"${fake_llc}" | ||
| chmod +x "${fake_llc}" | ||
|
|
||
| for name in test-a test-b test-c; do | ||
| test_dir="${dest}/candidate_tests/${name}" | ||
| mkdir -p "${test_dir}" | ||
| cp "${fixture_dir}/sample.bc" "${test_dir}/sample.bc" | ||
| cat >"${test_dir}/test.sh" <<EOF | ||
| #!/bin/bash | ||
| ${fake_llc} -O1 -mtriple=amdgcn-amd-amdhsa ${test_dir}/sample.bc | ||
| EOF | ||
| chmod +x "${test_dir}/test.sh" | ||
| done |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # COM: Scaffold-only batch-from-coverage for n=1 and n=3 (no llvm-reduce/creduce). | ||
|
|
||
| # RUN: rm -rf %t && mkdir -p %t | ||
| # RUN: bash %S/fixtures/reduction-batch/prepare-gap-fill-fixture.sh %t/gap-fill | ||
|
Collaborator
There was a problem hiding this comment. Would it be possible to run it on a real but small case? We could run a simple baseline with a one-test filter, run one candidate test, this will necessarily identify "gaps", and we reduce in regard to that. In this way we are testing a real scenario without having to maintain
Collaborator
Author
There was a problem hiding this comment. There is a real case in the e2e pipeline - do you mean something else? The
Collaborator
There was a problem hiding this comment. It would be handy to make sure that tool is performing a true reduction (not using a fake llc) without having to analyze a PR. Also, while making it real it could also reduce the boilerplate around the test (i.e.,
The integration test should be a real execution of the tool. This test is not only stubbing some initial files but almost the entire reduction. |
||
| # RUN: %reduce batch-from-coverage --csv %t/gap-fill/incremental/new_coverage.csv --candidate-tests %t/gap-fill/candidate_tests --output %t/out-n1 --template-dir %repo-root/example/amd/new-test-1 --n 1 | ||
| # RUN: test -d %t/out-n1/t-00001-test-a | ||
| # RUN: test -f %t/out-n1/t-00001-test-a/config.json | ||
| # RUN: test -f %t/out-n1/t-00001-test-a/interesting_ir.sh | ||
| # RUN: %FileCheck %s --input-file %t/out-n1/t-00001-test-a/config.json --check-prefix=CFG1 | ||
| # RUN: %FileCheck %s --input-file %t/out-n1/t-00001-test-a/interesting_ir.sh --check-prefix=IR1 | ||
|
|
||
| # RUN: %reduce batch-from-coverage --csv %t/gap-fill/incremental/new_coverage.csv --candidate-tests %t/gap-fill/candidate_tests --output %t/out-n3 --template-dir %repo-root/example/amd/new-test-1 --n 3 | ||
|
Collaborator
There was a problem hiding this comment. What's the difference between this call and the one above? Could we have just one?
Collaborator
Author
There was a problem hiding this comment. The difference is the number of tests - I am testing reducing 1 or reducing 3. I guess we could just have the reduce 3 if you want to simplify? |
||
| # RUN: test -d %t/out-n3/t-00001-test-a | ||
| # RUN: test -d %t/out-n3/t-00002-test-b | ||
| # RUN: test -d %t/out-n3/t-00003-test-c | ||
| # RUN: %FileCheck %s --input-file %t/out-n3/t-00002-test-b/config.json --check-prefix=CFG2 | ||
| # RUN: %FileCheck %s --input-file %t/out-n3/t-00002-test-b/interesting_ir.sh --check-prefix=IR2 | ||
|
|
||
| # CFG1: "file": "llvm/lib/Target/AMDGPU/FooA.cpp", | ||
|
ambergorzynski marked this conversation as resolved.
|
||
| # CFG1: "line": 100, | ||
| # CFG2: "file": "llvm/lib/Target/AMDGPU/FooB.cpp", | ||
| # CFG2: "line": 200, | ||
| # IR1: COVERED="0x1000" | ||
| # IR2: COVERED="0x2000" | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a new one or we could use an existing one under integration-tests?