Skip to content

Commit 59a81b6

Browse files
huydhnpytorchmergebot
authored andcommitted
Fix flaky linter clang-tidy relative path (pytorch#94093)
There are some occurrences when clang-tidy linter fails flakily with the following error, which is very weird: ``` >>> Lint for FILE: Error (CLANGTIDY) command-failed Failed due to FileNotFoundError: [Errno 2] No such file or directory: '.lintbin/clang-tidy' ``` For examples, * https://hud.pytorch.org/pytorch/pytorch/commit/0a93e6db5abdcc9196199d68f0c8e56578a74315 * https://hud.pytorch.org/pytorch/pytorch/commit/203b2cad3e4c650955a47d9973cfec83a3960056 The binary is definitely there as the log shows that it has been downloaded successfully from S3. Looking a bit closer, I notice that the linter uses `os.chdir` to jump around between the workspace and the build folder. And it also refers to the binary with the relative path `.lintbin/clang-tidy` which doesn't exist in the latter. AFAIK, the current working directory is per process (https://stackoverflow.com/questions/16388400/what-is-a-thread-specific-os-chdir-and-mkdir-in-python), so I suspect that there is a race here where one thread chdir into build while another thread tries to lint another file. Thus the fix to use the absolute path to clang-tidy Pull Request resolved: pytorch#94093 Approved by: https://github.com/malfet
1 parent e071d72 commit 59a81b6

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

tools/linter/adapters/clangtidy_linter.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,14 @@ def main() -> None:
253253

254254
abs_build_dir = Path(args.build_dir).resolve()
255255

256+
# Get the absolute path to clang-tidy and use this instead of the relative
257+
# path such as .lintbin/clang-tidy. The problem here is that os.chdir is
258+
# per process, and the linter uses it to move between the current directory
259+
# and the build folder. And there is no .lintbin directory in the latter.
260+
# When it happens in a race condition, the linter command will fails with
261+
# the following no such file or directory error: '.lintbin/clang-tidy'
262+
binary_path = os.path.abspath(args.binary)
263+
256264
with concurrent.futures.ThreadPoolExecutor(
257265
max_workers=os.cpu_count(),
258266
thread_name_prefix="Thread",
@@ -261,7 +269,7 @@ def main() -> None:
261269
executor.submit(
262270
check_file,
263271
filename,
264-
args.binary,
272+
binary_path,
265273
abs_build_dir,
266274
): filename
267275
for filename in args.filenames

0 commit comments

Comments
 (0)