DPDK: Fix source for tarball #3505
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixing one bug uncovered a few others. Fixes the way the Tar tool handles fetching the filename of the tar file it downloads.
squirrelsc
reviewed
Nov 11, 2024
When running Wget.get(..., force_run=False) and Tar.extract it is useful to allow Tar to skip extracting existing files on the second pass. Allow the skip-old-files option, so Tar.extract will not overwrite existing files in the output directory. Note: it's important to not use this option if you are providing LISA with a default filename for your tarballs. This option could silently allow Tar.extract to not update the contents of a directory with the newer file contents. I don't see anyone using that schema now. My apologies to future devs who find this commit message while debugging that issue.
|
Note: testing in progress. |
mcgov
force-pushed
the
mcgov/source_fix
branch
from
2bed636
to
a0638eb
Compare
squirrelsc
reviewed
Nov 12, 2024
| @@ -24,6 +24,10 @@ class Wget(Tool): | |||
| def command(self) -> str: | |||
| return "wget" | |||
|
|
|||
| def _initialize(self, *args: Any, **kwargs: Any) -> None: | |||
| self.__filename_result_cache: Dict[str, str] = dict() | |||
There was a problem hiding this comment.
- This variable should be used by subclasses, so one underscore is right.
- Call it like
_url_file_map. It explains the variable names.
squirrelsc
reviewed
Nov 12, 2024
| @@ -45,8 +49,26 @@ def get( | |||
| force_run: bool = False, | |||
| timeout: int = 600, | |||
| ) -> str: | |||
| if not force_run: | |||
There was a problem hiding this comment.
The logic could be simpler.
cached_filename = self.__filename_result_cache.get(url, None)
if cached_filename:
if force_run:
del self.__filename_result_cache[url]
else:
return cached_filename
squirrelsc
reviewed
Nov 12, 2024
| @@ -48,6 +49,21 @@ def extract( | |||
| if strip_components: | |||
| # optionally strip N top level components from a tar file | |||
| tar_cmd += f" --strip-components={strip_components}" | |||
|
|
|||
| if skip_old_files: | |||
There was a problem hiding this comment.
How about remove existing extracted files always, or always overwrites? The skip-old-files is hard to use with other tools right.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix the way the TarDownloader handles dowloading files and identifying their filenames. This required fixing a bug in the Tar tool where Linux would not infer the filename in the default case like the Windows version does. Previously, providing a directory and not a filename would result in the tool identifying an entire directory of files as the filename when the Wget tool fetches a cached result.
This PR fixes that issue by allowing the tool to infer the filename, this fixes the ability to use the Wget tool without force_run when using a default filename.