generated from oracle/template-repo
-
Notifications
You must be signed in to change notification settings - Fork 29
feat: add reproducible central buildspec generation #1115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tromai
wants to merge
14
commits into
main
Choose a base branch
from
tromai/add-build-spec-generation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
c225e28
feat: add macaron database extractor module
tromai 95432bd
feat: add maven and gradle cli parsers
tromai 8203dae
feat: add jdk version finder from maven central java artifacts
tromai e1f8994
feat: add cli build command patcher
tromai 226ac79
feat: add jdk version normalizer
tromai 01f235d
feat: add reproducible central buildspec generation
tromai d37e1ab
feat: expose macaron gen-build-spec cli command
tromai 9d8a2a7
test: modify the integration test script to use the compare rc build …
tromai da329fe
test: add integration tests for the gen-build-spec error
tromai f52c6d4
fix: add jdk version 22 and 23 into the list of supported jdk major v…
tromai 5c546f2
chore: add a small log message at the beginning of build spec generation
tromai 76bdd3a
fix: fix the sql statement for obtaining build check facts where the …
tromai 63d610f
chore: move the looking up of repository before the build tool lookup…
tromai 9b1d522
chore: support gen-build-spec for the Docker image
tromai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# Copyright (c) 2025 - 2025, Oracle and/or its affiliates. All rights reserved. | ||
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/. |
137 changes: 137 additions & 0 deletions
137
src/macaron/build_spec_generator/build_command_patcher.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
# Copyright (c) 2025 - 2025, Oracle and/or its affiliates. All rights reserved. | ||
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/. | ||
|
||
"""This module contains the implementation of the build command patching.""" | ||
|
||
import logging | ||
from collections.abc import Mapping, Sequence | ||
|
||
from macaron.build_spec_generator.cli_command_parser import CLICommand, CLICommandParser, PatchCommandBuildTool | ||
from macaron.build_spec_generator.cli_command_parser.gradle_cli_parser import ( | ||
GradleCLICommandParser, | ||
GradleOptionPatchValueType, | ||
) | ||
from macaron.build_spec_generator.cli_command_parser.maven_cli_parser import ( | ||
CommandLineParseError, | ||
MavenCLICommandParser, | ||
MavenOptionPatchValueType, | ||
PatchBuildCommandError, | ||
) | ||
from macaron.build_spec_generator.cli_command_parser.unparsed_cli_command import UnparsedCLICommand | ||
|
||
logger: logging.Logger = logging.getLogger(__name__) | ||
|
||
MVN_CLI_PARSER = MavenCLICommandParser() | ||
GRADLE_CLI_PARSER = GradleCLICommandParser() | ||
|
||
PatchValueType = GradleOptionPatchValueType | MavenOptionPatchValueType | ||
|
||
|
||
def _patch_commands( | ||
cmds_sequence: Sequence[list[str]], | ||
cli_parsers: Sequence[CLICommandParser], | ||
patches: Mapping[ | ||
PatchCommandBuildTool, | ||
Mapping[str, PatchValueType | None], | ||
], | ||
) -> list[CLICommand] | None: | ||
"""Patch the sequence of build commands, using the provided CLICommandParser instances. | ||
|
||
For each command in `cmds_sequence`, it will be checked against all CLICommandParser instances until there is | ||
one that can parse it, then a patch from ``patches`` is applied for this command if provided. | ||
|
||
If a command doesn't have any corresponding ``CLICommandParser`` instance it will be parsed as UnparsedCLICommand, | ||
which just holds the original command as a list of string, without any changes. | ||
""" | ||
result: list[CLICommand] = [] | ||
for cmds in cmds_sequence: | ||
effective_cli_parser = None | ||
for cli_parser in cli_parsers: | ||
if cli_parser.is_build_tool(cmds[0]): | ||
effective_cli_parser = cli_parser | ||
break | ||
|
||
if not effective_cli_parser: | ||
result.append(UnparsedCLICommand(original_cmds=cmds)) | ||
continue | ||
|
||
try: | ||
cli_command = effective_cli_parser.parse(cmds) | ||
except CommandLineParseError as error: | ||
logger.error( | ||
"Failed to parse the mvn command %s. Error %s.", | ||
" ".join(cmds), | ||
error, | ||
) | ||
return None | ||
|
||
patch = patches.get(effective_cli_parser.build_tool, None) | ||
if not patch: | ||
result.append(cli_command) | ||
continue | ||
|
||
try: | ||
new_cli_command = effective_cli_parser.apply_patch( | ||
cli_command=cli_command, | ||
options_patch=patch, | ||
) | ||
except PatchBuildCommandError as error: | ||
logger.error( | ||
"Failed to patch the mvn command %s. Error %s.", | ||
" ".join(cmds), | ||
error, | ||
) | ||
return None | ||
|
||
result.append(new_cli_command) | ||
|
||
return result | ||
|
||
|
||
def patch_commands( | ||
cmds_sequence: Sequence[list[str]], | ||
patches: Mapping[ | ||
PatchCommandBuildTool, | ||
Mapping[str, PatchValueType | None], | ||
], | ||
) -> list[list[str]] | None: | ||
"""Patch a sequence of CLI commands. | ||
|
||
For each command in this command sequence: | ||
|
||
- If the command is not a build command or the build tool is not supported by us, it will be leave intact. | ||
|
||
- If the command is a build command supported by us, it will be patch if a patch value is provided to ``patches``. | ||
If no patch value is provided for a build command, it will be leave intact. | ||
|
||
`patches` is a mapping with: | ||
|
||
- **Key**: an instance of the ``BuildTool`` enum | ||
|
||
- **Value**: the patch value provided to ``CLICommandParser.apply_patch``. For more information on the patch value | ||
see the concrete implementations of the ``CLICommandParser.apply_patch`` method. | ||
For example: :class:`macaron.cli_command_parser.maven_cli_parser.MavenCLICommandParser.apply_patch`, | ||
:class:`macaron.cli_command_parser.gradle_cli_parser.GradleCLICommandParser.apply_patch`. | ||
|
||
This means that all commands that matches a BuildTool will be apply by the same patch value. | ||
|
||
Returns | ||
------- | ||
list[list[str]] | None | ||
The patched command sequence or None if there is an error. The errors that can happen if any command | ||
which we support is invalid in ``cmds_sequence``, or the patch value is valid. | ||
""" | ||
result = [] | ||
patch_cli_commands = _patch_commands( | ||
cmds_sequence=cmds_sequence, | ||
cli_parsers=[MVN_CLI_PARSER, GRADLE_CLI_PARSER], | ||
patches=patches, | ||
) | ||
|
||
if patch_cli_commands is None: | ||
return None | ||
|
||
for patch_cmd in patch_cli_commands: | ||
result.append(patch_cmd.to_cmds()) | ||
|
||
return result |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.