Skip to content

Reference cleanup scripts in generateCleanupCommand.sh #307

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Reference cleanup scripts in generateCleanupCommand.sh #307

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fa45ede
Reference cleanup scripts in generateCleanupCommand.sh
dennis-behm Mar 25, 2025
ee7aa4e
Apply suggestions from code review
dennis-behm Mar 26, 2025
0dbcf11
Minor formatting updates
dennis-behm Mar 26, 2025
5532c1b
CBS script enhancements to manage packages lifecycle and ability to f…
dennis-behm Apr 7, 2025
087bf09
Merge remote-tracking branch 'upstream/main' into feature/306-delete-…
dennis-behm Apr 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions Templates/Common-Backend-Scripts/generateCleanupCommands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,12 @@ Help() {
}

#
# Customization
# Configuration file leveraged by the backend scripts
# Either an absolute path or a relative path to the current working directory
SCRIPT_HOME="$(dirname "$0")"
pipelineConfiguration="${SCRIPT_HOME}/pipelineBackend.config"
buildUtilities="${SCRIPT_HOME}/utilities/dbbBuildUtils.sh"
# Customization - End
deleteScript="${SCRIPT_HOME}/../../Utilities/DeletePDS/DeletePDS.groovy"

#
# Internal Variables
Expand Down Expand Up @@ -319,7 +318,7 @@ genDeleteStatementsCollections() {

for applicationCollection in ${collectionsToBeDeleted[@]}; do

echo "dbb collection delete $applicationCollection ${dbbMetadataStoreOptions}" >>${cmdFileDeleteCollections}
echo "dbb collection delete $applicationCollection ${cleanupDbbMetadataStoreOptions}" >>${cmdFileDeleteCollections}

done

Expand All @@ -341,7 +340,7 @@ genDeleteStatementsBuildGroups() {

for buildGroup in ${buildgroupsToBeDeleted[@]}; do

echo "dbb build-group delete $buildGroup ${dbbMetadataStoreOptions}" >>${cmdFileDeleteBuildGroups}
echo "dbb build-group delete $buildGroup ${cleanupDbbMetadataStoreOptions}" >>${cmdFileDeleteBuildGroups}

done

Expand Down Expand Up @@ -381,7 +380,7 @@ if [ $rc -eq 0 ]; then
echo $PGM": [INFO] ** Cmd obsolete collections:" ${cmdFileDeleteCollections}
echo $PGM": [INFO] ** Cmd obsolete build groups:" ${cmdFileDeleteBuildGroups}
echo $PGM": [INFO] ** Cmd obsolete build datasets:" ${cmdFileDeleteBuildDatasets}
echo $PGM": [INFO] ** DBB Metadastore Config:" ${dbbMetadataStoreOptions}
echo $PGM": [INFO] ** DBB Metadastore Config:" ${cleanupDbbMetadataStoreOptions}
echo $PGM": [INFO] ** Process Cleanup Scripts:" ${executeCleanupCommandScripts}
echo $PGM": [INFO] **************************************************************"
echo ""
Expand All @@ -390,7 +389,7 @@ fi
if [ $rc -eq 0 ]; then
# Retrieve existing DBB Collections
echo $PGM": [STAGE] Retrieve all collections with application qualifier $App"
applicationCollections=$(dbb collection list $dbbMetadataStoreOptions | grep $App)
applicationCollections=$(dbb collection list $cleanupDbbMetadataStoreOptions | grep $App)
rc=$?
if [ ! $rc -eq 0 ]; then
ERRMSG=$PGM": [ERROR] Retrieving Collections failed. Check Log. rc="$rc
Expand Down
20 changes: 20 additions & 0 deletions Templates/Common-Backend-Scripts/pipelineBackend.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ dbbMetadataStoreJdbcId=""
# Default = None, Required.
dbbMetadataStoreJdbcPwdFile=""

# Absolute path to Db2 conf file containing db2 schema etc
# Default = None, currently only referenced in generateCleanupCommands.sh
dbbMetadataStoreJdbcConfigFile=""

# JDBC connection server url
# sample jdbc:db2://10.3.20.201:4740/MOPDBC0
# Optional if zAppBuild is configured to use the
Expand Down Expand Up @@ -320,6 +324,22 @@ wdDeployPackageDir() {
## End of wazideploy-generate.sh, wazideploy-deploy.sh and wazideploy-evidences.sh parameters ####
#####################################################################################################

#####################################################################################################
## generateCleanupCommands.sh parameters ############################################################
#####################################################################################################

# DBB Metadatastore Options String to configure Db2 or File Metadatastore in cleanup commands
# e.g.
# for file metadatastore
# cleanupDbbMetadataStoreOptions="--type file --location /u/github/"
# for db2 metadatastore, reusing build settings
# cleanupDbbMetadataStoreOptions="--type db2 --user ${dbbMetadataStoreJdbcId} --password-file ${dbbMetadataStoreJdbcPwdFile} --db2-config ${dbbMetadataStoreJdbcConfigFile} --url ${dbbMetadataStoreJdbcUrl}"
cleanupDbbMetadataStoreOptions=""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not introduce a new variable for this cleanup, and compute the string in the generateCleanupCommands.sh based on the information that already present in the pipelineBackend.config file (dbbMetadataStoreJdbcId, dbbMetadataStoreJdbcPwdFile, dbbMetadataStoreJdbcConfigFile and dbbMetadataStoreJdbcUrl).

By the way, there should be a dbbMetadataStoreType (file|db2) parm and a dbbMetadataStoreFileLocation parm as well in pipelineBackend.config, to help compute the right connection string.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @M-DLB ,

By the way, there should be a dbbMetadataStoreType (file|db2) parm and a dbbMetadataStoreFileLocation parm as well in pipelineBackend.config, to help compute the right connection string.

I understand the point here - however, I don't want to maintain two variables with the same purpose - one in the build framework and another time in the pipelineBackend.config maintaining for maintaining the dbb metadatastore type.

Could we leave it as is now? Users need to customise the pipelineBackend.config anyway, and there are clear samples provided.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You already have variables for metadatstore duplicated in the build framework and in the pipelineBackend.config file. I think it would be cleaner to add the two that are missing (dbbMetadataStoreType (file|db2) parm and a dbbMetadataStoreFileLocation parm) to completeness, remove the cleanupDbbMetadataStoreOptions parm and compute it instead.
You should aim for consistency, instead of doing one way with DB2 and another way with file metadatastore.


#####################################################################################################
## End of generateCleanupCommands.sh parameters ####
#####################################################################################################

#####################################################################################################
## Central functions shared across scripts #######################################################
#####################################################################################################
Expand Down