diff --git a/README.md b/README.md index e4a6060..04624f7 100644 --- a/README.md +++ b/README.md @@ -7,13 +7,13 @@ Automated validation tool for testing DevWorkspace instances on OpenShift cluste ### Running Tests ```bash -# Basic validation (uses small numbers of entries - 3 images, 5 devfiles) +# Basic validation (uses small lists - images/images.txt and devfiles/devfiles.txt) ./dw-auto-validate.sh # Verbose mode - shows detailed output ./dw-auto-validate.sh -v -# Full test matrix (uses all entries from -full.txt files - all images, all devfiles. Takes significant time to complete!) +# Full test matrix (uses *-full.txt files - all images, all devfiles. Takes significant time to complete!) ./dw-auto-validate.sh -f # Debug mode - verbose + runs only first test + no cleanup @@ -58,7 +58,7 @@ Automated validation tool for testing DevWorkspace instances on OpenShift cluste - Iterates through devfiles × images matrix - For each combination: creates DevWorkspace, waits for Running state, validates, records results 5. **Cleanup**: Deletes DevWorkspace and temporary files (skipped in debug mode) -6. **Summary Report**: Shows test counts, success/failure, elapsed time, and lists failed tests +6. **Summary Report**: Shows test counts, success/failure/excluded, elapsed time, and lists failed tests ### Command-Line Flags @@ -81,38 +81,35 @@ Each scenario in `settings/settings-.env` exports: - `PROJECT_URL`: Git repository URL (must include surrounding double quotes) - `EDITOR_DEFINITION`: URL to the editor definition YAML - `EDITOR_COMPONENT_NAME`: Component name in the editor definition that contains the editor image (used by `-i`/`-p` to replace the correct image) -- `validate_devworkspace()`: Function that validates the running DevWorkspace +- `EXCLUDED_IMAGE_PATTERNS`: Array of Posix Extended Regular Expressions for images excluded from failure counts +- `LANDING_PAGE_PORT`: Port to curl inside the pod to validate the editor is running -#### Scenario Validation Methods +#### Scenario Validation -**sshd** (settings-sshd.env): -- Timeout: 60s -- Checks `/tmp/sshd.log` for "Server listening on" -- Verifies SSHD server started successfully +All scenarios use the same validation method: curl `localhost:${LANDING_PAGE_PORT}` inside the pod and check for HTTP 200. -**jetbrains** (settings-jetbrains.env): -- Timeout: 120s -- Port-forwards to 3400, curls `127.0.0.1:3400` -- Validates HTTP 200 response from JetBrains landing page -- On failure, outputs `/idea-server/std.out` for debugging - -**vscode** (settings-vscode.env): -- Timeout: 60s -- Checks `/checode/entrypoint-logs.txt` for "Extension host agent listening on 3100" -- Verifies VSCode extension host is listening +| Scenario | Timeout | Landing Page Port | Editor Component | +|----------|---------|-------------------|------------------| +| sshd | 60s | 3400 | che-code-sshd-page | +| jetbrains | 120s | 3400 | editor-injector | +| vscode | 90s | 3100 | che-code-injector | ### DevWorkspace Generation -Uses `devworkspace-template.yaml` as base, performs sed substitutions in two stages: +Uses `devworkspace-template.yaml` as base. The template uses ephemeral storage (`controller.devfile.io/storage-type: ephemeral`) to avoid PVC provisioning overhead during tests. + +Substitutions are performed in two stages: -**Stage 1** - Metadata and devfile injection: +**Stage 1** - Metadata, devfile, and projects injection: ```bash cat devworkspace-template.yaml | sed \ - -e "/DEVFILE/r ${TMP_DEVFILE}" \ # Inject devfile content - -e '/DEVFILE/ d' \ # Remove DEVFILE placeholder + -e "/DEVFILE/r ${TMP_DEVFILE}" \ # Inject devfile content + -e '/DEVFILE/ d' \ # Remove DEVFILE placeholder + -e "/PROJECTS/r ${TMP_PROJECTS}" \ # Inject projects block + -e '/PROJECTS/ d' \ # Remove PROJECTS placeholder -e "s|DEVWORKSPACE_NAME|...|" \ -e "s|DEVWORKSPACE_NS|...|" \ - -e "s|EDITOR_DEFINITION|...|" \ + -e "${EDITOR_SED_EXPR}" \ # Editor definition (uri or kubernetes ref) -e "s|PROJECT_URL|...|" ``` @@ -123,6 +120,18 @@ eval "sed \"s|image: .*|image: ${image}|\" > ${TMP_DEVWORKSPACE}" The two-stage approach ensures devfile content is injected before image replacement. +**Projects handling**: If the devfile contains `starterProjects`, those are extracted and converted into a `projects:` block. Otherwise, the scenario's `PROJECT_URL` is used as a fallback sample project. + +**Editor contribution**: When using `-p` or `-i` (override image), the editor contribution switches from `uri:` to `kubernetes: name:` referencing the applied DevWorkspaceTemplate. + +### DevWorkspace Lifecycle Management + +Between tests, the script handles the workspace depending on its current state: + +- **Running / Starting**: Gracefully stops by patching `spec.started: false` and waiting up to `TIMEOUT/4` seconds for `Stopped` state. +- **Failed**: Force-deletes the workspace (`oc delete dw`). A Failed workspace with CrashLoopBackOff containers can take too long to stop gracefully, which would cascade into subsequent test failures. The next `oc apply` recreates it cleanly. +- **Stopped / not found**: Proceeds directly to `oc apply`. + ### Logging and Output Control - `log()`: Outputs only when `VERBOSE=1` (set by `-v` or `-d` flags) @@ -140,51 +149,38 @@ Tracks test execution time using bash's `$SECONDS` variable: ``` settings/ - settings-sshd.env # SSHD scenario: timeout=60s, validates /tmp/sshd.log - settings-jetbrains.env # JetBrains scenario: timeout=120s, validates port 3400 - settings-vscode.env # VSCode scenario: timeout=60s, validates /checode/entrypoint-logs.txt + settings-sshd.env # SSHD scenario: timeout=60s, port 3400 + settings-jetbrains.env # JetBrains scenario: timeout=120s, port 3400 + settings-vscode.env # VSCode scenario: timeout=90s, port 3100 images/ - images.txt # Quick test list (3 UDI images: ubi8, ubi9, ubi10) - images-full.txt # Complete test matrix (UDI + base-developer-image variants) + images.txt # Default test list (3 UDI images: ubi8, ubi9, ubi10) + images-full.txt # Complete test matrix (227 images including UDI, base-developer-image, and UBI variants) devfiles/ - devfiles.txt # Quick test list (nodejs, go, python, php-laravel and java-quarkus devfile) + devfiles.txt # Default test list (nodejs, go, php-laravel, python, java-quarkus) devfiles-full.txt # Complete devfile list (32 devfiles from devfile registry) samples/ samples.txt # Sample project URLs (currently unused) samples-full.txt # Extended sample project list (currently unused) -devworkspace-template.yaml # Base template with placeholders +devworkspace-template.yaml # Base template with placeholders (ephemeral storage) dw-auto-validate.sh # Main validation orchestrator verify_images.sh # Skopeo-based image accessibility checker ``` ## Implementation Details -### Validation Function Pattern - -All `validate_devworkspace()` functions follow this pattern: - -```bash -validate_devworkspace() { - devfile_url=$1 # Receives devfile URL as first argument - - # Resolve pod and container via shared helper - resolve_devworkspace_pod || return 1 +### Validation Function - # Scenario-specific validation logic here - # Return 0 for pass, 1 for fail -} -``` +A single `validate_devworkspace()` function in the main script handles all scenarios: -The shared `resolve_devworkspace_pod()` function sets `podName` and `mainContainerName` globals. +1. Calls `resolve_devworkspace_pod()` to set `podName` and `mainContainerName` globals +2. Curls `localhost:${LANDING_PAGE_PORT}` inside the pod container +3. Returns 0 if HTTP 200, 1 otherwise -**Critical details**: -- Has access to `${DEVWORKSPACE_NS}`, `${DEVWORKSPACE_NAME}`, `log()` -- Must return 0 for success, 1 for failure -- Should use `&>/dev/null` on oc exec commands meant only for exit code checking +`resolve_devworkspace_pod()` finds the pod by DevWorkspace label and selects the main container from pod status, filtering out containers whose name starts with `che-`. ### Variable Quoting Requirements @@ -203,7 +199,7 @@ git: ```bash state="" count=0 -while [ "${state}" != "Running" ] && [ ${count} -lt ${TIMEOUT} ]; do +while [ "${state}" != "Running" ] && [ "${state}" != "Failed" ] && [ ${count} -lt ${TIMEOUT} ]; do state=$(oc get dw ${DEVWORKSPACE_NAME} -o 'jsonpath={.status.phase}') sleep 1s count=$((count+1)) @@ -212,18 +208,17 @@ done **Finding pod by DevWorkspace label**: ```bash -podNameAndDWName=$(oc get pods -o 'jsonpath={range .items[*]}{.metadata.name}{","}{.metadata.labels.controller\.devfile\.io/devworkspace_name}{end}') -podName=$(echo ${podNameAndDWName} | grep ${DEVWORKSPACE_NAME} | cut -d, -f1) +podNameAndDWName=$(oc get pods -o 'jsonpath={range .items[*]}{.metadata.name}{","}{.metadata.labels.controller\.devfile\.io/devworkspace_name}{"\n"}{end}') +podName=$(echo "${podNameAndDWName}" | grep ${DEVWORKSPACE_NAME} | cut -d, -f1) ``` -**Getting main container name**: +**Getting main container name** (from pod status, excluding `che-*` containers): ```bash -mainContainerName=$(oc get devworkspace ${DEVWORKSPACE_NAME} -o json | jq -r '[.spec.template.components[] | select(.container) | .name] | first') +mainContainerName=$(oc get pod "${podName}" -o json | jq -r '[.status.containerStatuses[] | select(.state.running and (.name | test("^che-") | not))] | first | .name // empty') ``` ### Adding a New Scenario 1. Create `settings/settings-.env` -2. Export required variables: `TIMEOUT`, `DEVWORKSPACE_NAME`, `PROJECT_URL`, `EDITOR_DEFINITION` -3. Implement `validate_devworkspace()` function that returns 0/1 -4. Update scenario selection in dw-auto-validate.sh (add option, update prompts) +2. Export required variables: `TIMEOUT`, `DEVWORKSPACE_NAME`, `PROJECT_URL`, `EDITOR_DEFINITION`, `EDITOR_COMPONENT_NAME`, `EXCLUDED_IMAGE_PATTERNS`, `LANDING_PAGE_PORT` +3. Update scenario selection in dw-auto-validate.sh (add option, update prompts and `-s` validation) diff --git a/devfiles/devfiles-full.txt b/devfiles/devfiles-full.txt index cdd242b..98f185d 100644 --- a/devfiles/devfiles-full.txt +++ b/devfiles/devfiles-full.txt @@ -1,5 +1,6 @@ https://registry.devfile.io/devfiles/dotnet80 https://registry.devfile.io/devfiles/go +https://registry.devfile.io/devfiles/code-with-quarkus https://registry.devfile.io/devfiles/java-maven https://registry.devfile.io/devfiles/java-openliberty https://registry.devfile.io/devfiles/java-openliberty-gradle @@ -20,13 +21,12 @@ https://registry.devfile.io/devfiles/nodejs-nuxtjs https://registry.devfile.io/devfiles/nodejs-react https://registry.devfile.io/devfiles/nodejs-svelte https://registry.devfile.io/devfiles/nodejs-vue -https://registry.devfile.io/devfiles/ollama https://registry.devfile.io/devfiles/php-laravel https://registry.devfile.io/devfiles/python https://registry.devfile.io/devfiles/python-django https://registry.devfile.io/devfiles/udi -https://registry.devfile.io/devfiles/nodejs-basic -https://registry.devfile.io/devfiles/code-with-quarkus -https://registry.devfile.io/devfiles/java-springboot-basic -https://registry.devfile.io/devfiles/python-basic -https://registry.devfile.io/devfiles/go-basic +https://registry.devfile.io/devfiles/hermes +https://registry.devfile.io/devfiles/ollama +https://registry.devfile.io/devfiles/openclaw +https://registry.devfile.io/devfiles/picoclaw +https://registry.devfile.io/devfiles/zeroclaw diff --git a/devfiles/devfiles.txt b/devfiles/devfiles.txt index a5beaa0..863eeee 100644 --- a/devfiles/devfiles.txt +++ b/devfiles/devfiles.txt @@ -2,3 +2,4 @@ https://registry.devfile.io/devfiles/nodejs https://registry.devfile.io/devfiles/go https://registry.devfile.io/devfiles/php-laravel https://registry.devfile.io/devfiles/python +https://registry.devfile.io/devfiles/java-quarkus diff --git a/devworkspace-template.yaml b/devworkspace-template.yaml index 4e15175..e389f2f 100644 --- a/devworkspace-template.yaml +++ b/devworkspace-template.yaml @@ -7,12 +7,10 @@ spec: started: true routingClass: che # thanks claude template: + attributes: + controller.devfile.io/storage-type: ephemeral DEVFILE - projects: - - name: project-sample - git: - remotes: - origin: PROJECT_URL + PROJECTS contributions: - name: editor-definition-component diff --git a/dw-auto-validate.sh b/dw-auto-validate.sh index e8d8ba3..dea7a8e 100755 --- a/dw-auto-validate.sh +++ b/dw-auto-validate.sh @@ -100,7 +100,7 @@ resolve_devworkspace_pod() { log "${YELLOW}podNameAndDWName: \n${NC}${podNameAndDWName}" podName=$(echo "${podNameAndDWName}" | grep ${DEVWORKSPACE_NAME} | cut -d, -f1) log "${YELLOW}podName: \n${NC}${podName}" - mainContainerName=$(oc get devworkspace ${DEVWORKSPACE_NAME} -o json | jq -r '[.spec.template.components[] | select(.container) | .name] | first') + mainContainerName=$(oc get pod "${podName}" -o json | jq -r '[.status.containerStatuses[] | select(.state.running and (.name | test("^che-") | not))] | first | .name // empty') log "${YELLOW}mainContainerName: \n${NC}${mainContainerName}" if [ -z "${podName}" ] || [ -z "${mainContainerName}" ]; then log "Could not find pod/container matching ${DEVWORKSPACE_NAME}" @@ -110,8 +110,22 @@ resolve_devworkspace_pod() { return 0 } +validate_devworkspace() { + resolve_devworkspace_pod || return 1 + + log "Checking editor on localhost:${LANDING_PAGE_PORT}" + http_code=$(oc exec -n "${DEVWORKSPACE_NS}" "${podName}" -c "${mainContainerName}" -- curl -s -o /dev/null -w '%{http_code}' http://localhost:${LANDING_PAGE_PORT}) + if [ "${http_code}" == "200" ]; then + log "${GREEN}localhost:${VALIDATION_PORT} returned HTTP ${http_code}${NC}" + return 0 + else + log "${YELLOW}localhost:${VALIDATION_PORT} returned HTTP ${http_code}${NC}" + return 1 + fi +} + shouldExclude() { - for imagePattern in ${EXCLUDED_IMAGE_PATTERNS[@]}; do + for imagePattern in "${EXCLUDED_IMAGE_PATTERNS[@]}"; do if [[ ${1} =~ ${imagePattern} ]]; then return 0 fi @@ -155,7 +169,7 @@ if [ -n "${PR_NUMBER}" ]; then echo -e "${GREEN}Ok!${NC}" echo -e "\n${BLUE}Checking PR image...${NC}" log "Executing 'skopeo inspect'..." - eval skopeo inspect --no-tags --retry-times 2 --override-arch amd64 --override-os linux "docker://${PR_IMAGE}" ${QUIET} + eval skopeo inspect --no-tags --retry-times 2 --override-arch amd64 --override-os linux "docker://${PR_IMAGE}" "${QUIET}" if [ $? -ne 0 ]; then echo -e "${RED}Error:${NC} PR image '${PR_IMAGE}' not found. Make sure the GitHub Action has published the image." >&2 exit 1 @@ -168,7 +182,7 @@ if [ -n "${CUSTOM_IMAGE}" ]; then if [ -x "$(command -v skopeo)" ]; then echo -e "\n${BLUE}Checking custom image...${NC}" log "Executing 'skopeo inspect'..." - eval skopeo inspect --no-tags --retry-times 2 --override-arch amd64 --override-os linux "docker://${CUSTOM_IMAGE}" ${QUIET} + eval skopeo inspect --no-tags --retry-times 2 --override-arch amd64 --override-os linux "docker://${CUSTOM_IMAGE}" "${QUIET}" if [ $? -ne 0 ]; then echo -e "${YELLOW}Warning:${NC} Could not verify custom image '${CUSTOM_IMAGE}'. Proceeding anyway." else @@ -188,11 +202,11 @@ fi echo -e "\n${BLUE}Checking cluster connection...${NC}" log "Executing 'oc whoami'..." current_cluster=$(oc config current-context) -eval oc whoami --insecure-skip-tls-verify ${QUIET} +eval oc whoami --insecure-skip-tls-verify "${QUIET}" if [ $? -eq 1 ]; then echo -e "${YELLOW}Not connected.${NC} Do you want to login to current cluster? Current cluster is ${PURPLE}${current_cluster}.${NC}" while true; do - read -p "(y/n)? : " yn + read -rp "(y/n)? : " yn case $yn in [Yy]* ) oc login --web; break;; [Nn]* ) exit;; @@ -207,7 +221,7 @@ fi if [ -z "${SCENARIO}" ]; then echo -e "\n${BLUE}Choose the dedicated scenario to run the validation test suite.${NC}\n1-sshd\n2-jetbrains\n3-vscode" while true; do - read -p "(1/2/3)? : " scenario + read -rp "(1/2/3)? : " scenario case $scenario in 1 ) SCENARIO=sshd; break;; 2 ) SCENARIO=jetbrains; break;; @@ -218,7 +232,8 @@ if [ -z "${SCENARIO}" ]; then fi # Read values from scenario's setting -. settings/settings-${SCENARIO}.env +# shellcheck source=settings/settings-vscode.env +. settings/settings-"${SCENARIO}".env # user namespace where testing will occur DEVWORKSPACE_NS=$(oc project -q) @@ -259,7 +274,7 @@ DWTEOF fi # Temporary storage for generated files -TMP_DEVFILE=$(mktemp -t devfile-${SCENARIO}-XXX.yaml) +TMP_DEVFILE=$(mktemp -t devfile-"${SCENARIO}"-XXX.yaml) TMP_DEVWORKSPACE=$(mktemp -t devworkspace-XXX.yaml) # parsing images list @@ -319,8 +334,33 @@ fi echo -e "${BLUE}There will be ${total_tests} tests performed in total.${NC}" for devfile_url in "${DEVFILE_URL_LIST[@]}"; do - curl -sL -o ${TMP_DEVFILE} ${devfile_url} - sed -i.tmp 's/^/ /' ${TMP_DEVFILE} && rm -f "${TMP_DEVFILE}.tmp" + http_code=$(curl -sL -o "${TMP_DEVFILE}" -w '%{http_code}' "${devfile_url}") + if [ "${http_code}" != "200" ]; then + echo "${devfile_url} — fetch failed (HTTP ${http_code}), skipping this devfile. Numbers might not be accurate. ❌" + failed_test+=("Devfile '${devfile_url}' — fetch failed (HTTP ${http_code})") + continue + fi + + # Build the projects block: use the devfile's starterProjects if present, + # otherwise fall back to the default sample project. + # Must check before indenting the devfile. + TMP_PROJECTS=$(mktemp -t projects-XXX.yaml) + if grep -q '^starterProjects:' "${TMP_DEVFILE}"; then + sed -n '/^starterProjects:/,/^[a-zA-Z]/{/^starterProjects:/p; /^ /p}' "${TMP_DEVFILE}" | \ + sed 's/^starterProjects:/projects:/' | \ + sed 's/^/ /' > "${TMP_PROJECTS}" + + else + cat > "${TMP_PROJECTS}" <<'PROJEOF' + projects: + - name: project-sample + git: + remotes: + origin: PROJECT_URL +PROJEOF + fi + + sed -i.tmp 's/^/ /' "${TMP_DEVFILE}" && rm -f "${TMP_DEVFILE}.tmp" for image in "${IMAGES_LIST[@]}"; do #debug mode: stop after one iteration @@ -346,6 +386,8 @@ for devfile_url in "${DEVFILE_URL_LIST[@]}"; do sed \ -e "/DEVFILE/r ${TMP_DEVFILE}" \ -e '/DEVFILE/ d' \ + -e "/PROJECTS/r ${TMP_PROJECTS}" \ + -e '/PROJECTS/ d' \ -e "s|DEVWORKSPACE_NAME|${DEVWORKSPACE_NAME}|" \ -e "s|DEVWORKSPACE_NS|${DEVWORKSPACE_NS}|" \ -e "${EDITOR_SED_EXPR}" \ @@ -353,21 +395,49 @@ for devfile_url in "${DEVFILE_URL_LIST[@]}"; do # Modify the result (must be separate) # here is the replacement of the container image used in the devfile from an image in the list eval "sed \"s|image: .*|image: ${image}|\" > ${TMP_DEVWORKSPACE}" + # Stop the DevWorkspace before applying to force a pod restart + current_phase=$(oc get dw ${DEVWORKSPACE_NAME} -o 'jsonpath={.status.phase}' 2>/dev/null) + if [ "${current_phase}" == "Failed" ]; then + log -n "Force-deleting ${DEVWORKSPACE_NAME} (Failed state) ." + eval "oc delete dw ${DEVWORKSPACE_NAME} ${QUIET}" + log " deleted." + elif [ "${current_phase}" == "Running" ] || [ "${current_phase}" == "Starting" ]; then + eval "oc patch dw ${DEVWORKSPACE_NAME} --type merge -p '{\"spec\":{\"started\":false}}' ${QUIET}" + log -n "Stopping ${DEVWORKSPACE_NAME} ." + stop_count=0 + stop_timeout=$((TIMEOUT / 4)) + while [ "$(oc get dw ${DEVWORKSPACE_NAME} -o 'jsonpath={.status.phase}' 2>/dev/null)" != "Stopped" ] && [ ${stop_count} -lt ${stop_timeout} ]; do + sleep 1s + log -n "." + stop_count=$((stop_count+1)) + done + if [ ${stop_count} -ge ${stop_timeout} ]; then + log "\n${YELLOW}${DEVWORKSPACE_NAME} failed to stop (timed out after ${stop_timeout}s)${NC}" + echo "TEST [${total_count}/${total_tests}] ${devfile_url} with ${image} FAILED ❌" + failed_test+=("Devfile '$devfile_url' using image '$image'") + continue + fi + log " stopped." + fi eval "oc apply -f ${TMP_DEVWORKSPACE} ${QUIET}" state="" - log -n "Waiting for ${DEVWORKSPACE_NAME} .." + log -n "Waiting for ${DEVWORKSPACE_NAME} to run ." count=0 - while [ "${state}" != "Running" ] && [ ${count} -lt ${TIMEOUT} ]; do + while [ "${state}" != "Running" ] && [ "${state}" != "Failed" ] && [ ${count} -lt ${TIMEOUT} ]; do state=$(oc get dw ${DEVWORKSPACE_NAME} -o 'jsonpath={.status.phase}') sleep 1s log -n "." count=$((count+1)) done - if [ ${state} == "Running" ]; then - log "\n${GREEN}${DEVWORKSPACE_NAME} is Running${NC}" + if [ "${state}" == "Running" ]; then + log "\n${GREEN}${DEVWORKSPACE_NAME} is running.${NC}" else - log "\n${YELLOW}${DEVWORKSPACE_NAME} failed to start${NC}" - if shouldExclude ${image}; then + if [ "${state}" == "Failed" ]; then + log "\n${YELLOW}${DEVWORKSPACE_NAME} failed to start (state: Failed after ${count}s)${NC}" + else + log "\n${YELLOW}${DEVWORKSPACE_NAME} failed to start (timed out after ${TIMEOUT}s, last state: ${state})${NC}" + fi + if shouldExclude "${image}"; then echo "TEST [${total_count}/${total_tests}] ${devfile_url} with ${image} FAILED ❌ (EXCLUDED ↩️ )" excluded_test+=("Devfile '$devfile_url' using image '$image'") else @@ -377,12 +447,12 @@ for devfile_url in "${DEVFILE_URL_LIST[@]}"; do continue fi log "Validating ${DEVWORKSPACE_NAME} .." - validate_devworkspace ${devfile_url} + validate_devworkspace if [ $? -eq 0 ]; then echo "TEST [${total_count}/${total_tests}] ${devfile_url} with ${image} PASSED ✅" ((success_count++)) else - if shouldExclude ${image}; then + if shouldExclude "${image}"; then echo "TEST [${total_count}/${total_tests}] ${devfile_url} with ${image} FAILED ❌ (EXCLUDED ↩️ )" excluded_test+=("Devfile '$devfile_url' using image '$image'") else @@ -392,7 +462,7 @@ for devfile_url in "${DEVFILE_URL_LIST[@]}"; do fi sleep 1s done # image loop - + [[ ${DEBUG} -eq 1 && ${total_count} -ge 1 ]] && break done # devfile loop # cleanup @@ -404,11 +474,12 @@ cleanup() { fi sleep 1s - rm $TMP_DEVFILE - rm $TMP_DEVWORKSPACE + rm "$TMP_DEVFILE" + rm "$TMP_PROJECTS" + rm "$TMP_DEVWORKSPACE" if [ -n "${OVERRIDE_IMAGE}" ]; then - rm $TMP_EDITOR_DEF - rm $TMP_DWT + rm "$TMP_EDITOR_DEF" + rm "$TMP_DWT" fi } diff --git a/images/images-full.txt b/images/images-full.txt index 3c6bd0d..25e967e 100644 --- a/images/images-full.txt +++ b/images/images-full.txt @@ -133,6 +133,49 @@ registry.access.redhat.com/ubi9/toolbox:9.7 registry.access.redhat.com/ubi9/s2i-base:9.7 registry.access.redhat.com/ubi9/s2i-core:9.7 +registry.access.redhat.com/ubi9:9.8 +registry.access.redhat.com/ubi9-minimal:9.8 +registry.access.redhat.com/ubi9-micro:9.8 +registry.access.redhat.com/ubi9-init:9.8 +registry.access.redhat.com/ubi9/ubi-stig:9.8 +registry.access.redhat.com/ubi9/python-39:9.8 +registry.access.redhat.com/ubi9/python-311:9.8 +registry.access.redhat.com/ubi9/python-312:9.8 +registry.access.redhat.com/ubi9/python-312-minimal:9.8 +registry.access.redhat.com/ubi9/python-314:9.8 +registry.access.redhat.com/ubi9/python-314-minimal:9.8 +registry.access.redhat.com/ubi9/nodejs-22:9.8 +registry.access.redhat.com/ubi9/nodejs-22-minimal:9.8 +registry.access.redhat.com/ubi9/nodejs-24:9.8 +registry.access.redhat.com/ubi9/nodejs-24-minimal:9.8 +registry.access.redhat.com/ubi9/ruby-30:9.8 +registry.access.redhat.com/ubi9/ruby-33:9.8 +registry.access.redhat.com/ubi9/ruby-40:9.8 +registry.access.redhat.com/ubi9/php-80:9.8 +registry.access.redhat.com/ubi9/php-82:9.8 +registry.access.redhat.com/ubi9/php-83:9.8 +registry.access.redhat.com/ubi9/perl-532:9.8 +registry.access.redhat.com/ubi9/openjdk-17:1.24 +registry.access.redhat.com/ubi9/openjdk-17-runtime:1.24 +registry.access.redhat.com/ubi9/openjdk-21:1.24 +registry.access.redhat.com/ubi9/openjdk-21-runtime:1.24 +registry.access.redhat.com/ubi9/openjdk-25:1.24 +registry.access.redhat.com/ubi9/openjdk-25-runtime:1.24 +registry.access.redhat.com/ubi9/dotnet-100:9.8 +registry.access.redhat.com/ubi9/dotnet-100-runtime:9.8 +registry.access.redhat.com/ubi9/dotnet-100-aspnet:9.8 +registry.access.redhat.com/ubi9/go-toolset:9.8 +registry.access.redhat.com/ubi9/nginx-120:9.8 +registry.access.redhat.com/ubi9/nginx-124:9.8 +registry.access.redhat.com/ubi9/nginx-126:9.8 +registry.access.redhat.com/ubi9/httpd-24:9.8 +registry.access.redhat.com/ubi9/buildah:9.8 +registry.access.redhat.com/ubi9/skopeo:9.8 +registry.access.redhat.com/ubi9/podman:9.8 +registry.access.redhat.com/ubi9/toolbox:9.8 +registry.access.redhat.com/ubi9/s2i-base:9.8 +registry.access.redhat.com/ubi9/s2i-core:9.8 + registry.access.redhat.com/ubi10:10.0 registry.access.redhat.com/ubi10/ubi-minimal:10.0 registry.access.redhat.com/ubi10/ubi-micro:10.0 @@ -150,6 +193,10 @@ registry.access.redhat.com/ubi10/buildah:10.0 registry.access.redhat.com/ubi10/toolbox:10.0 registry.access.redhat.com/ubi10/podman:10.0 registry.access.redhat.com/ubi10/skopeo:10.0 +registry.access.redhat.com/ubi10/openjdk-21:1.24 +registry.access.redhat.com/ubi10/openjdk-21-runtime:1.24 +registry.access.redhat.com/ubi10/openjdk-25:1.24 +registry.access.redhat.com/ubi10/openjdk-25-runtime:1.24 registry.access.redhat.com/ubi10/s2i-base:10.0 registry.access.redhat.com/ubi10/s2i-core:10.0 @@ -172,5 +219,9 @@ registry.access.redhat.com/ubi10/buildah:10.1 registry.access.redhat.com/ubi10/toolbox:10.1 registry.access.redhat.com/ubi10/podman:10.1 registry.access.redhat.com/ubi10/skopeo:10.1 +registry.access.redhat.com/ubi10/openjdk-21:1.24 +registry.access.redhat.com/ubi10/openjdk-21-runtime:1.24 +registry.access.redhat.com/ubi10/openjdk-25:1.24 +registry.access.redhat.com/ubi10/openjdk-25-runtime:1.24 registry.access.redhat.com/ubi10/s2i-base:10.1 registry.access.redhat.com/ubi10/s2i-core:10.1 diff --git a/samples/samples-full.txt b/samples/samples-full.txt index 8d13ecf..9d94786 100644 --- a/samples/samples-full.txt +++ b/samples/samples-full.txt @@ -1,5 +1,3 @@ -https://github.com/che-samples/web-nodejs-sample -https://github.com/che-samples/nodejs-mongodb-sample https://github.com/devfile-samples/devfile-stack-go https://github.com/devfile-samples/devfile-stack-nodejs-angular https://github.com/devfile-samples/devfile-stack-nodejs-nextjs @@ -8,6 +6,15 @@ https://github.com/devfile-samples/devfile-stack-nodejs-svelte https://github.com/devfile-samples/devfile-stack-nodejs-vue https://github.com/devfile-samples/devfile-stack-php-laravel https://github.com/devfile-samples/devfile-stacks-nodejs-react +https://github.com/devfile-samples/devfile-sample-code-with-quarkus +https://github.com/devfile-samples/devfile-sample-dotnet60-basic +https://github.com/devfile-samples/devfile-sample-go-basic +https://github.com/devfile-samples/devfile-sample-java-springboot-basic +https://github.com/devfile-samples/devfile-sample-python-basic +https://github.com/devfile-samples/node-express-hello-devfile-no-dockerfile +https://github.com/devfile-samples/nodejs-ex +https://github.com/devfile-samples/nodejs-mongodb-sample +https://github.com/devfile-samples/web-coolstore https://github.com/devfile-samples/python-django-ex https://github.com/devfile-samples/python-ex https://github.com/devfile-samples/springboot-ex @@ -37,3 +44,4 @@ https://github.com/wildfly-extras/wildfly-devfile-examples https://github.com/wildfly/quickstart https://github.com/RedHat-Middleware-Workshops/devsandbox-camel https://github.com/che-incubator/quarkus-api-example + diff --git a/samples/samples.txt b/samples/samples.txt index 12c09b5..5896abf 100644 --- a/samples/samples.txt +++ b/samples/samples.txt @@ -1 +1 @@ -https://github.com/che-samples/web-nodejs-sample \ No newline at end of file +https://github.com/devfile-samples/nodejs-ex \ No newline at end of file diff --git a/settings/settings-jetbrains.env b/settings/settings-jetbrains.env index a3e8c17..5b7181a 100644 --- a/settings/settings-jetbrains.env +++ b/settings/settings-jetbrains.env @@ -18,44 +18,5 @@ export EDITOR_COMPONENT_NAME='editor-injector' # array of image patterns (Posix Extended Regular Expressions) excluded from failed tests export EXCLUDED_IMAGE_PATTERNS=('ubi[0-9]{0,2}-micro' 'ubi[0-9]{0,2}-stig' 'ubi[0-9]{0,2}/pause') -# Function that evaluates whether DevWorkspace is valid -validate_devworkspace() { - devfile_url=$1 - LANDING_PAGE_PORT=3400 - - resolve_devworkspace_pod || return 1 - - log "Waiting for response .." - # Access the landing page - if [ ${VERBOSE} -eq 0 ]; then - oc port-forward -n ${DEVWORKSPACE_NS} ${podName} ${LANDING_PAGE_PORT}:${LANDING_PAGE_PORT} &>/dev/null & - else - oc port-forward -n ${DEVWORKSPACE_NS} ${podName} ${LANDING_PAGE_PORT}:${LANDING_PAGE_PORT} & - fi - PORT_FORWARD_PID=$! - - response="000" - count=0 - while [ "${response}" != "200" ] && [ ${count} -lt ${TIMEOUT} ]; do - sleep 1s - response=$(curl -s -o /dev/null -w "%{http_code}" "127.0.0.1:${LANDING_PAGE_PORT}") - count=$((count+1)) - [ "${response}" != "000" ] && break - log -n "." - done - log "response: ${response}" - - # Terminate oc-port-forward - kill ${PORT_FORWARD_PID} 2>/dev/null - wait ${PORT_FORWARD_PID} 2>/dev/null - sleep 1s - - if [ "${response}" == "200" ]; then - # pass - return 0 - else - # fail - log $(oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /idea-server/std.out) - return 1 - fi -} +# port to check for editor readiness +export LANDING_PAGE_PORT=3400 diff --git a/settings/settings-sshd.env b/settings/settings-sshd.env index 602af49..d27c69d 100644 --- a/settings/settings-sshd.env +++ b/settings/settings-sshd.env @@ -18,25 +18,5 @@ export EDITOR_COMPONENT_NAME='che-code-sshd-page' # array of image patterns (Posix Extended Regular Expressions) excluded from failed tests export EXCLUDED_IMAGE_PATTERNS=('ubi[0-9]{0,2}-micro' 'ubi[0-9]{0,2}-stig' 'ubi[0-9]{0,2}/pause') -# Function that evaluates whether DevWorkspace is valid -validate_devworkspace() { - devfile_url=$1 - - resolve_devworkspace_pod || return 1 - - oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /tmp/sshd.log &>/dev/null #always quiet this message an rely on return code - if [ $? -eq 1 ]; then - # fail to cat ssh log file - log "Failed to cat /tmp/sshd.log file on container" - return 1 - fi - res=$(oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /tmp/sshd.log | grep -q 'Server listening on') - if [ $? -eq 0 ]; then - # pass - return 0 - else - # fail - log "Failed to retrieve sucessfull sshd status log in /tmp/sshd.log file on container" - return 1 - fi -} +# port to check for editor readiness +export LANDING_PAGE_PORT=3400 diff --git a/settings/settings-vscode.env b/settings/settings-vscode.env index 9b07d16..73e2507 100644 --- a/settings/settings-vscode.env +++ b/settings/settings-vscode.env @@ -1,7 +1,7 @@ #! /bin/bash # Rough time (seconds) to wait for DevWorkspace to enter 'Running' state -export TIMEOUT=60 +export TIMEOUT=90 # name to give all created (singleton) DevWorkspace instances export DEVWORKSPACE_NAME='vscode-test' @@ -18,26 +18,5 @@ export EDITOR_COMPONENT_NAME='che-code-injector' # array of image patterns (Posix Extended Regular Expressions) excluded from failed tests export EXCLUDED_IMAGE_PATTERNS=('ubi[0-9]{0,2}-micro' 'ubi[0-9]{0,2}-stig' 'ubi[0-9]{0,2}/pause') -# Function that evaluates whether DevWorkspace is valid -validate_devworkspace() { - devfile_url=$1 - - resolve_devworkspace_pod || return 1 - - oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /checode/entrypoint-logs.txt &>/dev/null #always quiet this message an rely on return code - if [ $? -eq 1 ]; then - # fail to cat vscode log file - log "Failed to cat /checode/entrypoint-logs.txt file on container" - return 1 - fi - res=$(oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /checode/entrypoint-logs.txt | grep -q 'Extension host agent listening on 3100') - if [ $? -eq 0 ]; then - # pass - return 0 - else - # fail - log "Failed to retrieve sucessfull vscode status log in /checode/entrypoint-logs.txt file on container" - log $(oc exec -n ${DEVWORKSPACE_NS} ${podName} -c ${mainContainerName} -- cat /checode/entrypoint-logs.txt) - return 1 - fi -} +# port to check for editor readiness +export LANDING_PAGE_PORT=3100 diff --git a/verify_images.sh b/verify_images.sh index 019fc9b..c485b13 100755 --- a/verify_images.sh +++ b/verify_images.sh @@ -2,13 +2,14 @@ failed_images=() success_count=0 total_count=0 +image_count=$(grep -c '[^[:space:]]' images/images-full.txt) while IFS= read -r image; do # Skip empty lines [[ -z "$image" ]] && continue ((total_count++)) - echo "Checking [$total_count]: $image" + echo "Checking [$total_count/$image_count]: $image" if skopeo inspect --no-tags --retry-times 2 "docker://$image" > /dev/null 2>&1; then ((success_count++))