Add scorer based on active loras

.tekton/README.md

@@ -1,6 +1,10 @@
 ## 🛠️ CI/CD Pipeline Overview – Your Project
 
-This pipeline is designed to support safe, efficient, and traceable development and deployment workflows using OpenShift Pipelines-as-Code, GitHub, and Quay.io.
+<!-- NOTE TO CONTRIBUTORS: every repo in the hc4ai organization is intended to have the same contents in this file. The origin is the copy in https://github.ibm.com/mspreitz/hc4ai-hello-neural/blob/dev/.tekton/README.md; submit PRs against that one -->
+
+This pipeline is designed to support safe, efficient, and traceable development and deployment workflows using [OpenShift Pipelines-as-Code](https://pipelinesascode.com/), [Tekton](https://tekton.dev/), [buildah](https://buildah.io/), GitHub, and Quay.io.
+
+This pipeline is used for CI/CD of the `dev` and `main` branches. This pipeline runs from source through container image build to deployment and testing in the hc4ai cluster.
 
 ---
 
@@ -24,19 +28,28 @@ Each repo includes a `.version.json` file at its root. This file controls:
 
 #### 🔑 Fields:
 - **dev-version**: Current version of the dev branch. Used to tag dev images.
-- **dev-registry**: Container registry location for development image pushes.
+- **dev-registry**: Container repository location for development image pushes.
 - **prod-version**: Managed by automation. Updated during promotion to match the dev-version.
-- **prod-registry**: Container registry for production image pushes. The promoted dev image is re-tagged and pushed here.
+- **prod-registry**: Container repository for production image pushes. The promoted dev image is re-tagged and pushed here.
 
 The pipeline reads this file to:
 - Extract the appropriate version tag
-- Determine the correct registry for image pushes
+- Determine the correct repository for image pushes
 - Promote and tag dev images for prod
 
 ---
 
+### Container Repositories
+
+This pipeline maintains two container repositories for this GitHub repository, as follows.
+
+- `quay.io/vllm-d/<repoName>-dev`. Hold builds from the `dev` branch as described below.
+- `quay.io/vllm-d/<repoName>`. Holds promotions to prod, as described below.
+
+---
+
 ### ⚙️ Pipeline Triggers
-Triggered on `push` and `pull_request` events targeting the `dev` or `main` branches.
+Triggered on `push` and `pull_request` events targeting the `dev` or `main` branches. The following workflows are the two behaviors of this pipeline.
 
 ### 🔧 dev Branch Workflow
 1. Checkout repository
@@ -47,20 +60,20 @@ Triggered on `push` and `pull_request` events targeting the `dev` or `main` bran
     - prod-version
     - prod-registry
 4. Build and push container image to:
-   → `<dev-registry>:<dev-version>`
+   → `<dev-repository>:<dev-version>`
 5. Tag the Git commit using the `dev-version`
-6. Optionally redeploy objects to OpenShift in `hc4ai-operator-dev`
+6. Optionally redeploy objects to OpenShift in the `hc4ai-operator-dev` namespace.
 
 ✅ This process ensures that all code merged into dev is validated and deployed for testing.
 
 ### 🚀 main Branch Workflow
 1. Checkout, lint, test, and parse `.version.json`
 2. Skip image rebuild
 3. Promote image by copying from:
-   → `<dev-registry:<dev-version>` → `<prod-registry>:<prod-version>`
+   → `<dev-repository:<dev-version>` → `<prod-repository>:<prod-version>`
 4. Tag the Git commit using the `prod-version`
 5. Update the upstream repo’s submodule to reference the new tag
-6. Redeploy to OpenShift in `hc4ai-operator`
+6. Redeploy to OpenShift in the `hc4ai-operator` namespace.
 
 ✅ No image rebuilds occur on main. Only validated dev images are promoted, ensuring reproducibility.
 
@@ -84,8 +97,8 @@ Tags are created using the configured Git credentials and pushed to the remote r
 
 ### ☸️ OpenShift Deployment
 The pipeline includes automated deployment:
-- On `dev`: Deploys to `hc4ai-operator-dev`
-- On `main`: Deploys to `hc4ai-operator`
+- On `dev`: Deploys to the `hc4ai-operator-dev` namespace. The Pod is named `<repoName>-major-minor`, using the `dev-version` from `.version.json`.
+- On `main`: Deploys to `hc4ai-operator` namespace. The Pod is named `<repoName>-major-minor`, using the `prod-version` from `.version.json`.
 
 Using `make uninstall-openshift` and `make install-openshift`, resources are cleanly reset.
 
@@ -112,6 +125,7 @@ After deployment, the pipeline:
 
 ### 🧠 Why `.version.json` Matters
 - Decouples versioning from Git commit hashes
-- Provides a single source of truth for version and registry info
+- Provides a single source of truth for version and repository info
 - Enables deterministic builds and controlled releases
 - Simplifies debugging and auditing across environments
+

.tekton/benchmark.yaml

@@ -0,0 +1,43 @@
+apiVersion: tekton.dev/v1
+kind: Task
+metadata:
+  name: benchmark-task
+spec:
+  params:
+    - name: openshift_host
+      description: "The OpenShift API server URL"
+      type: string
+    - name: openshift_namespace
+      description: "The OpenShift namespace to use"
+      type: string
+  steps:
+    - name: clone-and-install-fmperf
+      image: continuumio/miniconda3:latest
+      script: |
+        #!/bin/bash
+        set -ex
+
+        # Initialize conda (this sets up the environment for conda commands)
+        source /opt/conda/etc/profile.d/conda.sh
+
+        echo "Cloning fmperf repository..."
+        git clone https://github.com/fmperf-project/fmperf.git
+        cd fmperf
+
+        echo "Creating conda environment 'fmperf-env' with Python 3.11..."
+        conda create -y -n fmperf-env python=3.11
+
+        echo "Activating the conda environment..."
+        conda activate fmperf-env
+
+        echo "Installing required dependencies..."
+        pip install -r requirements.txt
+        pip install -e .
+
+        echo "Setting up environment variables for OpenShift connection..."
+        export OPENSHIFT_HOST="$(params.openshift_host)"
+        export OPENSHIFT_TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
+        export OPENSHIFT_NAMESPACE="$(params.openshift_namespace)"
+
+        echo "Running fmperf benchmark..."
+        python examples/example_vllm.py || true

.tekton/buildah-build.yaml

@@ -8,6 +8,9 @@ spec:
       description: "Application version"
     - name: image_tag_base
       description: "Image tag base"
+  results:
+    - name: image-url
+      description: "The full image URL including tag"
   workspaces:
     - name: source
     - name: registry
@@ -65,3 +68,4 @@ spec:
         echo "🚀 Calling make buildah-build with IMG=$IMG..."
         make buildah-build IMG=$IMG
 
+        echo "$IMG" > /tekton/results/image-url

.tekton/deploy-infra-to-openshift.yaml

@@ -0,0 +1,88 @@
+apiVersion: tekton.dev/v1
+kind: Task
+metadata:
+  name: openshift-redeploy-infra-task
+spec:
+  params:
+    - name: source-branch
+      type: string
+      description: "Git branch name"
+    - name: prod-version
+      type: string
+    - name: dev-version
+      type: string
+    - name: prod_image_tag_base
+      type: string
+    - name: dev_image_tag_base
+      type: string
+  workspaces:
+    - name: source
+  steps:
+  - name: redeploy
+    image: quay.io/projectquay/golang:1.24
+    imagePullPolicy: IfNotPresent
+    securityContext:
+      privileged: true
+    workingDir: $(workspaces.source.path)
+    env:
+      - name: STORAGE_DRIVER
+        value: vfs
+    script: |
+      #!/bin/bash
+      set -e
+
+      echo "📦 Installing dependencies with dnf..."
+      dnf install -y make jq curl gettext && dnf clean all
+
+      echo "📥 Installing kubectl..."
+      curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
+      install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
+
+      echo "📥 Installing kustomize..."
+      KUSTOMIZE_TAG=$(curl -s https://api.github.com/repos/kubernetes-sigs/kustomize/releases/latest | jq -r '.tag_name')
+      KUSTOMIZE_VERSION="${KUSTOMIZE_TAG##*/}"  # strips prefix like 'kustomize/' from tag
+
+      curl -LO "https://github.com/kubernetes-sigs/kustomize/releases/download/${KUSTOMIZE_TAG}/kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz"
+
+      tar -xzf "kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz" -C /usr/local/bin
+      chmod +x /usr/local/bin/kustomize
+      kustomize version
+
+      echo "🔧 Getting namespace and project_name from Makefile..."
+      DEFAULT_NAMESPACE=$(make -s print-namespace)
+      PROJECT_NAME=$(make -s print-project-name)
+
+      if [ "$(params.source-branch)" = "main" ]; then
+        NS="${DEFAULT_NAMESPACE}"
+        IMAGE_TAG_BASE=$(params.prod_image_tag_base)
+        VERSION=$(params.prod-version)
+      else
+        NS="${DEFAULT_NAMESPACE}-dev"
+        IMAGE_TAG_BASE=$(params.dev_image_tag_base)
+        VERSION=$(params.dev-version)
+      fi
+
+      echo "🔧 Using namespace: $NS"
+      echo "🔧 Using project_name: $PROJECT_NAME"
+      echo "🔧 Using image_tag_base: $IMAGE_TAG_BASE"
+      echo "🔧 Using version: $VERSION"
+
+      # echo "🧹 Uninstalling existing deployment..."
+      # # make uninstall-openshift NAMESPACE=$NS PROJECT_NAME=$PROJECT_NAME IMAGE_TAG_BASE=$IMAGE_TAG_BASE VERSION=$VERSION || echo "❗️ Failed to uninstall deployment"
+
+      # (make uninstall || echo "❗️ Failed to uninstall") && (make undeploy IMG=$IMAGE_TAG_BASE:$VERSION || echo "❗️ Failed to uninstall deployment")
+
+      # echo "⏳ Waiting 3 seconds before reinstall..."
+      # sleep 3
+
+      echo "🚀 Reinstalling OpenShift deployment..."
+      INFRASTRUCTURE_OVERRIDE=true make install-openshift-infrastructure
+
+      echo "⏳ Waiting 20 seconds before verifying resources..."
+      sleep 20
+
+      echo "🔍 Checking status of resources in namespace: $NS"
+      kubectl get pods -n $NS || echo "❗️ Failed to get pods"
+      kubectl get deploy -n $NS || echo "❗️ Failed to get deployments"
+      kubectl get svc -n $NS || echo "❗️ Failed to get services"
+      kubectl get routes -n $NS || echo "❗️ Failed to get routes"