Skip to content

✨ feat: add pre-release validation tests to release workflow - #268

Merged
codekiln merged 1 commit into
mainfrom
codekiln/207-pre-release-validation
Nov 24, 2025
Merged

✨ feat: add pre-release validation tests to release workflow#268
codekiln merged 1 commit into
mainfrom
codekiln/207-pre-release-validation

Conversation

@codekiln

Copy link
Copy Markdown
Owner

Summary

Add comprehensive pre-release validation job that runs full lifecycle deployment test before creating releases. This quality gate ensures the complete deployment workflow (create, update, delete) works correctly before publishing releases.

Changes

  • Added pre-release-validation job to .github/workflows/release.yml

    • Runs before create-release job as a quality gate
    • Executes test_deployment_workflow_full_lifecycle from SDK tests
    • Has 45-minute timeout (test takes ~20-30 minutes)
    • Requires secrets: LANGSMITH_API_KEY and LANGSMITH_WORKSPACE_ID
    • Blocks release if validation fails
  • Updated job dependencies

    • create-release now depends on both verify-ci and pre-release-validation
    • Ensures deployment lifecycle is validated before release creation
  • Updated CI/CD documentation (docs/dev/ci-cd.md)

    • Documented new pre-release validation step
    • Added details about timeout, duration, and required secrets

What the Test Validates

The full lifecycle test validates the complete deployment workflow:

  1. ✅ Creates fresh deployment with unique timestamp-based name
  2. ✅ Waits for deployment to be READY (~20-30 minutes)
  3. ✅ Patches deployment (triggers new revision)
  4. ✅ Waits for new revision to be DEPLOYED
  5. ✅ Deletes deployment (cleanup via DeploymentGuard RAII pattern)

Workflow Flow

Tag pushed (v*) 
    ↓
verify-ci (waits for CI jobs)
    ↓
pre-release-validation (20-30 min, quality gate)
    ↓
create-release (only if validation passes)
    ↓
build-release

Related Issues

Fixes #207

Part of epic #195 (new-release-ci milestone)

Test Plan

  • Review existing test implementation in sdk/tests/integration_deployment_workflow.rs
  • Verify workflow YAML syntax
  • Update CI/CD documentation
  • Confirm job dependencies are correct

Next Steps

Before release, verify:

  • LANGSMITH_API_KEY secret is configured in repository
  • LANGSMITH_WORKSPACE_ID secret is configured in repository
  • Test with pre-release tag (e.g., v0.4.4-alpha.1) to verify end-to-end

Benefits

  • 🛡️ Quality gate prevents broken releases
  • ✅ Validates complete deployment lifecycle
  • 🔍 Catches integration issues before release
  • 🧹 Ensures cleanup functionality works

🤖 Generated with Claude Code

Add comprehensive pre-release validation job that runs full lifecycle
deployment test before creating releases. This quality gate ensures
the complete deployment workflow (create, update, delete) works
correctly before publishing releases.

Changes:
- Add pre-release-validation job to release.yml
- Run test_deployment_workflow_full_lifecycle before release creation
- Configure 45-minute timeout (test takes ~20-30 minutes)
- Require LANGSMITH_API_KEY and LANGSMITH_WORKSPACE_ID secrets
- Block release if validation fails
- Update CI/CD documentation with new workflow step

The test validates:
- Creating fresh deployment with unique name
- Waiting for deployment to be READY
- Patching deployment (triggers new revision)
- Waiting for new revision to be DEPLOYED
- Deleting deployment (cleanup via DeploymentGuard)

Fixes #207

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings November 24, 2025 15:47
@codekiln codekiln added this to the new-release-ci milestone Nov 24, 2025

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a comprehensive pre-release validation step to the release workflow that runs a full deployment lifecycle test before creating releases. This quality gate ensures the SDK's deployment functionality (create, update, delete) works correctly before publishing any release.

  • Added a pre-release-validation job that runs a ~20-30 minute integration test
  • Updated job dependencies so releases only proceed after validation passes
  • Documented the new validation step in CI/CD documentation

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/release.yml Added pre-release-validation job with 45-minute timeout that runs full lifecycle deployment test; updated create-release job to depend on validation
docs/dev/ci-cd.md Documented new pre-release validation step including timeout, duration, required secrets, and its role as a quality gate

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

run: |
cargo test --test integration_deployment_workflow \
test_deployment_workflow_full_lifecycle -- \
--ignored --nocapture

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot what does --ignored --nocapture do here?

@codekiln
codekiln merged commit 6237e60 into main Nov 24, 2025
16 checks passed

Copilot AI commented Nov 24, 2025

Copy link
Copy Markdown
Contributor

@codekiln I've opened a new pull request, #269, to work on those changes. Once the pull request is ready, I'll request review from you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

195.6-pre-release-validation Add comprehensive pre-release validation tests

3 participants