feat(pipeline): allow variable substitution in pipeline.tasks[].onError

[l-qing]

fix #8564

In the reconcile logic of PipelineRun, add the ability to replace the pipeline.tasks[].onError field. At the same time, the validation of pipelineSpec legality has been delayed.

Changes

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
• Has Tests included if any functionality added or changed
• pre-commit Passed
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
• Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

feat(pipeline): allow variable substitution in pipeline.tasks[].onError

/kind feature

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign jeromeju after the PR has been reviewed.
You can assign the PR to them by writing /assign @jeromeju in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

[l-qing]

The replacement of spec.tasks[].conditions[].params[].value is a capability of v1alpha1, which no longer has that field.

The current complete definition of the pipelinetask structure is as follows:

pipeline/pkg/apis/pipeline/v1/pipeline_types.go

Lines 179 to 257 in aec465e

	// PipelineTask defines a task in a Pipeline, passing inputs from both
	// Params and from the output of previous tasks.
	type PipelineTask struct {
	// Name is the name of this task within the context of a Pipeline. Name is
	// used as a coordinate with the `from` and `runAfter` fields to establish
	// the execution order of tasks relative to one another.
	Name string `json:"name,omitempty"`
	// DisplayName is the display name of this task within the context of a Pipeline.
	// This display name may be used to populate a UI.
	// +optional
	DisplayName string `json:"displayName,omitempty"`
	// Description is the description of this task within the context of a Pipeline.
	// This description may be used to populate a UI.
	// +optional
	Description string `json:"description,omitempty"`
	// TaskRef is a reference to a task definition.
	// +optional
	TaskRef *TaskRef `json:"taskRef,omitempty"`
	// TaskSpec is a specification of a task
	// Specifying TaskSpec can be disabled by setting
	// `disable-inline-spec` feature flag..
	// +optional
	TaskSpec *EmbeddedTask `json:"taskSpec,omitempty"`
	// When is a list of when expressions that need to be true for the task to run
	// +optional
	When WhenExpressions `json:"when,omitempty"`
	// Retries represents how many times this task should be retried in case of task failure: ConditionSucceeded set to False
	// +optional
	Retries int `json:"retries,omitempty"`
	// RunAfter is the list of PipelineTask names that should be executed before
	// this Task executes. (Used to force a specific ordering in graph execution.)
	// +optional
	// +listType=atomic
	RunAfter []string `json:"runAfter,omitempty"`
	// Parameters declares parameters passed to this task.
	// +optional
	// +listType=atomic
	Params Params `json:"params,omitempty"`
	// Matrix declares parameters used to fan out this task.
	// +optional
	Matrix *Matrix `json:"matrix,omitempty"`
	// Workspaces maps workspaces from the pipeline spec to the workspaces
	// declared in the Task.
	// +optional
	// +listType=atomic
	Workspaces []WorkspacePipelineTaskBinding `json:"workspaces,omitempty"`
	// Time after which the TaskRun times out. Defaults to 1 hour.
	// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
	// +optional
	Timeout *metav1.Duration `json:"timeout,omitempty"`
	// PipelineRef is a reference to a pipeline definition
	// Note: PipelineRef is in preview mode and not yet supported
	// +optional
	PipelineRef *PipelineRef `json:"pipelineRef,omitempty"`
	// PipelineSpec is a specification of a pipeline
	// Note: PipelineSpec is in preview mode and not yet supported
	// Specifying PipelineSpec can be disabled by setting
	// `disable-inline-spec` feature flag..
	// +optional
	PipelineSpec *PipelineSpec `json:"pipelineSpec,omitempty"`
	// OnError defines the exiting behavior of a PipelineRun on error
	// can be set to [ continue | stopAndFail ]
	// +optional
	OnError PipelineTaskOnErrorType `json:"onError,omitempty"`
	}

The previous code for replacing variables in PipelineTask is as follows:

pipeline/pkg/reconciler/pipelinerun/resources/apply.go

Lines 301 to 352 in aec465e

	// ApplyReplacements replaces placeholders for declared parameters with the specified replacements.
	func ApplyReplacements(p *v1.PipelineSpec, replacements map[string]string, arrayReplacements map[string][]string, objectReplacements map[string]map[string]string) *v1.PipelineSpec {
	p = p.DeepCopy()
	for i := range p.Tasks {
	p.Tasks[i].Params = p.Tasks[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	if p.Tasks[i].IsMatrixed() {
	p.Tasks[i].Matrix.Params = p.Tasks[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)
	for j := range p.Tasks[i].Matrix.Include {
	p.Tasks[i].Matrix.Include[j].Params = p.Tasks[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)
	}
	} else {
	p.Tasks[i].DisplayName = substitution.ApplyReplacements(p.Tasks[i].DisplayName, replacements)
	}
	for j := range p.Tasks[i].Workspaces {
	p.Tasks[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Tasks[i].Workspaces[j].SubPath, replacements)
	}
	p.Tasks[i].When = p.Tasks[i].When.ReplaceVariables(replacements, arrayReplacements)
	if p.Tasks[i].TaskRef != nil {
	if p.Tasks[i].TaskRef.Params != nil {
	p.Tasks[i].TaskRef.Params = p.Tasks[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	}
	p.Tasks[i].TaskRef.Name = substitution.ApplyReplacements(p.Tasks[i].TaskRef.Name, replacements)
	}
	p.Tasks[i] = propagateParams(p.Tasks[i], replacements, arrayReplacements, objectReplacements)
	}
	for i := range p.Finally {
	p.Finally[i].Params = p.Finally[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	if p.Finally[i].IsMatrixed() {
	p.Finally[i].Matrix.Params = p.Finally[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)
	for j := range p.Finally[i].Matrix.Include {
	p.Finally[i].Matrix.Include[j].Params = p.Finally[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)
	}
	} else {
	p.Finally[i].DisplayName = substitution.ApplyReplacements(p.Finally[i].DisplayName, replacements)
	}
	for j := range p.Finally[i].Workspaces {
	p.Finally[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Finally[i].Workspaces[j].SubPath, replacements)
	}
	p.Finally[i].When = p.Finally[i].When.ReplaceVariables(replacements, arrayReplacements)
	if p.Finally[i].TaskRef != nil {
	if p.Finally[i].TaskRef.Params != nil {
	p.Finally[i].TaskRef.Params = p.Finally[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	}
	p.Finally[i].TaskRef.Name = substitution.ApplyReplacements(p.Finally[i].TaskRef.Name, replacements)
	}
	p.Finally[i] = propagateParams(p.Finally[i], replacements, arrayReplacements, objectReplacements)
	}
	return p
	}

[l-qing]

It's convenient to reuse this verification logic in the reconcile logic of PipelineRun.

[l-qing]

This is what is called deferred validation.
Targeted validation is performed after applying parameters.
The reason for not calling pipelineSpec.Validate again is that many field validations are unnecessary.

pipeline/pkg/reconciler/pipelinerun/pipelinerun.go

Lines 481 to 487 in aec465e

	if err := pipelineSpec.Validate(ctx); err != nil {
	// This Run has failed, so we need to mark it as failed and stop reconciling it
	pr.Status.MarkFailed(v1.PipelineRunReasonFailedValidation.String(),
	"Pipeline %s/%s can't be Run; it has an invalid spec: %s",
	pipelineMeta.Namespace, pipelineMeta.Name, pipelineErrors.WrapUserError(err))
	return controller.NewPermanentError(err)
	}

[l-qing]

The previous code contained a lot of duplication; here it has been simply refactored.

[l-qing]

The most important line of code in this new update is the support for variable replacement logic in OnError. 😆

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9