Skip to content

Add Production Dependency Validation #95

@transientlunatic

Description

@transientlunatic

Summary: Validate that productions in needs will provide required data products before submission.

Description:

Add validate_needs() method to Analysis class that:

  • Checks if productions listed in needs provide the required inputs (we can find this via the enhancements described in Add Pipeline Input/Output Metadata System #94)
  • Warns (or errors) if required data products won't be available
  • Runs at build/submission time, not runtime

Example validation:

def validate_needs(self):
    required = self.pipeline.get_actual_inputs(self)
    for requirement in required:
        satisfied = any(
            requirement in dep.pipeline.get_actual_outputs(dep)
            for dep in self.needs
        )
        if not satisfied:
            warnings.warn(f"Production requires '{requirement}' but no dependency provides it")

Benefits:

  • Catch configuration errors before wasting compute time
  • Better user feedback about missing dependencies
  • Foundation for smarter workflow orchestration

Metadata

Metadata

Labels

enhancementNew feature or request

Projects

Status

Todo

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions