Exception: 'ProcessingStep' object is not iterable #3887
Unanswered
aravinddeveloper
asked this question in
Help
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Executing the below pipeline script generates exception: 'ProcessingStep' object is not iterable
import os
import boto3
import sagemaker
import sagemaker.session
from sagemaker.estimator import Estimator
from sagemaker.inputs import TrainingInput
from sagemaker.model_metrics import MetricsSource, ModelMetrics
from sagemaker.processing import ProcessingInput, ProcessingOutput, ScriptProcessor
from sagemaker.pytorch.processing import PyTorchProcessor
from sagemaker.pytorch.estimator import PyTorch
from sagemaker.workflow.condition_step import ConditionStep
from sagemaker.workflow.conditions import ConditionGreaterThanOrEqualTo
from sagemaker.workflow.functions import JsonGet
from sagemaker.workflow.parameters import ParameterInteger, ParameterString
from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.properties import PropertyFile
from sagemaker.workflow.step_collections import RegisterModel
from sagemaker.workflow.steps import ProcessingStep, TrainingStep, CacheConfig
from sagemaker.workflow.functions import Join
from sagemaker.workflow.pipeline_context import PipelineSession
BASE_DIR = os.path.dirname(os.path.realpath(file))
BUCKET_NAME = ""
PROCESSING_PATH = "/opt/ml/processing"
MODEL_PATH = "/opt/ml/model"
train_folder_name="train"
val_folder_name="val"
eval_folder_name="eval"
def get_session(region, default_bucket=BUCKET_NAME):
boto_session = boto3.Session(region_name=region)
sagemaker_client = boto_session.client("sagemaker")
runtime_client = boto_session.client("sagemaker-runtime")
return PipelineSession(
boto_session=boto_session,
sagemaker_client=sagemaker_client,
default_bucket=default_bucket,
)`
def get_pipeline( region,
default_bucket=BUCKET_NAME,
role=None,
model_package_group_name="YogaPoseDetectionPackageGroup",
pipeline_name="yogapose-detection-pipeline",
base_job_prefix="yogapose"):
Logs:
2023-05-29T18:11:30.560+05:30 Setting parameters for pipeline
2023-05-29T18:11:30.560+05:30 Setting pytorch processor
2023-05-29T18:11:30.560+05:30 The input argument instance_type of function (sagemaker.image_uris.retrieve) is a pipeline variable (<class 'sagemaker.workflow.parameters.ParameterString'>), which is not allowed. The default_value of this Parameter object will be used to override it. Please make sure the default_value is valid.
2023-05-29T18:11:30.560+05:30 setting processing step args
2023-05-29T18:11:30.560+05:30 /root/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sagemaker/workflow/pipeline_context.py:233: UserWarning: Running within a PipelineSession, there will be No Wait, No Logs, and No Job being started.
2023-05-29T18:11:30.560+05:30 warnings.warn(
2023-05-29T18:11:30.560+05:30
2023-05-29T18:11:30.560+05:30 Job Name: pytorch-2023-05-29-12-41-27-228
2023-05-29T18:11:30.560+05:30 Building processing step
2023-05-29T18:11:30.560+05:30 Setting Training estimator
2023-05-29T18:11:30.560+05:30 Setting hyperparameters
2023-05-29T18:11:30.560+05:30 Setting training step
2023-05-29T18:11:30.560+05:30 /root/.pyenv/versions/3.8.13/lib/python3.8/site-packages/sagemaker/workflow/steps.py:444: UserWarning: Profiling is enabled on the provided estimator. The default profiler rule includes a timestamp which will change each time the pipeline is upserted, causing cache misses. If profiling is not needed, set disable_profiler to True on the estimator.
2023-05-29T18:11:30.560+05:30 warnings.warn(msg)
2023-05-29T18:11:30.560+05:30 Setting model evaluation processor
2023-05-29T18:11:30.560+05:30 The input argument instance_type of function (sagemaker.image_uris.retrieve) is a pipeline variable (<class 'sagemaker.workflow.parameters.ParameterString'>), which is not allowed. The default_value of this Parameter object will be used to override it. Please make sure the default_value is valid.
2023-05-29T18:11:30.560+05:30 setting evaluation step args
2023-05-29T18:11:30.560+05:30
2023-05-29T18:11:30.560+05:30 Job Name: pytorch-2023-05-29-12-41-27-706
2023-05-29T18:11:30.560+05:30 Setting model evaluation step
2023-05-29T18:11:30.560+05:30 Setting model metrics
2023-05-29T18:11:30.560+05:30 Registering model
2023-05-29T18:11:30.560+05:30 condition step
2023-05-29T18:11:30.560+05:30 Building final pipeline
2023-05-29T18:11:30.560+05:30 ###### Creating/updating a SageMaker Pipeline with the following definition:
2023-05-29T18:11:30.560+05:30 SMDebug does not currently support distributed training jobs with checkpointing enabled. Therefore, to allow parameterized instance_count and allow to change it to any values in execution time, the debugger_hook_config is disabled.
2023-05-29T18:11:30.560+05:30 The input argument instance_type of function (sagemaker.image_uris.retrieve) is a pipeline variable (<class 'sagemaker.workflow.parameters.ParameterString'>), which is not allowed. The default_value of this Parameter object will be used to override it. Please make sure the default_value is valid.
2023-05-29T18:11:30.560+05:30 Exception: 'ProcessingStep' object is not iterable
2023-05-29T18:11:30.560+05:30
2023-05-29T18:11:30.560+05:30 [Container] 2023/05/29 12:41:28 Command did not exit successfully run-pipeline
Environment:
Beta Was this translation helpful? Give feedback.
All reactions