AWS Airflow Operators Reference

← Back to Main README

This document provides a comprehensive list of AWS operators available for Apache Airflow workflows in SageMaker Unified Studio.

Core Operators

EmptyOperator

EmptyOperator - airflow.operators.empty.EmptyOperator - No-op operator for workflow control

AWS Service Operators

AWS Batch

BatchCreateComputeEnvironmentOperator - airflow.providers.amazon.aws.operators.batch.BatchCreateComputeEnvironmentOperator - BatchCreateComputeEnvironment operations
BatchOperator - airflow.providers.amazon.aws.operators.batch.BatchOperator - Batch operations

AWS CloudFormation

CloudFormationCreateStackOperator - airflow.providers.amazon.aws.operators.cloud_formation.CloudFormationCreateStackOperator - CloudFormationCreateStack operations
CloudFormationDeleteStackOperator - airflow.providers.amazon.aws.operators.cloud_formation.CloudFormationDeleteStackOperator - CloudFormationDeleteStack operations

AWS DMS

DmsCreateTaskOperator - airflow.providers.amazon.aws.operators.dms.DmsCreateTaskOperator - DmsCreateTask operations
DmsDeleteTaskOperator - airflow.providers.amazon.aws.operators.dms.DmsDeleteTaskOperator - DmsDeleteTask operations
DmsDescribeTasksOperator - airflow.providers.amazon.aws.operators.dms.DmsDescribeTasksOperator - DmsDescribeTasks operations
DmsStartTaskOperator - airflow.providers.amazon.aws.operators.dms.DmsStartTaskOperator - DmsStartTask operations
DmsStopTaskOperator - airflow.providers.amazon.aws.operators.dms.DmsStopTaskOperator - DmsStopTask operations

AWS DataSync

DataSyncOperator - airflow.providers.amazon.aws.operators.datasync.DataSyncOperator - DataSync operations

AWS Glue

GlueCrawlerOperator - airflow.providers.amazon.aws.operators.glue.GlueCrawlerOperator - Glue Crawler operations
GlueDataBrewStartJobOperator - airflow.providers.amazon.aws.operators.glue.GlueDataBrewStartJobOperator - Glue DataBrewStartJob operations
GlueDataQualityOperator - airflow.providers.amazon.aws.operators.glue.GlueDataQualityOperator - Glue DataQuality operations
GlueDataQualityRuleRecommendationRunOperator - airflow.providers.amazon.aws.operators.glue.GlueDataQualityRuleRecommendationRunOperator - Glue DataQualityRuleRecommendationRun operations
GlueDataQualityRuleSetEvaluationRunOperator - airflow.providers.amazon.aws.operators.glue.GlueDataQualityRuleSetEvaluationRunOperator - Glue DataQualityRuleSetEvaluationRun operations
GlueJobOperator - airflow.providers.amazon.aws.operators.glue.GlueJobOperator - Glue Job operations

AWS Lambda

LambdaCreateFunctionOperator - airflow.providers.amazon.aws.operators.lambda_function.LambdaCreateFunctionOperator - Lambda CreateFunction operations
LambdaInvokeFunctionOperator - airflow.providers.amazon.aws.operators.lambda_function.LambdaInvokeFunctionOperator - Lambda InvokeFunction operations

AWS Step Functions

StepFunctionGetExecutionOutputOperator - airflow.providers.amazon.aws.operators.step_function.StepFunctionGetExecutionOutputOperator - Step Function GetExecutionOutput operations
StepFunctionStartExecutionOperator - airflow.providers.amazon.aws.operators.step_function.StepFunctionStartExecutionOperator - Step Function StartExecution operations

Amazon AppFlow

AppflowBaseOperator - airflow.providers.amazon.aws.operators.appflow.AppflowBaseOperator - AppflowBase operations
AppflowRecordsShortCircuitOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRecordsShortCircuitOperator - AppflowRecordsShortCircuit operations
AppflowRunAfterOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRunAfterOperator - AppflowRunAfter operations
AppflowRunBeforeOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRunBeforeOperator - AppflowRunBefore operations
AppflowRunDailyOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRunDailyOperator - AppflowRunDaily operations
AppflowRunFullOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRunFullOperator - AppflowRunFull operations
AppflowRunOperator - airflow.providers.amazon.aws.operators.appflow.AppflowRunOperator - AppflowRun operations

Amazon Athena

AthenaOperator - airflow.providers.amazon.aws.operators.athena.AthenaOperator - Athena operations

Amazon Bedrock

BedrockCreateDataSourceOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockCreateDataSourceOperator - BedrockCreateDataSource operations
BedrockCreateKnowledgeBaseOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockCreateKnowledgeBaseOperator - BedrockCreateKnowledgeBase operations
BedrockCreateProvisionedModelThroughputOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockCreateProvisionedModelThroughputOperator - BedrockCreateProvisionedModelThroughput operations
BedrockCustomizeModelOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockCustomizeModelOperator - BedrockCustomizeModel operations
BedrockIngestDataOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockIngestDataOperator - BedrockIngestData operations
BedrockInvokeModelOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockInvokeModelOperator - BedrockInvokeModel operations
BedrockRaGOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockRaGOperator - BedrockRaG operations
BedrockRetrieveOperator - airflow.providers.amazon.aws.operators.bedrock.BedrockRetrieveOperator - BedrockRetrieve operations

Amazon Comprehend

ComprehendBaseOperator - airflow.providers.amazon.aws.operators.comprehend.ComprehendBaseOperator - ComprehendBase operations
ComprehendCreateDocumentClassifierOperator - airflow.providers.amazon.aws.operators.comprehend.ComprehendCreateDocumentClassifierOperator - ComprehendCreateDocumentClassifier operations
ComprehendStartPiiEntitiesDetectionJobOperator - airflow.providers.amazon.aws.operators.comprehend.ComprehendStartPiiEntitiesDetectionJobOperator - ComprehendStartPiiEntitiesDetectionJob operations

Amazon EC2

EC2CreateInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2CreateInstanceOperator - EC2 CreateInstance operations
EC2HibernateInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2HibernateInstanceOperator - EC2 HibernateInstance operations
EC2RebootInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2RebootInstanceOperator - EC2 RebootInstance operations
EC2StartInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2StartInstanceOperator - EC2 StartInstance operations
EC2StopInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2StopInstanceOperator - EC2 StopInstance operations
EC2TerminateInstanceOperator - airflow.providers.amazon.aws.operators.ec2.EC2TerminateInstanceOperator - EC2 TerminateInstance operations

Amazon ECS

EcsBaseOperator - airflow.providers.amazon.aws.operators.ecs.EcsBaseOperator - ECS Base operations
EcsCreateClusterOperator - airflow.providers.amazon.aws.operators.ecs.EcsCreateClusterOperator - ECS CreateCluster operations
EcsDeleteClusterOperator - airflow.providers.amazon.aws.operators.ecs.EcsDeleteClusterOperator - ECS DeleteCluster operations
EcsDeregisterTaskDefinitionOperator - airflow.providers.amazon.aws.operators.ecs.EcsDeregisterTaskDefinitionOperator - ECS DeregisterTaskDefinition operations
EcsRegisterTaskDefinitionOperator - airflow.providers.amazon.aws.operators.ecs.EcsRegisterTaskDefinitionOperator - ECS RegisterTaskDefinition operations
EcsRunTaskOperator - airflow.providers.amazon.aws.operators.ecs.EcsRunTaskOperator - ECS RunTask operations

Amazon EKS

EksCreateClusterOperator - airflow.providers.amazon.aws.operators.eks.EksCreateClusterOperator - EKS CreateCluster operations
EksCreateFargateProfileOperator - airflow.providers.amazon.aws.operators.eks.EksCreateFargateProfileOperator - EKS CreateFargateProfile operations
EksCreateNodegroupOperator - airflow.providers.amazon.aws.operators.eks.EksCreateNodegroupOperator - EKS CreateNodegroup operations
EksDeleteClusterOperator - airflow.providers.amazon.aws.operators.eks.EksDeleteClusterOperator - EKS DeleteCluster operations
EksDeleteFargateProfileOperator - airflow.providers.amazon.aws.operators.eks.EksDeleteFargateProfileOperator - EKS DeleteFargateProfile operations
EksDeleteNodegroupOperator - airflow.providers.amazon.aws.operators.eks.EksDeleteNodegroupOperator - EKS DeleteNodegroup operations
EksPodOperator - airflow.providers.amazon.aws.operators.eks.EksPodOperator - EKS Pod operations

Amazon EMR

EmrAddStepsOperator - airflow.providers.amazon.aws.operators.emr.EmrAddStepsOperator - EMR AddSteps operations
EmrContainerOperator - airflow.providers.amazon.aws.operators.emr.EmrContainerOperator - EMR Container operations
EmrCreateJobFlowOperator - airflow.providers.amazon.aws.operators.emr.EmrCreateJobFlowOperator - EMR CreateJobFlow operations
EmrEksCreateClusterOperator - airflow.providers.amazon.aws.operators.emr.EmrEksCreateClusterOperator - EMR EKS CreateCluster operations
EmrModifyClusterOperator - airflow.providers.amazon.aws.operators.emr.EmrModifyClusterOperator - EMR ModifyCluster operations
EmrServerlessCreateApplicationOperator - airflow.providers.amazon.aws.operators.emr.EmrServerlessCreateApplicationOperator - EMR ServerlessCreateApplication operations
EmrServerlessDeleteApplicationOperator - airflow.providers.amazon.aws.operators.emr.EmrServerlessDeleteApplicationOperator - EMR ServerlessDeleteApplication operations
EmrServerlessStartJobOperator - airflow.providers.amazon.aws.operators.emr.EmrServerlessStartJobOperator - EMR ServerlessStartJob operations
EmrServerlessStopApplicationOperator - airflow.providers.amazon.aws.operators.emr.EmrServerlessStopApplicationOperator - EMR ServerlessStopApplication operations
EmrStartNotebookExecutionOperator - airflow.providers.amazon.aws.operators.emr.EmrStartNotebookExecutionOperator - EMR StartNotebookExecution operations
EmrStopNotebookExecutionOperator - airflow.providers.amazon.aws.operators.emr.EmrStopNotebookExecutionOperator - EMR StopNotebookExecution operations
EmrTerminateJobFlowOperator - airflow.providers.amazon.aws.operators.emr.EmrTerminateJobFlowOperator - EMR TerminateJobFlow operations

Amazon EventBridge

EventBridgeDisableRuleOperator - airflow.providers.amazon.aws.operators.eventbridge.EventBridgeDisableRuleOperator - EventBridgeDisableRule operations
EventBridgeEnableRuleOperator - airflow.providers.amazon.aws.operators.eventbridge.EventBridgeEnableRuleOperator - EventBridgeEnableRule operations
EventBridgePutEventsOperator - airflow.providers.amazon.aws.operators.eventbridge.EventBridgePutEventsOperator - EventBridgePutEvents operations
EventBridgePutRuleOperator - airflow.providers.amazon.aws.operators.eventbridge.EventBridgePutRuleOperator - EventBridgePutRule operations

Amazon Glacier

GlacierCreateJobOperator - airflow.providers.amazon.aws.operators.glacier.GlacierCreateJobOperator - GlacierCreateJob operations
GlacierUploadArchiveOperator - airflow.providers.amazon.aws.operators.glacier.GlacierUploadArchiveOperator - GlacierUploadArchive operations

Amazon Kinesis Analytics

KinesisAnalyticsV2CreateApplicationOperator - airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2CreateApplicationOperator - KinesisAnalyticsV2CreateApplication operations
KinesisAnalyticsV2StartApplicationOperator - airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StartApplicationOperator - KinesisAnalyticsV2StartApplication operations
KinesisAnalyticsV2StopApplicationOperator - airflow.providers.amazon.aws.operators.kinesis_analytics.KinesisAnalyticsV2StopApplicationOperator - KinesisAnalyticsV2StopApplication operations

Amazon Neptune

NeptuneStartDbClusterOperator - airflow.providers.amazon.aws.operators.neptune.NeptuneStartDbClusterOperator - NeptuneStartDbCluster operations
NeptuneStopDbClusterOperator - airflow.providers.amazon.aws.operators.neptune.NeptuneStopDbClusterOperator - NeptuneStopDbCluster operations

Amazon QuickSight

QuickSightCreateIngestionOperator - airflow.providers.amazon.aws.operators.quicksight.QuickSightCreateIngestionOperator - QuickSightCreateIngestion operations

Amazon RDS

RdsBaseOperator - airflow.providers.amazon.aws.operators.rds.RdsBaseOperator - RdsBase operations
RdsCancelExportTaskOperator - airflow.providers.amazon.aws.operators.rds.RdsCancelExportTaskOperator - RdsCancelExportTask operations
RdsCopyDbSnapshotOperator - airflow.providers.amazon.aws.operators.rds.RdsCopyDbSnapshotOperator - RdsCopyDbSnapshot operations
RdsCreateDbInstanceOperator - airflow.providers.amazon.aws.operators.rds.RdsCreateDbInstanceOperator - RdsCreateDbInstance operations
RdsCreateDbSnapshotOperator - airflow.providers.amazon.aws.operators.rds.RdsCreateDbSnapshotOperator - RdsCreateDbSnapshot operations
RdsCreateEventSubscriptionOperator - airflow.providers.amazon.aws.operators.rds.RdsCreateEventSubscriptionOperator - RdsCreateEventSubscription operations
RdsDeleteDbInstanceOperator - airflow.providers.amazon.aws.operators.rds.RdsDeleteDbInstanceOperator - RdsDeleteDbInstance operations
RdsDeleteDbSnapshotOperator - airflow.providers.amazon.aws.operators.rds.RdsDeleteDbSnapshotOperator - RdsDeleteDbSnapshot operations
RdsDeleteEventSubscriptionOperator - airflow.providers.amazon.aws.operators.rds.RdsDeleteEventSubscriptionOperator - RdsDeleteEventSubscription operations
RdsStartDbOperator - airflow.providers.amazon.aws.operators.rds.RdsStartDbOperator - RdsStartDb operations
RdsStartExportTaskOperator - airflow.providers.amazon.aws.operators.rds.RdsStartExportTaskOperator - RdsStartExportTask operations
RdsStopDbOperator - airflow.providers.amazon.aws.operators.rds.RdsStopDbOperator - RdsStopDb operations

Amazon Redshift

RedshiftCreateClusterOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftCreateClusterOperator - Redshift CreateCluster operations
RedshiftCreateClusterSnapshotOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftCreateClusterSnapshotOperator - Redshift CreateClusterSnapshot operations
RedshiftDataOperator - airflow.providers.amazon.aws.operators.redshift_data.RedshiftDataOperator - Redshift Data operations
RedshiftDeleteClusterOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftDeleteClusterOperator - Redshift DeleteCluster operations
RedshiftDeleteClusterSnapshotOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftDeleteClusterSnapshotOperator - Redshift DeleteClusterSnapshot operations
RedshiftPauseClusterOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftPauseClusterOperator - Redshift PauseCluster operations
RedshiftResumeClusterOperator - airflow.providers.amazon.aws.operators.redshift_cluster.RedshiftResumeClusterOperator - Redshift ResumeCluster operations

Amazon S3

S3CopyObjectOperator - airflow.providers.amazon.aws.operators.s3.S3CopyObjectOperator - S3 CopyObject operations
S3CreateBucketOperator - airflow.providers.amazon.aws.operators.s3.S3CreateBucketOperator - S3 CreateBucket operations
S3CreateObjectOperator - airflow.providers.amazon.aws.operators.s3.S3CreateObjectOperator - S3 CreateObject operations
S3DeleteBucketOperator - airflow.providers.amazon.aws.operators.s3.S3DeleteBucketOperator - S3 DeleteBucket operations
S3DeleteBucketTaggingOperator - airflow.providers.amazon.aws.operators.s3.S3DeleteBucketTaggingOperator - S3 DeleteBucketTagging operations
S3DeleteObjectsOperator - airflow.providers.amazon.aws.operators.s3.S3DeleteObjectsOperator - S3 DeleteObjects operations
S3GetBucketTaggingOperator - airflow.providers.amazon.aws.operators.s3.S3GetBucketTaggingOperator - S3 GetBucketTagging operations
S3ListOperator - airflow.providers.amazon.aws.operators.s3.S3ListOperator - S3 List operations
S3ListPrefixesOperator - airflow.providers.amazon.aws.operators.s3.S3ListPrefixesOperator - S3 ListPrefixes operations
S3PutBucketTaggingOperator - airflow.providers.amazon.aws.operators.s3.S3PutBucketTaggingOperator - S3 PutBucketTagging operations

Amazon SNS

SnsPublishOperator - airflow.providers.amazon.aws.operators.sns.SnsPublishOperator - SNS Publish operations

Amazon SQS

SqsPublishOperator - airflow.providers.amazon.aws.operators.sqs.SqsPublishOperator - SQS Publish operations

Amazon SageMaker

SageMakerAutoMLOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerAutoMLOperator - SageMaker AutoML operations
SageMakerBaseOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerBaseOperator - SageMaker Base operations
SageMakerCreateExperimentOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerCreateExperimentOperator - SageMaker CreateExperiment operations
SageMakerCreateNotebookOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerCreateNotebookOperator - SageMaker CreateNotebook operations
SageMakerDeleteModelOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerDeleteModelOperator - SageMaker DeleteModel operations
SageMakerDeleteNotebookOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerDeleteNotebookOperator - SageMaker DeleteNotebook operations
SageMakerEndpointConfigOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerEndpointConfigOperator - SageMaker EndpointConfig operations
SageMakerEndpointOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerEndpointOperator - SageMaker Endpoint operations
SageMakerModelOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerModelOperator - SageMaker Model operations
SageMakerNotebookOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerNotebookOperator - SageMaker Notebook operations
SageMakerProcessingOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerProcessingOperator - SageMaker Processing operations
SageMakerRegisterModelVersionOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerRegisterModelVersionOperator - SageMaker RegisterModelVersion operations
SageMakerStartNoteBookOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerStartNoteBookOperator - SageMaker StartNoteBook operations
SageMakerStartPipelineOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerStartPipelineOperator - SageMaker StartPipeline operations
SageMakerStopNotebookOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerStopNotebookOperator - SageMaker StopNotebook operations
SageMakerStopPipelineOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerStopPipelineOperator - SageMaker StopPipeline operations
SageMakerTrainingOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerTrainingOperator - SageMaker Training operations
SageMakerTransformOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerTransformOperator - SageMaker Transform operations
SageMakerTuningOperator - airflow.providers.amazon.aws.operators.sagemaker.SageMakerTuningOperator - SageMaker Tuning operations
SageMakerUnifiedStudioOperator - airflow.providers.amazon.aws.operators.sagemaker_unified_studio.SageMakerUnifiedStudioOperator - SageMaker Unified Studio operations
SageMakerNotebookOperator (Unified Studio) - airflow.providers.amazon.aws.operators.sagemaker_unified_studio.SageMakerNotebookOperator - Execute Jupyter notebooks in SageMaker Unified Studio

Sensors

AWS Batch

BatchComputeEnvironmentSensor - airflow.providers.amazon.aws.sensors.batch.BatchComputeEnvironmentSensor - Monitor BatchComputeEnvironment operations
BatchJobQueueSensor - airflow.providers.amazon.aws.sensors.batch.BatchJobQueueSensor - Monitor BatchJobQueue operations
BatchSensor - airflow.providers.amazon.aws.sensors.batch.BatchSensor - Monitor Batch operations

AWS CloudFormation

CloudFormationCreateStackSensor - airflow.providers.amazon.aws.sensors.cloud_formation.CloudFormationCreateStackSensor - Monitor CloudFormationCreateStack operations
CloudFormationDeleteStackSensor - airflow.providers.amazon.aws.sensors.cloud_formation.CloudFormationDeleteStackSensor - Monitor CloudFormationDeleteStack operations

AWS DMS

DmsTaskBaseSensor - airflow.providers.amazon.aws.sensors.dms.DmsTaskBaseSensor - Monitor DmsTaskBase operations
DmsTaskCompletedSensor - airflow.providers.amazon.aws.sensors.dms.DmsTaskCompletedSensor - Monitor DmsTaskCompleted operations

AWS Glue

GlueCatalogPartitionSensor - airflow.providers.amazon.aws.sensors.glue.GlueCatalogPartitionSensor - Monitor Glue CatalogPartition operations
GlueCrawlerSensor - airflow.providers.amazon.aws.sensors.glue.GlueCrawlerSensor - Monitor Glue Crawler operations
GlueDataQualityRuleRecommendationRunSensor - airflow.providers.amazon.aws.sensors.glue.GlueDataQualityRuleRecommendationRunSensor - Monitor Glue DataQualityRuleRecommendationRun operations
GlueDataQualityRuleSetEvaluationRunSensor - airflow.providers.amazon.aws.sensors.glue.GlueDataQualityRuleSetEvaluationRunSensor - Monitor Glue DataQualityRuleSetEvaluationRun operations
GlueJobSensor - airflow.providers.amazon.aws.sensors.glue.GlueJobSensor - Monitor Glue Job operations

AWS Lambda

LambdaFunctionStateSensor - airflow.providers.amazon.aws.sensors.lambda_function.LambdaFunctionStateSensor - Monitor Lambda FunctionState operations

AWS Step Functions

StepFunctionExecutionSensor - airflow.providers.amazon.aws.sensors.step_function.StepFunctionExecutionSensor - Monitor Step Function Execution operations

Amazon Athena

AthenaSensor - airflow.providers.amazon.aws.sensors.athena.AthenaSensor - Monitor Athena operations

Amazon Bedrock

BedrockBaseSensor - airflow.providers.amazon.aws.sensors.bedrock.BedrockBaseSensor - Monitor BedrockBase operations
BedrockCustomizeModelCompletedSensor - airflow.providers.amazon.aws.sensors.bedrock.BedrockCustomizeModelCompletedSensor - Monitor BedrockCustomizeModelCompleted operations
BedrockIngestionJobSensor - airflow.providers.amazon.aws.sensors.bedrock.BedrockIngestionJobSensor - Monitor BedrockIngestionJob operations
BedrockKnowledgeBaseActiveSensor - airflow.providers.amazon.aws.sensors.bedrock.BedrockKnowledgeBaseActiveSensor - Monitor BedrockKnowledgeBaseActive operations
BedrockProvisionModelThroughputCompletedSensor - airflow.providers.amazon.aws.sensors.bedrock.BedrockProvisionModelThroughputCompletedSensor - Monitor BedrockProvisionModelThroughputCompleted operations

Amazon Comprehend

ComprehendBaseSensor - airflow.providers.amazon.aws.sensors.comprehend.ComprehendBaseSensor - Monitor ComprehendBase operations
ComprehendCreateDocumentClassifierCompletedSensor - airflow.providers.amazon.aws.sensors.comprehend.ComprehendCreateDocumentClassifierCompletedSensor - Monitor ComprehendCreateDocumentClassifierCompleted operations
ComprehendStartPiiEntitiesDetectionJobCompletedSensor - airflow.providers.amazon.aws.sensors.comprehend.ComprehendStartPiiEntitiesDetectionJobCompletedSensor - Monitor ComprehendStartPiiEntitiesDetectionJobCompleted operations

Amazon EC2

EC2InstanceStateSensor - airflow.providers.amazon.aws.sensors.ec2.EC2InstanceStateSensor - Monitor EC2 InstanceState operations

Amazon ECS

EcsBaseSensor - airflow.providers.amazon.aws.sensors.ecs.EcsBaseSensor - Monitor ECS Base operations
EcsClusterStateSensor - airflow.providers.amazon.aws.sensors.ecs.EcsClusterStateSensor - Monitor ECS ClusterState operations
EcsTaskDefinitionStateSensor - airflow.providers.amazon.aws.sensors.ecs.EcsTaskDefinitionStateSensor - Monitor ECS TaskDefinitionState operations
EcsTaskStateSensor - airflow.providers.amazon.aws.sensors.ecs.EcsTaskStateSensor - Monitor ECS TaskState operations

Amazon EKS

EksBaseSensor - airflow.providers.amazon.aws.sensors.eks.EksBaseSensor - Monitor EKS Base operations
EksClusterStateSensor - airflow.providers.amazon.aws.sensors.eks.EksClusterStateSensor - Monitor EKS ClusterState operations
EksFargateProfileStateSensor - airflow.providers.amazon.aws.sensors.eks.EksFargateProfileStateSensor - Monitor EKS FargateProfileState operations
EksNodegroupStateSensor - airflow.providers.amazon.aws.sensors.eks.EksNodegroupStateSensor - Monitor EKS NodegroupState operations

Amazon EMR

EmrBaseSensor - airflow.providers.amazon.aws.sensors.emr.EmrBaseSensor - Monitor EMR Base operations
EmrContainerSensor - airflow.providers.amazon.aws.sensors.emr.EmrContainerSensor - Monitor EMR Container operations
EmrJobFlowSensor - airflow.providers.amazon.aws.sensors.emr.EmrJobFlowSensor - Monitor EMR JobFlow operations
EmrNotebookExecutionSensor - airflow.providers.amazon.aws.sensors.emr.EmrNotebookExecutionSensor - Monitor EMR NotebookExecution operations
EmrServerlessApplicationSensor - airflow.providers.amazon.aws.sensors.emr.EmrServerlessApplicationSensor - Monitor EMR ServerlessApplication operations
EmrServerlessJobSensor - airflow.providers.amazon.aws.sensors.emr.EmrServerlessJobSensor - Monitor EMR ServerlessJob operations
EmrStepSensor - airflow.providers.amazon.aws.sensors.emr.EmrStepSensor - Monitor EMR Step operations

Amazon Glacier

GlacierJobOperationSensor - airflow.providers.amazon.aws.sensors.glacier.GlacierJobOperationSensor - Monitor GlacierJobOperation operations

Amazon Kinesis Analytics

KinesisAnalyticsV2BaseSensor - airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2BaseSensor - Monitor KinesisAnalyticsV2Base operations
KinesisAnalyticsV2StartApplicationCompletedSensor - airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StartApplicationCompletedSensor - Monitor KinesisAnalyticsV2StartApplicationCompleted operations
KinesisAnalyticsV2StopApplicationCompletedSensor - airflow.providers.amazon.aws.sensors.kinesis_analytics.KinesisAnalyticsV2StopApplicationCompletedSensor - Monitor KinesisAnalyticsV2StopApplicationCompleted operations

Amazon QuickSight

QuickSightSensor - airflow.providers.amazon.aws.sensors.quicksight.QuickSightSensor - Monitor QuickSight operations

Amazon RDS

RdsBaseSensor - airflow.providers.amazon.aws.sensors.rds.RdsBaseSensor - Monitor RdsBase operations
RdsDbSensor - airflow.providers.amazon.aws.sensors.rds.RdsDbSensor - Monitor RdsDb operations
RdsExportTaskExistenceSensor - airflow.providers.amazon.aws.sensors.rds.RdsExportTaskExistenceSensor - Monitor RdsExportTaskExistence operations
RdsSnapshotExistenceSensor - airflow.providers.amazon.aws.sensors.rds.RdsSnapshotExistenceSensor - Monitor RdsSnapshotExistence operations

Amazon Redshift

RedshiftClusterSensor - airflow.providers.amazon.aws.sensors.redshift_cluster.RedshiftClusterSensor - Monitor Redshift Cluster operations

Amazon S3

S3KeySensor - airflow.providers.amazon.aws.sensors.s3.S3KeySensor - Monitor S3 Key operations
S3KeysUnchangedSensor - airflow.providers.amazon.aws.sensors.s3.S3KeysUnchangedSensor - Monitor S3 KeysUnchanged operations

Amazon SQS

SqsSensor - airflow.providers.amazon.aws.sensors.sqs.SqsSensor - Monitor SQS operations

Amazon SageMaker

SageMakerAutoMLSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerAutoMLSensor - Monitor SageMaker AutoML operations
SageMakerBaseSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerBaseSensor - Monitor SageMaker Base operations
SageMakerEndpointSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerEndpointSensor - Monitor SageMaker Endpoint operations
SageMakerNotebookSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerNotebookSensor - Monitor SageMaker Notebook operations
SageMakerPipelineSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerPipelineSensor - Monitor SageMaker Pipeline operations
SageMakerProcessingSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerProcessingSensor - Monitor SageMaker Processing operations
SageMakerTrainingSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTrainingSensor - Monitor SageMaker Training operations
SageMakerTransformSensor - airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTransformSensor - Monitor SageMaker Transform operations

Other AWS Services

AwsBaseSensor - airflow.providers.amazon.aws.sensors.base_sensor.AwsBaseSensor - Base class for AWS sensors
DynamoDBValueSensor - airflow.providers.amazon.aws.sensors.dynamodb.DynamoDBValueSensor - Monitor DynamoDBValue operations
OpenSearchServerlessCollectionActiveSensor - airflow.providers.amazon.aws.sensors.opensearch_serverless.OpenSearchServerlessCollectionActiveSensor - Monitor OpenSearchServerlessCollectionActive operations

Usage Example

from airflow import DAG
from airflow.providers.amazon.aws.operators.sagemaker import SageMakerTrainingOperator
from airflow.providers.amazon.aws.operators.s3 import S3CopyObjectOperator

dag = DAG('ml_pipeline', schedule_interval='@daily')

train_model = SageMakerTrainingOperator(
    task_id='train_model',
    config={...},
    dag=dag
)

copy_results = S3CopyObjectOperator(
    task_id='copy_results',
    source_bucket_key='model/output',
    dest_bucket_key='production/model',
    dag=dag
)

train_model >> copy_results

FilesExpand file tree

airflow-aws-operators.md

Latest commit

History

airflow-aws-operators.md

File metadata and controls

AWS Airflow Operators Reference

Core Operators

EmptyOperator

AWS Service Operators

AWS Batch

AWS CloudFormation

AWS DMS

AWS DataSync

AWS Glue

AWS Lambda

AWS Step Functions

Amazon AppFlow

Amazon Athena

Amazon Bedrock

Amazon Comprehend

Amazon EC2

Amazon ECS

Amazon EKS

Amazon EMR

Amazon EventBridge

Amazon Glacier

Amazon Kinesis Analytics

Amazon Neptune

Amazon QuickSight

Amazon RDS

Amazon Redshift

Amazon S3

Amazon SNS

Amazon SQS

Amazon SageMaker

Sensors

AWS Batch

AWS CloudFormation

AWS DMS

AWS Glue

AWS Lambda

AWS Step Functions

Amazon Athena

Amazon Bedrock

Amazon Comprehend

Amazon EC2

Amazon ECS

Amazon EKS

Amazon EMR

Amazon Glacier

Amazon Kinesis Analytics

Amazon QuickSight

Amazon RDS

Amazon Redshift

Amazon S3

Amazon SQS

Amazon SageMaker

Other AWS Services

Usage Example

Resources