New Parallel execution and early stopping pattern #408
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bfreiberg
suggested changes
Nov 20, 2025
| aws_services = [ | ||
| "EC2", "S3", "Lambda", "DynamoDB", "RDS", "Aurora", "ECS", "EKS", | ||
| "SQS", "SNS", "API Gateway", "CloudFormation", "CloudFront", "Route 53", | ||
| "2PC", "IAM", "CloudWatch", "Step Functions", "EventBridge", "Cognito", |
Comment on lines
38
to
39
| "repoURL": "https://github.com/aws-samples/step-functions-workflows-collection/tree/main/dynamodb-transfer/", | ||
| "templateDir":"dynamodb-transfer", |
Contributor
There was a problem hiding this comment.
forgot to update the folder name?
| "repoURL": "https://github.com/aws-samples/step-functions-workflows-collection/tree/main/dynamodb-transfer/", | ||
| "templateDir":"dynamodb-transfer", | ||
| "templateFile": "template.yaml", | ||
| "ASL": "statemachine/statemachine.asl.json" |
| { | ||
| "name": "Dave Horne", | ||
| "image": "https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2025/04/07/me.jpeg", | ||
| "bio": "Dave is a senior solutions architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.", |
Contributor
There was a problem hiding this comment.
Suggested change
| "bio": "Dave is a senior solutions architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.", | |
| "bio": "Dave is a senior Solutions Architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.", |
| { | ||
| "name": "Satya Vedamtam", | ||
| "image": "", | ||
| "bio": "Satya is a senior solutions architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.", |
Contributor
There was a problem hiding this comment.
Suggested change
| "bio": "Satya is a senior solutions architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.", | |
| "bio": "Satya is a senior Solutions Architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.", |
early-stopping-sf-bedrock/README.md
Outdated
Comment on lines
82
to
83
| - You'll see the workflow spawn multiple parallel agents with different specialized approaches | ||
| - Watch as some agents complete successfully while others are terminated early |
Contributor
There was a problem hiding this comment.
I don't think it's that visual. I would talk more about the map state and how to check the state of each iteration
early-stopping-sf-bedrock/README.md
Outdated
| aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus" | ||
| ``` | ||
| ---- | ||
| Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved. |
Contributor
There was a problem hiding this comment.
Suggested change
| Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved. | |
| Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved. |
| Properties: | ||
| CodeUri: functions/ | ||
| Handler: awsdocmcpserver.handler | ||
| Runtime: python3.12 |
Contributor
There was a problem hiding this comment.
Please use the latest runtime version everywhere
| - Effect: Allow | ||
| Action: | ||
| - states:StopExecution | ||
| Resource: "*" |
Contributor
There was a problem hiding this comment.
Please scope this down further
| MemorySize: 256 | ||
| Environment: | ||
| Variables: | ||
| MODEL_ID: us.anthropic.claude-3-5-haiku-20241022-v1:0 |
Contributor
There was a problem hiding this comment.
Any particular reason why Haiku 3.5? Anyway, add a blurb regarding model selection to the README please
Author
|
All comments updated in code.
Satya Vedamtam
-------------------------------------------------------------------------------------------------------------------------------------
Sr. Solutions Architect
Word Wide Public Sector - Federal Civilian
Department of Justice and Federal Judiciary
***@***.******@***.***> | 571-421-3000 (m)
Thoughts on our interaction? Provide feedback here<https://feedback.aws.amazon.com/?ea=sspvedam&fn=Satya&ln=Vedamtam>
From: Ben ***@***.***>
Sent: Thursday, November 20, 2025 8:47 AM
To: aws-samples/step-functions-workflows-collection ***@***.***>
Cc: Vedamtam, Satya ***@***.***>; Author ***@***.***>
Subject: Re: [aws-samples/step-functions-workflows-collection] New Parallel execution and early stopping pattern (PR #408)
@bfreiberg requested changes on this pull request.
________________________________
In early-stopping-sf-bedrock/functions/synthesis.py<#408 (comment)>:
+ print(f"Error in lambda_handler: {str(e)}")
+ print(traceback.format_exc())
+ # Fallback response if anything fails
+ return {
+ 'synthesized_result': f"I encountered an error while processing your request about {original_query if 'original_query' in locals() else 'your query'}. Please try again.",
+ 'source_count': 0,
+ 'services_mentioned': [],
+ 'original_query': original_query if 'original_query' in locals() else "Unknown query"
+ }
+
+def extract_aws_services(text):
+ """Extract mentioned AWS services from the response"""
+ aws_services = [
+ "EC2", "S3", "Lambda", "DynamoDB", "RDS", "Aurora", "ECS", "EKS",
+ "SQS", "SNS", "API Gateway", "CloudFormation", "CloudFront", "Route 53",
+ "2PC", "IAM", "CloudWatch", "Step Functions", "EventBridge", "Cognito",
What is 2PC?
________________________________
In early-stopping-sf-bedrock/example-workflow.json<#408 (comment)>:
+ "repoURL": "https://github.com/aws-samples/step-functions-workflows-collection/tree/main/dynamodb-transfer/",
+ "templateDir":"dynamodb-transfer",
forgot to update the folder name?
________________________________
In early-stopping-sf-bedrock/example-workflow.json<#408 (comment)>:
+ "headline": "Cleanup",
+ "text": [
+ "1. Delete the stack: <code>sam delete</code>."
+ ]
+ },
+ "deploy": {
+ "text": [
+ "sam deploy --guided"
+ ]
+ },
+ "gitHub": {
+ "template": {
+ "repoURL": "https://github.com/aws-samples/step-functions-workflows-collection/tree/main/dynamodb-transfer/",
+ "templateDir":"dynamodb-transfer",
+ "templateFile": "template.yaml",
+ "ASL": "statemachine/statemachine.asl.json"
use correct filename
________________________________
In early-stopping-sf-bedrock/example-workflow.json<#408 (comment)>:
+ ]
+ },
+ "resources": {
+ "headline": "Additional resources",
+ "bullets": [
+ {
+ "text": "The AWS Step Functions Workshop",
+ "link": "https://catalog.workshops.aws/stepfunctions/en-US"
+ }
+ ]
+ },
+ "authors": [
+ {
+ "name": "Dave Horne",
+ "image": "https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2025/04/07/me.jpeg",
+ "bio": "Dave is a senior solutions architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.",
⬇️ Suggested change
- "bio": "Dave is a senior solutions architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.",
+ "bio": "Dave is a senior Solutions Architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.",
________________________________
In early-stopping-sf-bedrock/example-workflow.json<#408 (comment)>:
+ }
+ ]
+ },
+ "authors": [
+ {
+ "name": "Dave Horne",
+ "image": "https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2025/04/07/me.jpeg",
+ "bio": "Dave is a senior solutions architect supporting Federal System Integrators at AWS. He is based in Washington, DC, and has 15 years of experience building, modernizing, and integrating systems for public sector customers. Outside of work, Dave enjoys playing with his kids, hiking, and watching Penn State football.",
+ "linkedin": "davidjhorne",
+ "twitter": ""
+ },
+ {
+ "name": "Satya Vedamtam",
+ "image": "",
+ "bio": "Satya is a senior solutions architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.",
⬇️ Suggested change
- "bio": "Satya is a senior solutions architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.",
+ "bio": "Satya is a senior Solutions Architect supporting US Federal customers at AWS. He is passionate about serverless technologies and everyday is an opportunity to solve customer challenges.",
________________________________
In early-stopping-sf-bedrock/README.md<#408 (comment)>:
+
+2. Each agent independently retrieves information from the AWS Documentation MCP Server (Context Provider). If relevant information isn't available, agents fall back to Amazon Bedrock models to generate responses.
+
+3. As results arrive, an Evaluation Agent assesses each response's confidence level. When any agent produces a result exceeding the confidence threshold (typically 0.95), Step Functions automatically terminates the other executing workflows to optimize resources and costs.
+
+4. If no single agent achieves high confidence, a Synthesis Agent combines insights from multiple agents into a comprehensive answer. The final result is returned to the user with processing metadata.
+
+
+## Image
+
+
+
+
+
+## Testing
+Based on the uploaded document, here are the testing instructions for logging into the console and starting execution of the main workflow:
Remove prompt :)
________________________________
In early-stopping-sf-bedrock/README.md<#408 (comment)>:
+1. Log in to the AWS Management Console
+ - Navigate to console.aws.amazon.com
+ - Enter your AWS account credentials
+ - Select the region where you deployed the solution (e.g., us-east-1)
+
+2. Access AWS Step Functions Console
+ - From the AWS services menu, search for and select "Step Functions"
+ - In the Step Functions dashboard, locate and click on the "MainStateMachine" that was created during deployment
+ - You should see details about the state machine including its Amazon Resource Name (ARN)
No need to be so detailed, can be condensed to go to the SFN console
________________________________
In early-stopping-sf-bedrock/README.md<#408 (comment)>:
+ - You'll see the workflow spawn multiple parallel agents with different specialized approaches
+ - Watch as some agents complete successfully while others are terminated early
I don't think it's that visual. I would talk more about the map state and how to check the state of each iteration
________________________________
In early-stopping-sf-bedrock/README.md<#408 (comment)>:
+ - Review the "processing_details" section to confirm early stopping functionality
+ - Verify that responses include attribution to AWS Documentation
+
+
+## Cleanup
+
+1. Delete the stack
+ ```bash
+ aws cloudformation delete-stack --stack-name STACK_NAME
+ ```
+1. Confirm the stack has been deleted
+ ```bash
+ aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
+ ```
+----
+Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
⬇️ Suggested change
-Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
________________________________
In early-stopping-sf-bedrock/template.yaml<#408 (comment)>:
@@ -0,0 +1,212 @@
+AWSTemplateFormatVersion: '2010-09-09'
+Transform: AWS::Serverless-2016-10-31
+Description: SAM Template for Worker Job Processing
+
+Resources:
+ # AWS MCP Server Lambda Function (First Deployment)
+ AWSMCPServerFunction:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: functions/
+ Handler: awsdocmcpserver.handler
+ Runtime: python3.12
Please use the latest runtime version everywhere
________________________________
In early-stopping-sf-bedrock/template.yaml<#408 (comment)>:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: functions/
+ Handler: evaluationfn.lambda_handler
+ Runtime: python3.12
+ Timeout: 30
+ MemorySize: 128
+ Environment:
+ Variables:
+ CONFIDENCE_THRESHOLD: "0.8"
+ Policies:
+ - Statement:
+ - Effect: Allow
+ Action:
+ - states:StopExecution
+ Resource: "*"
Please scope this down further
________________________________
In early-stopping-sf-bedrock/template.yaml<#408 (comment)>:
+ - Effect: Allow
+ Action:
+ - states:StopExecution
+ Resource: "*"
+
+ SynthesisFunction:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: functions/
+ Handler: synthesis.lambda_handler
+ Runtime: python3.12
+ Timeout: 60
+ MemorySize: 256
+ Environment:
+ Variables:
+ MODEL_ID: us.anthropic.claude-3-5-haiku-20241022-v1:0
Any particular reason why Haiku 3.5? Anyway, add a blurb regarding model selection to the README please
—
Reply to this email directly, view it on GitHub<#408 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A74C36QE5NCJ4OVDGKDCTJD35XA5XAVCNFSM6AAAAACMI6WXXWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTIOBXHEZDANRSHA>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
|
bfreiberg
approved these changes
Nov 25, 2025
Contributor
bfreiberg
left a comment
bfreiberg
left a comment
There was a problem hiding this comment.
Looks good, thank you for your contribution. There might be a delay in merging it to the site due to re:Invent
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Parallel Agent Early Stopping pattern uses AWS Step Functions and Amazon Bedrock to run multiple AI agents simultaneously on the same problem, with different approaches, and automatically terminates unnecessary processes once a high-confidence solution is discovered. The workflow coordinates Worker Agents that either retrieve information from the AWS Documentation MCP Server or generate responses using Amazon Bedrock models, while an Evaluation Agent continuously assesses confidence levels and triggers early stopping when a predetermined threshold is met. This design optimizes both performance and cost through parallel exploration, intelligent termination, and resource optimization techniques including agent tiering, token optimization, and Lambda memory tuning.
#408
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.