Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQS event source support for TestTool #2008

Open
wants to merge 15 commits into
base: dev
Choose a base branch
from
Open

Conversation

normj
Copy link
Member

@normj normj commented Mar 11, 2025

Issue #, if available:
aws/integrations-on-dotnet-aspire-for-aws#41

Description of changes:
Adds the ability to add an SQS queue as an event source within the test tool. The outline of the control flow is

  • An instance of SQSEventSourceBackgroundService which is an IHostedService will be created for each configured SQS event source.
  • Poll the queue for messages
  • Convert the messages from SDK POCO to Lambda event POCO
  • Invoke the Lambda function via the Lambda Runtime API with the messages
  • Unless disabled delete the processed messages. (This includes handling SQSBatchResponse with partial failures)

The configuration is controlled by the --sqs-eventsource-config switch. The default format is comma delimited key pairs. For example QueueUrl=<queue-url>,FunctionName=<function-name>,VisibilityTimeout=100. To add consistency with the API Gatway environment variable support it also supports JSON format. The JSON format also supports an array of configs to spin up multiple event sources. Using JSON on the command line is awkward though because you have to handle escaping quotes and using the comma delimited key pairs format avoids that.

For Aspire I need to pass the config via an environment variable. I introduced the idea that could be expanded later that if the value for --sqs-eventsource-config starts with env: then the tool will look for the config in the environment variables in the suffix of the value.

An example of the .NET Aspire experience

var builder = DistributedApplication.CreateBuilder(args);

var awsSdkConfig = builder.AddAWSSDKConfig().WithRegion(Amazon.RegionEndpoint.USWest2);

var cdkStackResource = builder.AddAWSCDKStack("AWSLambdaPlaygroundResources");
var sqsDemoQueue = cdkStackResource.AddSQSQueue("DemoQueue");

builder.AddAWSLambdaFunction<Projects.SQSProcessorFunction>("SQSProcessorFunction", "SQSProcessorFunction::SQSProcessorFunction.Function::FunctionHandler")
        .WithReference(awsSdkConfig)
        .WithSQSEventSource(sqsDemoQueue);

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@normj
Copy link
Member Author

normj commented Mar 11, 2025

I'm tracking down why the integ tests are failing in CI but not locally.

@normj normj force-pushed the normj/testtool-sqs branch 2 times, most recently from f5b23de to 5cc0850 Compare March 12, 2025 00:48
@normj normj force-pushed the normj/testtool-sqs branch from 5cc0850 to 2eca321 Compare March 12, 2025 04:35
@philasmar
Copy link
Contributor

Could you take another look at the CI for this PR? I reran it again and it seems like the tests are hanging in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants