Implement local Lambda testing with Runtime Interface Emulator

Author: AntoniaSzecsi

.gitignore

@@ -156,3 +156,6 @@ deps/curl-*/
 
 # local caches
 .DS_Store
+
+# local build artifacts
+build-artifacts

Dockerfile.build

@@ -0,0 +1,28 @@
+FROM python:3.9-alpine AS build-image
+
+# Install build dependencies
+RUN apk add --no-cache \
+    build-base \
+    libtool \
+    autoconf \
+    automake \
+    elfutils-dev \
+    make \
+    cmake \
+    libcurl \
+    curl \
+    libstdc++ \
+    binutils
+
+# Build awslambdaric
+ARG RIC_BUILD_DIR="/home/build/"
+RUN mkdir -p ${RIC_BUILD_DIR}
+WORKDIR ${RIC_BUILD_DIR}
+COPY . .
+RUN pip install setuptools
+RUN curl -sSL https://install.python-poetry.org | python3 - && \
+    ln -s /root/.local/bin/poetry /usr/local/bin/poetry
+RUN make init build
+
+# Keep the built wheel accessible
+CMD ["sh", "-c", "echo 'Build complete. Wheel available in /home/build/dist/'"]

Dockerfile.rie

@@ -0,0 +1,26 @@
+FROM python:3.9-alpine
+
+RUN apk add --no-cache libstdc++ curl
+
+# Install RIE
+RUN curl -Lo /usr/local/bin/aws-lambda-rie https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie && \
+    chmod +x /usr/local/bin/aws-lambda-rie
+
+# Add the pre-built wheel
+ADD build-artifacts/*.whl /tmp/
+
+# Install the wheel
+RUN pip install /tmp/*.whl
+
+# Copy test handler
+COPY tests/integration/test-handlers/echo/app.py /var/task/app.py
+
+# Set environment for local testing
+ENV AWS_LAMBDA_RUNTIME_API="127.0.0.1:8080"
+ENV LAMBDA_TASK_ROOT="/var/task"
+ENV _HANDLER="app.handler"
+
+WORKDIR /var/task
+
+ENTRYPOINT ["/usr/local/bin/aws-lambda-rie"]
+CMD ["python", "-m", "awslambdaric", "app.handler"]

Makefile

@@ -57,6 +57,14 @@ pr: init check-format check-security dev
 codebuild: setup-codebuild-agent
 	CODEBUILD_IMAGE_TAG=codebuild-agent DISTRO="$(DISTRO)" tests/integration/codebuild-local/test_all.sh tests/integration/codebuild
 
+.PHONY: build-container
+build-container:
+	./scripts/build-container.sh
+
+.PHONY: test-rie
+test-rie:
+	./scripts/test-rie.sh
+
 define HELP_MESSAGE
 
 Usage: $ make [TARGETS]
@@ -68,6 +76,8 @@ TARGETS
 	clean       	Cleans the working directory using scripts/dev.py.
 	dev         	Run all development tests using scripts/dev.py.
 	init        	Install dependencies via scripts/dev.py.
+	build-container	Build awslambdaric wheel in isolated container.
+	test-rie    	Test with RIE using pre-built wheel (run build-container first).
 	pr          	Perform all checks before submitting a Pull Request.
 	test        	Run unit tests using scripts/dev.py.
 	lint        	Run all linters via scripts/dev.py.

README.md

@@ -14,6 +14,26 @@ The Python Runtime Interface Client package currently supports Python versions:
 
 ## Usage
 
+### Container-Based Builds
+
+For development or when you need to build awslambdaric from source, you can use container-based builds to ensure consistent compilation across different platforms, and native dependencies linking.
+
+```shell script
+# Build awslambdaric wheel in a Linux container
+make build-container
+# Or with poetry (run 'poetry install' first):
+poetry run build-container
+
+# Test with RIE using the built wheel
+make test-rie
+# Or with poetry:
+poetry run test-rie
+```
+
+This approach builds the C++ extensions in a Linux environment, ensuring compatibility with Lambda's runtime environment regardless of your host OS.
+
+**Note**: Running `make build` (or `poetry run build`) on non-Linux machines will not properly link the native C++ dependencies, resulting in a non-functional runtime client. Always use container-based builds for development.
+
 ### Creating a Docker Image for Lambda with the Runtime Interface Client
 First step is to choose the base image to be used. The supported Linux OS distributions are:
 
@@ -103,38 +123,23 @@ def handler(event, context):
 
 ### Local Testing
 
-To make it easy to locally test Lambda functions packaged as container images we open-sourced a lightweight web-server, Lambda Runtime Interface Emulator (RIE), which allows your function packaged as a container image to accept HTTP requests. You can install the [AWS Lambda Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator) on your local machine to test your function. Then when you run the image function, you set the entrypoint to be the emulator.
-
-*To install the emulator and test your Lambda function*
-
-1) From your project directory, run the following command to download the RIE from GitHub and install it on your local machine.
+To test Lambda functions with the Runtime Interface Client, use the [AWS Lambda Runtime Interface Emulator (RIE)](https://github.com/aws/aws-lambda-runtime-interface-emulator). To test your local changes with RIE (Runtime Interface Emulator):
 
 ```shell script
-mkdir -p ~/.aws-lambda-rie && \
-    curl -Lo ~/.aws-lambda-rie/aws-lambda-rie https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie && \
-    chmod +x ~/.aws-lambda-rie/aws-lambda-rie
+# Build your current code (do this when you make changes) 
+# We build on a linux machine to ensure native build dependencies are met
+make build-container
+# Or with poetry:
+poetry run build-container
+
+# Test with RIE (fast, repeatable)
+make test-rie
+# Or with poetry:
+poetry run test-rie
+
+# Test the function
+curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{"message":"test"}'
 ```
-2) Run your Lambda image function using the docker run command.
-
-```shell script
-docker run -d -v ~/.aws-lambda-rie:/aws-lambda -p 9000:8080 \
-    --entrypoint /aws-lambda/aws-lambda-rie \
-    myfunction:latest \
-        /usr/local/bin/python -m awslambdaric app.handler
-```
-
-This runs the image as a container and starts up an endpoint locally at `http://localhost:9000/2015-03-31/functions/function/invocations`.
-
-3) Post an event to the following endpoint using a curl command:
-
-```shell script
-curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
-```
-
-This command invokes the function running in the container image and returns a response.
-
-*Alternately, you can also include RIE as a part of your base image. See the AWS documentation on how to [Build RIE into your base image](https://docs.aws.amazon.com/lambda/latest/dg/images-test.html#images-test-alternative).*
-
 
 ## Development
 
@@ -145,7 +150,6 @@ Clone this repository and run:
 make init
 make build
 ```
-
 ### Running tests
 
 Make sure the project is built:

pyproject.toml

@@ -29,7 +29,8 @@ lint = "scripts.dev:lint"
 format = "scripts.dev:format_code"
 clean = "scripts.dev:clean"
 build = "scripts.dev:build"
-local-test = "scripts.dev:local_test"
+build-container = "scripts.dev:build_container"
+test-rie = "scripts.dev:test_rie"
 
 [build-system]
 requires = ["poetry-core>=2.0.0,<3.0.0", "setuptools>=68", "wheel"]

scripts/build-container.sh

@@ -0,0 +1,25 @@
+#!/bin/bash
+set -euo pipefail
+
+TAG=${TAG:-latest}
+PYTHON_VERSION=${PYTHON_VERSION:-3.9}
+
+echo "Building awslambdaric wheel for Python ${PYTHON_VERSION}..."
+
+echo "Building wheel in container..."
+docker build \
+    --build-arg PYTHON_VERSION="${PYTHON_VERSION}" \
+    -f Dockerfile.build \
+    -t "awslambdaric-builder:${TAG}" \
+    .
+
+echo "Extracting built wheel..."
+mkdir -p build-artifacts
+
+docker run --rm -v $(pwd)/build-artifacts:/output awslambdaric-builder:${TAG} /bin/sh -c "
+    cp /home/build/dist/*.whl /output/
+    echo 'Wheel copied to build-artifacts/'
+    ls -la /output/
+"
+
+echo "Build complete! Wheel is available in build-artifacts/"

scripts/dev.py

@@ -62,10 +62,20 @@ def build():
     run([sys.executable, "setup.py", "sdist", "bdist_wheel"], env=env)
 
 
+def build_container():
+    print("Building awslambdaric wheel in container")
+    run(["./scripts/build-container.sh"])
+
+
+def test_rie():
+    print("Testing with RIE using pre-built wheel")
+    run(["./scripts/test-rie.sh"])
+
+
 def main():
     parser = argparse.ArgumentParser(description="Development scripts")
     parser.add_argument("command", choices=[
-        "init", "test", "lint", "format", "clean", "build"
+        "init", "test", "lint", "format", "clean", "build", "build-container", "test-rie"
     ])
 
     args = parser.parse_args()
@@ -77,7 +87,8 @@ def main():
         "format": format_code,
         "clean": clean,
         "build": build,
-
+        "build-container": build_container,
+        "test-rie": test_rie,
     }
 
     command_map[args.command]()

scripts/test-rie.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+set -euo pipefail
+
+DOCKERFILE="Dockerfile.rie"
+
+echo "Starting RIE test setup..."
+
+# Check if build artifacts exist
+if [ ! -d "build-artifacts" ] || [ -z "$(ls -A build-artifacts/*.whl 2>/dev/null)" ]; then
+    echo "No build artifacts found. Please run 'make build-container' first."
+    exit 1
+fi
+
+echo "Building test Docker image..."
+docker build \
+    -f "${DOCKERFILE}" \
+    -t awslambdaric-rie-test .
+
+echo "Starting test container on port 9000..."
+echo ""
+echo "Test with:"
+echo "curl -XPOST \"http://localhost:9000/2015-03-31/functions/function/invocations\" -d '{\"message\":\"test\"}'"
+echo ""
+
+docker run -it -p 9000:8080 \
+    --rm awslambdaric-rie-test