Skip to content

[Issue]: Node HTTPS POSTs to AWS STS get an empty 400 from the proxy, breaking configure-aws-credentials (curl and botocore unaffected) #14

Description

@iankyslytsya

Repository

Quivo-GmbH/shared-workflows (reproduction run in the private repo Quivo-GmbH/aws-account-security)

Adoption path

Firewall enabled label for standard runners (ubuntu-24.04-firewall)

What happened?

On ubuntu-24.04-firewall, Node-issued HTTPS POSTs to the AWS STS endpoint receive an empty HTTP 400 from the egress proxy. The same request from curl or from Python/botocore on the same runner succeeds, and Node POSTs to other hosts on the same runner behave normally.

This breaks aws-actions/configure-aws-credentials completely, so any job that assumes an AWS role via OIDC fails 100% of the time on this runner. For us that was every nightly Terraform drift detection job across 12 repositories, all failing on the first night after we adopted the label.

The user-visible error is misleading, because the AWS SDK cannot deserialize an empty 400 and the action then mishandles the resulting error object:

##[error]Could not assume role with OIDC: Cannot set properties of undefined (setting 'message')
  Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object.

Calling @aws-sdk/client-sts directly and dumping error.$response shows what actually arrives:

name: Unknown
message: Unknown
statusCode: 400
headers: {"connection":"close","content-length":"0","date":"Sat, 01 Aug 2026 08:20:00 GMT"}
$metadata: {"httpStatusCode":400,"attempts":1,"totalRetryDelay":0}

Measured matrix, identical job, role and OIDC token, ubuntu-22.04 as control:

Client Request ubuntu-22.04 ubuntu-24.04-firewall
curl (h2, h1.1, chunked, Expect: 100-continue, amz-sdk headers, custom UA) POST sts.eu-west-1.amazonaws.com 200 200
aws CLI (python/botocore) AssumeRoleWithWebIdentity success success
Node https.request POST sts.eu-west-1.amazonaws.com 403 (genuine AWS error) 400, 0 bytes
Node fetch (undici) POST sts.eu-west-1.amazonaws.com 403 (genuine AWS error) 400, 0 bytes
Node https.request POST api.github.com/graphql 403 403
Node https.request POST example.com 405 405
Node https.get GET sts.eu-west-1.amazonaws.com 302 302

So it is not host reachability and not the request method on its own. It reproduces only for a Node HTTP client POSTing to the STS host. Every curl variation I tried against the same host succeeded, including forcing HTTP/1.1, chunked transfer encoding, Expect: 100-continue, and adding the SDK's amz-sdk-invocation-id / amz-sdk-request headers and user-agent, so I could not reproduce it from curl at all.

Two things I ruled out, in case they save you time:

  • Not TLS trust. The mkcert development CA root is present at /usr/local/share/ca-certificates/, plain Node TLS to the same host works (the GET returns 302 normally), and setting NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt changes nothing.
  • Not the 24.04 image. OIDC succeeds on plain ubuntu-24.04. Only the -firewall label fails.

Worth flagging on the docs side: the guide describes audit mode as recording traffic without blocking anything, and that is what we relied on when adopting the label. Audit mode does appear to pass traffic, but this proxy behaviour still takes a job down hard, so "audit mode cannot break a workflow" is not currently a safe assumption for anyone using a Node-based action that talks to STS.

What did you expect to happen?

The Node-issued POST to sts.<region>.amazonaws.com should reach AWS and return the genuine STS response, the same way curl and botocore do on the same runner, and the same way Node does on ubuntu-22.04.

Steps to reproduce

jobs:
  probe:
    strategy:
      fail-fast: false
      matrix:
        runner: [ubuntu-22.04, ubuntu-24.04-firewall]
    runs-on: ${{ matrix.runner }}
    permissions:
      contents: read
      id-token: write
    steps:
      # Fails only on ubuntu-24.04-firewall
      - uses: aws-actions/configure-aws-credentials@v6.0.0
        continue-on-error: true
        with:
          role-to-assume: arn:aws:iam::<account>:role/<role>
          aws-region: eu-west-1

      # Minimal client comparison, no AWS credentials needed.
      # curl -> 403 from AWS on both runners.
      # node  -> 403 on ubuntu-22.04, empty 400 on ubuntu-24.04-firewall.
      - run: |
          BODY='Action=GetSessionToken&Version=2011-06-15'
          curl -sS -o /dev/null -w 'curl http=%{http_code}\n' -X POST \
            https://sts.eu-west-1.amazonaws.com/ \
            -H 'Content-Type: application/x-www-form-urlencoded' --data-binary "$BODY"
          node -e '
            const https=require("https"),b="Action=GetSessionToken&Version=2011-06-15";
            const r=https.request({host:"sts.eu-west-1.amazonaws.com",path:"/",method:"POST",
              headers:{"content-type":"application/x-www-form-urlencoded","content-length":Buffer.byteLength(b)}},
              x=>{let d="";x.on("data",c=>d+=c);x.on("end",()=>console.log("node http=",x.statusCode,"bytes=",d.length))});
            r.on("error",e=>console.log("node ERR",e.code)); r.end(b);'

Additional context

Runner image reported Current runner version: 2.331.0, Node v22.23.1, @aws-sdk/client-sts 3.1101.0. Region eu-west-1. No HTTP_PROXY/HTTPS_PROXY variables are set in the job environment, and nothing about the firewall appears in the job log, so from a workflow author's point of view the failure has no attribution back to the firewall at all. Surfacing proxy-generated 4xx responses in the run summary would have made this much faster to diagnose.

Account identifiers and role ARNs have been redacted. Happy to re-run any variation of the probe if it helps narrow it down.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions