Add second kms list entry when SOPS file created

[thecosmicfrog]

Hi folks. My approach to allowing users to create secrets, and then for CI to decrypt and use those secrets, is to allow users to create encrypted files using the sops CLI.

Users are auth'd using AWS SSO with an AWS CLI profile called development.

My creation rules in .sops.yaml look like this:

creation_rules:
    - kms: arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key
      aws_profile: development

So when the encrypted file (say, secrets.yaml) is created, it looks like this:

username: ENC[...]
password: ENC[...]
sops:
    kms:
        - arn: arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key
          created_at: "1970-01-01T00:00:00Z"
          enc: ABCDEF
          aws_profile: development

This is great, but I also need my CI system to decrypt these files. The CI server has an AWS IAM role attached (let's call it CIServerRole) with permissions to access the KMS key. My solution at the moment is to manually add another kms list entry like this:

[...]
sops:
    kms:
        - arn: arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key
          created_at: "1970-01-01T00:00:00Z"
          enc: ABCDEF
          aws_profile: development
+       # Added manually after file has been encrypted.
+       - arn: arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key
+         created_at: "1970-01-01T00:00:00Z"
+         enc: ABCDEF
+         aws_profile: ""
+         role: arn:aws:iam::123456789123:role/CIServerRole

I then commit and push this, and the CI server can happily decrypt the file using that IAM role.

I'd rather avoid having to add this manually, due to the toil and possibility of human error. I'd like the sops CLI to automatically populate this second "use a role, not a profile" entry to the kms list. Is this possible using .sops.yaml?

Thanks, and hugely appreciate the awesome work on SOPS!

Aaron

[snovikov]

I think before v3.9.0 you could specify key_groups for that, like

creation_rules:
    - key_groups:
          - kms:
                - arn: "arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key"
                  aws_profile: development
                - arn: "arn:aws:kms:us-east-1:123456789123:alias/sops-kms-key"
                  role: arn:aws:iam::123456789123:role/CIServerRole

But since #1493 change was introduced - that does not work anymore

[benlangfeld]

Indeed, #1493 broke absolutely necessary behaviour of SOPS. Can it please be reverted? /cc @jonasbadstuebner @felixfontein

[felixfontein]

It definitely will not get reverted. Bugs should be fixed, but simply reverting a feature because of one unwanted side-effect is a terrible idea.

Could you please explain in more detail what exactly stopped working? I do not have access to AWS KMS so I cannot use the above example to reproduce it. My guess is that deduplication is removing the second key; this also came up during the review of #1493: #1493 (comment) (but unfortunately nobody with AWS knowledge commented there).

[benlangfeld]

When using KMS, it is often necessary to list the same key multiple times with different parameters, like role as documented above. Deduplication should not be based only on key, but on the full set of parameters.

[felixfontein]

I created a potential fix in #1733. This likely also affects the rotate, updatekey, and publish commands, which so far only compared the ARN and ignored everything else. Would be glad if someone could test this!