feat: Add support for using ephemeral credentials for S3 artifacts. F…

…ixes #5446 (#5446)

Signed-off-by: Raymond Chow <[email protected]>

docs/configure-artifact-repository.md

@@ -75,12 +75,15 @@ artifacts:
 
 ## Configuring AWS S3
 
-Create your bucket and access keys for the bucket. AWS access keys have the same
-permissions as the user they are associated with. In particular, you cannot
-create access keys with reduced scope. If you want to limit the permissions for
-an access key, you will need to create a user with just the permissions you want
-to associate with the access key. Otherwise, you can just create an access key
-using your existing user account.
+First, create your bucket:
+
+```bash
+export mybucket=bucket249
+aws s3 mb s3://$mybucket [--region xxx]
+```
+
+Next, create a policy which will be saved locally.
+You will attach this to the relevant authentication method in one of the sections below.
 
 ```bash
 $ export mybucket=bucket249
@@ -107,43 +110,176 @@ $ cat > policy.json <<EOF
    ]
 }
 EOF
-$ aws s3 mb s3://$mybucket [--region xxx]
-$ aws iam create-user --user-name $mybucket-user
-$ aws iam put-user-policy --user-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
-$ aws iam create-access-key --user-name $mybucket-user > access-key.json
 ```
 
 If you do not have Artifact Garbage Collection configured, you should remove `s3:DeleteObject` from the list of Actions above.
 
-NOTE: if you want argo to figure out which region your buckets belong in, you
-must additionally set the following statement policy. Otherwise, you must
-specify a bucket region in your workflow configuration.
+!!! Note "Discovering bucket regions"
+    If you want Argo Workflows to figure out which region your buckets belong in, you must additionally set the following statement policy.
+    Otherwise, you must specify a bucket region in your workflow configuration.
 
 ```json
       {
-         "Effect":"Allow",
-         "Action":[
+        "Effect":"Allow",
+        "Action":[
             "s3:GetBucketLocation"
-         ],
-         "Resource":"arn:aws:s3:::*"
+        ],
+        "Resource":"arn:aws:s3:::*"
       }
     ...
 ```
 
 ### AWS S3 IRSA
 
-If you wish to use S3 IRSA instead of passing in an `accessKey` and `secretKey`, you need to annotate the service account of both the running workflow (in order to save logs/artifacts) and the argo-server pod (in order to retrieve the logs/artifacts).
+Using IRSA is the recommended Kubernetes' native mechanism.
+If you are using AWS EKS, see [the AWS documentation to enable it in your cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
+If you are not, see the [EKS Pod Identity Webhook instructions](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md).
+
+If you have created a bucket and policy as described above, you can then create an IAM role and add the policy with:
+
+```bash
+aws iam create-role --role-name $mybucket-role
+aws iam put-role-policy --role-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+```
+
+To use S3 IRSA, you need to annotate the service account of both the running workflow (in order to save logs/artifacts) and the argo-server pod (in order to retrieve the logs/artifacts).
+You can attach only one IAM role to a service account, so your roles must have permission for all buckets you may wish to use.
 
 ```yaml
 apiVersion: v1
 kind: ServiceAccount
 metadata:
   annotations:
-    eks.amazonaws.com/role-arn: arn:aws:iam::012345678901:role/mybucket
+    eks.amazonaws.com/role-arn: arn:aws:iam::012345678901:role/mybucket-role
   name: myserviceaccount
   namespace: mynamespace
 ```
 
+You can then use the service account in a workflow as below:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Workflow
+spec:
+  serviceAccountName: myserviceaccount
+```
+
+### AWS S3 with IAM user credentials
+
+AWS access keys have the same permissions as the user they are associated with.
+In particular, you cannot create access keys with reduced scope.
+If you want to limit the permissions for an access key, you will need to create a user with just the permissions you want to associate with the access key.
+Otherwise, you can just create an access key using your existing user account.
+
+If you have created a bucket and policy as described above, you can create an IAM user, add the policy and create a set of access keys with:
+
+```bash
+aws iam create-user --user-name $mybucket-user
+aws iam put-user-policy --user-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+aws iam create-access-key --user-name $mybucket-user > access-key.json
+```
+
+You can then use the access keys by configuring your artifact as below:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following secret entries are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey' and 'secretKey',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+```
+
+### AWS S3 with IAM role credentials
+
+You can use IAM role credentials for temporary access.
+
+If you have created a bucket and policy as described above, you can then create an IAM role and add the policy with:
+
+```bash
+aws iam create-role --role-name $mybucket-role
+aws iam put-role-policy --role-name $mybucket-user --policy-name $mybucket-policy --policy-document file://policy.json
+```
+
+You can then obtain the credentials set by running:
+
+```bash
+aws sts assume-role --role-arn rn:aws:iam::012345678901:role/$mybucket-role
+```
+
+After which you can use the credentials in your artifact configuration:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following secret entries are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey', 'secretKey', and 'sessionToken',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+      sessionTokenSecret:
+        name: my-s3-credentials
+        key: sessionToken
+```
+
+!!! Note "IAM role credentials are temporary"
+    IAM role credentials are temporary, so you must refresh them periodically via an external mechanism.
+    Argo Workflows will not refresh them for you.
+
+### AWS S3 Access Grants
+
+You can use [S3 Access Grants](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants.html) for temporary, reduced scope access.
+Read the [getting started guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-grants-get-started.html) to set this up in your AWS account.
+Once you have created an access grant and received a set of credentials, you can use them in your artifact configuration:
+
+```yaml
+artifacts:
+  - name: my-output-artifact
+    path: /my-output-artifact
+    s3:
+      endpoint: s3.amazonaws.com
+      bucket: my-s3-bucket
+      key: path/in/bucket/my-output-artifact.tgz
+      # The following secret entries are secret selectors.
+      # They reference the k8s secret named 'my-s3-credentials'.
+      # This secret is expected to have the keys 'accessKey', 'secretKey', and 'sessionToken',
+      # containing the base64 encoded credentials to the bucket.
+      accessKeySecret:
+        name: my-s3-credentials
+        key: accessKey
+      secretKeySecret:
+        name: my-s3-credentials
+        key: secretKey
+      sessionTokenSecret:
+        name: my-s3-credentials
+        key: sessionToken
+```
+
+!!! Note "S3 access grants are temporary"
+    S3 Access Grants are temporary, so you must refresh them periodically via an external mechanism.
+    Argo Workflows will not refresh them for you.
+
 ## Configuring GCS (Google Cloud Storage)
 
 Create a bucket from the GCP Console

docs/executor_swagger.md

@@ -3139,6 +3139,7 @@ cause implementors to also use a fixed point implementation.
 | region | string| `string` |  | | Region contains the optional bucket region |  |
 | roleARN | string| `string` |  | | RoleARN is the Amazon Resource Name (ARN) of the role to assume. |  |
 | secretKeySecret | [SecretKeySelector](#secret-key-selector)| `SecretKeySelector` |  | |  |  |
+| sessionTokenSecret | [SecretKeySelector](#secret-key-selector)| `SecretKeySelector` |  | |  |  |
 | useSDKCreds | boolean| `bool` |  | | UseSDKCreds tells the driver to figure out credentials based on sdk defaults. |  |
 
 

docs/fields.md

@@ -3526,6 +3526,7 @@ S3Artifact is the location of an S3 artifact
 |`region`|`string`|Region contains the optional bucket region|
 |`roleARN`|`string`|RoleARN is the Amazon Resource Name (ARN) of the role to assume.|
 |`secretKeySecret`|[`SecretKeySelector`](#secretkeyselector)|SecretKeySecret is the secret selector to the bucket's secret key|
+|`sessionTokenSecret`|[`SecretKeySelector`](#secretkeyselector)|SessionTokenSecret is used for credentials which are ephemeral (e.g. IAM assume role or S3 access grant)|
 |`useSDKCreds`|`boolean`|UseSDKCreds tells the driver to figure out credentials based on sdk defaults.|
 
 ## ValueFrom
@@ -4246,6 +4247,7 @@ S3ArtifactRepository defines the controller configuration for an S3 artifact rep
 |`region`|`string`|Region contains the optional bucket region|
 |`roleARN`|`string`|RoleARN is the Amazon Resource Name (ARN) of the role to assume.|
 |`secretKeySecret`|[`SecretKeySelector`](#secretkeyselector)|SecretKeySecret is the secret selector to the bucket's secret key|
+|`sessionTokenSecret`|[`SecretKeySelector`](#secretkeyselector)|SessionTokenSecret is used for credentials which are ephemeral (e.g. IAM assume role or S3 access grant)|
 |`useSDKCreds`|`boolean`|UseSDKCreds tells the driver to figure out credentials based on sdk defaults.|
 
 ## MutexHolding

go.mod

@@ -13,7 +13,7 @@ require (
 	github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible
 	github.com/aliyun/credentials-go v1.3.2
 	github.com/argoproj/argo-events v1.9.1
-	github.com/argoproj/pkg v0.13.7-0.20240208112602-3bb8fe9a0527
+	github.com/argoproj/pkg v0.13.7-0.20240704113442-a69fd34a8117
 	github.com/blushft/go-diagrams v0.0.0-20201006005127-c78c821223d9
 	github.com/colinmarc/hdfs/v2 v2.4.0
 	github.com/coreos/go-oidc/v3 v3.9.0
@@ -46,7 +46,7 @@ require (
 	github.com/sirupsen/logrus v1.9.3
 	github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966
 	github.com/soheilhy/cmux v0.1.5
-	github.com/spf13/cobra v1.8.0
+	github.com/spf13/cobra v1.8.1
 	github.com/spf13/pflag v1.0.5
 	github.com/spf13/viper v1.18.2
 	github.com/stretchr/testify v1.9.0
@@ -87,8 +87,8 @@ require (
 	dario.cat/mergo v1.0.0 // indirect
 	github.com/TylerBrock/colorjson v0.0.0-20200706003622-8a50f05110d2 // indirect
 	github.com/alibabacloud-go/debug v0.0.0-20190504072949-9472017b5c68 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.2 // indirect
 	github.com/cenkalti/backoff/v4 v4.2.1 // indirect
 	github.com/cloudflare/circl v1.3.7 // indirect
 	github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
@@ -165,25 +165,25 @@ require (
 	github.com/ajg/form v1.5.1 // indirect
 	github.com/andybalholm/brotli v1.0.4 // indirect
 	github.com/awalterschulze/gographviz v0.0.0-20200901124122-0eecad45bd71 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.24.1 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.26.6 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.16.16 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.7.3 // indirect
+	github.com/aws/aws-sdk-go-v2 v1.30.1 // indirect
+	github.com/aws/aws-sdk-go-v2/config v1.27.23 // indirect
+	github.com/aws/aws-sdk-go-v2/credentials v1.17.24 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.9 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.13 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.13 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ecr v1.17.8 // indirect
 	github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.13.8 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.18.7 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.26.7 // indirect
-	github.com/aws/smithy-go v1.19.0 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.15 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.22.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.30.1 // indirect
+	github.com/aws/smithy-go v1.20.3 // indirect
 	github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20220706184558-ce46abcd012b // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/chai2010/gettext-go v1.0.2 // indirect
 	github.com/chrismellard/docker-credential-acr-env v0.0.0-20220327082430-c57b701bfc08 // indirect
-	github.com/cpuguy83/go-md2man/v2 v2.0.3 // indirect
+	github.com/cpuguy83/go-md2man/v2 v2.0.4 // indirect
 	github.com/creack/pty v1.1.21
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/daviddengcn/go-colortext v1.0.0 // indirect