Connecting EKS Crossplane to GCP using workload identity federation

Introduction

The cloud native control plane framework.
Crossplane, a promising tool from CNCF, uses the power of Kubernetes to manage an organization’s entire infrastructure. By providing continuous reconciliation and declarative state management, it aims to streamline infrastructure provisioning and management.

Crossplane

Steps to setup GCP workload identity federation for Crossplane hosted on AWS EKS:

Steps to be performed at GCP end -

1. Enable the IAM, Resource Manager, Service Account Credentials, and Security Token Service APIs.
   Reference: Configure workload identity federation with AWS or Azure | IAM Documentation | Google Cloud
2. Create the crossplane service-account using the following gcloud command:

gcloud iam service-accounts create crossplane-sa \
--description="Master SA for child accounts deployment" \
--display-name="crossplane-sa"

3. Create the workload identity pool and provider:

Required Roles:
1. Workload Identity Pool Admin (roles/iam.workloadIdentityPoolAdmin)
2. Service Account Admin (roles/iam.serviceAccountAdmin)

4. Create a new workload identity pool:

gcloud iam workload-identity-pools create crossplane-aws-pool \
--location="global" \
--description="workload identity pool for crossplane deployed on AWS" \
--display-name="crossplane-aws-pool"

5. Add a workload identity pool provider:

gcloud iam workload-identity-pools providers create-aws aws-provider \
--location=”global” \
--workload-identity-pool=”crossplane-aws-pool” \
--account-id=”<aws-account-id>” \
--attribute-mapping="google.subject=assertion.arn,attribute.aws_role=assertion.arn.contains(‘assumed-role’) ? assertion.arn.extract(‘{account_arn}assumed-role/’) + ‘assumed-role/’ + assertion.arn.extract(‘assumed-role/{role_name}/’) : assertion.arn”

6. Authenticate a workload:
We must perform these steps once per workload.

Create a service account that represents the workload. It’s best to use a dedicated service account for each workload.
Grant the service account access to resources that you want external identities to access.

For example, In our case we created awx-gcp-crossplane service-account with Storage Admin access.

7. Allow the external workload to impersonate the service account:

To obtain the project number of your current project, execute the following command:

gcloud projects describe $(gcloud config get-value core/project) \
--format=value\(projectNumber\)

To grant the Workload Identity User role (roles/iam.workloadIdentityUser) to external identities that meet a certain criteria:

NOTE: Please replace <instance-profile-id> with the instance-profile name which is attached with EKS Node pool.

For example, In our case, the instance-profile-id attached with EKS Nodes is “eks-e8c690db-2357–2046–5f49–7682558395aa”.(Please refer the below attached snapshots)

In case of attached instance-profile:

gcloud iam service-accounts add-iam-policy-binding \
crossplane-sa@<PROJECT_ID>.iam.gserviceaccount.com \
--role=roles/iam.workloadIdentityUser \
--member="principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/crossplane-aws-pool/attribute.aws_role/arn:aws:sts::<aws-account-id>:assumed-role/<instance-profile-id>"

In case of attached role:

gcloud iam service-accounts add-iam-policy-binding \
crossplane-sa@<PROJECT_ID>.iam.gserviceaccount.com \
--role=roles/iam.workloadIdentityUser \
--member="principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/crossplane-aws-pool/attribute.aws_role/arn:aws:sts::<aws-account-id>:assumed-role/<role-id>"

8. Create a credential configuration:
To create a credential configuration file that lets the library obtain an access token from EC2 instance metadata, do the following:

gcloud iam workload-identity-pools create-cred-config \
projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/crossplane-aws-pool/providers/aws-provider \
--service-account=crossplane-sa@<PROJECT_ID>.iam.gserviceaccount.com \
--aws \
--output-file=auth.json

If you use AWS IMDSv2, an additional flag — enable-imdsv2 needs to be added to the gcloud iam workload-identity-pools create-cred-config command:

gcloud iam workload-identity-pools create-cred-config \
projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/crossplane-aws-pool/providers/aws-provider \
--service-account=crossplane-sa@<PROJECT_ID>.iam.gserviceaccount.com \
--aws \
--enable-imdsv2 \
--output-file=auth.json

If using the AWS metadata server isn’t an option, you can provide AWS security credentials through the following AWS environment variables:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Either of AWS_REGION or AWS_DEFAULT_REGION
Optional: AWS_SESSION_TOKEN

The gcloud CLI and libraries use these AWS environment variables when the AWS metadata server is unavailable.

Steps to be performed at AWS end -

1. Crossplane Setup:
   Reference: GCP Quickstart · Crossplane v1.14
2. Install the GCP provider:
   Install the provider into the Kubernetes cluster with a Kubernetes configuration file.

$ cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
 name: provider-gcp-storage
spec:
 package: xpkg.upbound.io/upbound/provider-gcp-storage:v0.41.0
EOF

The Crossplane Provider installs the Kubernetes Custom Resource Definitions (CRDs) representing GCP storage services. These CRDs allow you to create GCP resources directly inside Kubernetes.

3. Verify the provider installed with kubectl get providers.

$ kubectl get providers
NAME INSTALLED HEALTHY PACKAGE AGE
provider-gcp-storage True True xpkg.upbound.io/upbound/provider-gcp-storage:v0.41.0 36s
upbound-provider-family-gcp True True xpkg.upbound.io/upbound/provider-family-gcp:v0.41.0 29s

4. Create a Kubernetes secret for GCP:
The auth.json is the file that is generated in Step#8 above.

$ kubectl create secret \
generic gcp-secret \
-n crossplane-system \
--from-file=creds=./auth.json

View the secret with kubectl describe secret.

$ kubectl describe secret gcp-secret -n crossplane-system
Name: gcp-secret
Namespace: crossplane-system
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
creds: 2330 bytes

5. Create a ProviderConfig:

$ cat <<EOF | kubectl apply -f -
apiVersion: gcp.upbound.io/v1beta1
kind: ProviderConfig
metadata:
 name: root-gcp
spec:
 projectID: <PROJECT_ID>
 credentials:
 source: Secret
 secretRef:
 namespace: crossplane-system
 name: gcp-secret
 key: creds
EOF

This attaches the GCP credentials, saved as a Kubernetes secret, as a secretRef.

6. Create a managed resource:
A managed resource is anything Crossplane creates and manages outside of the Kubernetes cluster. This example creates a GCP storage bucket with Crossplane.

Create the Bucket with the following command:

$ cat <<EOF | kubectl create -f -
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
metadata:
 name: crossplane-bucket-aws
 labels:
 docs.crossplane.io/example: provider-gcp
spec:
 forProvider:
 location: US
 providerConfigRef:
 name: root-gcp
EOF

Use kubectl get bucket or kubectl get bucket.storage.gcp.upbound.io to verify Crossplane created the bucket.

$ kubectl get bucket/bucket.storage.gcp.upbound.io
NAME READY SYNCED EXTERNAL-NAME AGE
crossplane-bucket-aws True True crossplane-bucket-aws 2m2s

7. Delete the managed resource:

$ kubectl delete bucket/bucket.storage.gcp.upbound.io crossplane-bucket-aws
bucket.storage.gcp.upbound.io “crossplane-bucket-aws” deleted