csi interface for eks by terraform

It has been a while since my last post, I was busy these last couples of weeks; meanwhile I’ve learned how to implement a csi interface on eks (aws k8s) and it’s pretty interesting on AWS by EKS service, thus I though it can be useful this content into a post.

Requirements:

You will need your eks already running.

I’ll point each tools, techs, etc I used on this procedure:

Cloud: AWS
Service: EKS
tool: Kustomize
cmd_tool: eksctl
cmd_api: kubectl
IaC: Terraform
Repo: Gitlab

A summary of the steps we will follow:

1 create .json policy. - Terraform
2 create irsa file entry. - Terraform
3 bind and create SA - eksctl
4 create the csi objects - from repo kustomize

Terraform

1 create .json policy.

You can find the policy right here and paste it onto your Terraform folder.

2 create irsa file entry.

Add the following file to your Terraform folder.

module "irsa-aws-efs-csi-controller" {
  source                        = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
  version                       = "~> 4"
  create_role                   = true
  role_name                     = "aws-efs-csi-controller"
  provider_url                  = replace(module.eks.cluster_oidc_issuer_url, "https://", "")
  role_policy_arns              = [aws_iam_policy.aws-efs-csi-controller.arn]
  oidc_fully_qualified_subjects = ["system:serviceaccount:kube-system:aws-efs-csi-controller"]
}

resource "aws_iam_policy" "aws-efs-csi-controller" {
  name_prefix = "aws-efs-csi-controller"
  description = "EKS aws-efs-csi-controller policy for cluster ${module.eks.cluster_id}"
  policy      = file("aws-efs-csi-controller-policy.json")
  path        = "/"
}

3 create SA and bind role on

eksctl create iamserviceaccount \
--cluster=<cluster-name> \
--region <your-region> \
--namespace=kube-system \
--name=efs-csi-controller-sa \
--override-existing-serviceaccounts \ # delete if you don't need it
--attach-policy-arn=<arn-policy> \ # your arn policy created by TF
--approve

4 create csi objects.

Using your credentials to access the eks cluster by kubectl, create a folder and copy and paste the following kustomize yaml file.

note: below as you can see there I use a “images” entry as patching method where it will parse each image and replaces them by “newName” value, there are storaged on ECR if do you want just pull these image from amazon official site (not recommended on PRO) you can delete the “images”; thus you can chose a) or b).

a) Kustomize file (patching method):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
  - git@gitlab.com/castrillo/k8s-olimpo
images:
  - name: amazon/aws-efs-csi-driver:v1.3.4
    newName: <>/path/aws-efs-csi-driver
  - name: public.ecr.aws/eks-distro/kubernetes-csi/csi-provisioner:v3.1.0
    newName: <>/path/csi-provisioner
  - name: public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.5.0
    newName: <>/path/livenessprobe
  - name: public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.5.0
    newName: <>/path/csi-node-driver-registrar

b) Kustomize file (no patching method):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
  - git@gitlab.com/castrillo/k8s-olimpo

Check out the diff plan and apply:

kustomize build . | kubectl diff -f -
kustomize build . | kubectl apply -f -

At the end you will see something like:

$ kubectl get all -l app.kubernetes.io/name=aws-efs-csi-driver -n kube-system
NAME READY STATUS RESTARTS AGE
pod/efs-csi-controller-589f98dc58-4qbrw 3/3 Running 0 6m26s
pod/efs-csi-controller-589f98dc58-mcbsg 3/3 Running 0 6m26s
pod/efs-csi-node-2njd7 3/3 Running 0 18m
pod/efs-csi-node-4cvjb 3/3 Running 0 18m
pod/efs-csi-node-9j874 3/3 Running 0 18m
pod/efs-csi-node-9pkjw 3/3 Running 0 18m
pod/efs-csi-node-bjm2b 3/3 Running 0 18m

NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/efs-csi-node 22 22 22 22 22 beta.kubernetes.io/os=linux 18m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/efs-csi-controller 2/2 2 2 18m

NAME DESIRED CURRENT READY AGE
replicaset.apps/efs-csi-controller-589f98dc58 2 2 2 18m

On this part we have everything done, if do you need test the driver go through the examples shown from official repo eg multiple_pods

official docu

Terraform#

Terraform