Encryption of Kubernetes Persistent Local Volumes

Dario Nascimento
3 min readJun 7, 2018

--

Data can be encrypted at rest at filesystem or on block device level. At file system level, e.g., eCryptfs, a stacked file system appends metadata to each file that is used to decrypt the file. At block level, the encryption is transparent and faster.

Kubernetes Local Persistent Volumes allows to manage local block storages as we would manage AWS EBS disks in Kubernetes. We will create multiple virtual block devices, or loopback devices, in the existing file system, encrypt this volumes using aes-xtx-plain64 with password hash sha256 and make them available as persistent storage volumes.

function create_disk {
dd bs=512 count=4 if=/dev/urandom of=/disk1-key
chmod 600 /disk1-key
fallocate -l $((4*1024*1024*1024)) /disk1
cat /disk$1-key | cryptsetup -q luksFormat --hash sha256 /disk1 -d -
cryptsetup luksOpen /disk1 device-disk-1 — key-file /disk1-key
mkfs.ext4 /dev/mapper/device-disk-1
mkdir /data/disk1
mount /dev/mapper/device-disk-1 /data/disk1
}

This snippet stores a random password in /disk1-key , creates a 4GB file is /disk1, creates the loopback device /dev/mapper/device-disk-1 and mounts as /data/disk1. All content in this device is stored encrypted in the file. Yet, the disk is accessible while mounted. You still need access control to prevent disk access.

The next step is to create a local persistent volume:

# Only create this for K8s 1.9+
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-disk
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
# Supported policies: Delete, Retain
reclaimPolicy: Delete

We can create the persistent volumes manually or use a provisioner:

---
# Source: provisioner/templates/provisioner.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: local-provisioner-config
namespace: kube-system
data:
storageClassMap: |
local-disk:
hostDir: /data
mountDir: /data
blockCleanerCommand:
- "/scripts/shred.sh"
- "2"
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: local-volume-provisioner
namespace: kube-system
labels:
app: local-volume-provisioner
spec:
selector:
matchLabels:
app: local-volume-provisioner
template:
metadata:
labels:
app: local-volume-provisioner
spec:
serviceAccountName: local-storage-admin
containers:
- image: "quay.io/external_storage/local-volume-provisioner:v2.1.0"
imagePullPolicy: "Always"
name: provisioner
securityContext:
privileged: true
env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: MY_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: JOB_CONTAINER_IMAGE
value: "quay.io/external_storage/local-volume-provisioner:v2.1.0"
volumeMounts:
- mountPath: /etc/provisioner/config
name: provisioner-config
readOnly: true
- mountPath: /data
name: local-disk
mountPropagation: "HostToContainer"
volumes:
- name: provisioner-config
configMap:
name: local-provisioner-config
- name: local-disk
hostPath:
path: /data
---
# Source: provisioner/templates/provisioner-service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: local-storage-admin
namespace: kube-system
---
# Source: provisioner/templates/provisioner-cluster-role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: local-storage-provisioner-pv-binding
namespace: kube-system
subjects:
- kind: ServiceAccount
name: local-storage-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: system:persistent-volume-provisioner
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: local-storage-provisioner-node-clusterrole
namespace: kube-system
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: local-storage-provisioner-node-binding
namespace: kube-system
subjects:
- kind: ServiceAccount
name: local-storage-admin
namespace: kube-system
roleRef:
kind: ClusterRole
name: local-storage-provisioner-node-clusterrole
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jobsrole
namespace: kube-system
rules:
- apiGroups:
- 'batch'
resources:
- jobs
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: local-storage-provisioner-job-binding
namespace: kube-system
subjects:
- kind: ServiceAccount
name: local-storage-admin
namespace: kube-system
roleRef:
kind: Role
name: jobsrole
apiGroup: rbac.authorization.k8s.io

This will create a persistent volume for each volume in the dir /data.

I expect the next the next step to be kubelet to mount/unmount local persistent volumes with a password.

--

--

Dario Nascimento
Dario Nascimento

Written by Dario Nascimento

DevOps & Security @CommBank (Commonwealth Bank Australia)

No responses yet