I’m trying to deploy the Kubernetes operator and integrate AWS secrets manager. When I do this in a deployment, I use the secrets-store.csi.k8s.io driver to mount the secret as a volume like below:
apiVersion: apps/v1
kind: Deployment
metadata:
name: pgadmin
namespace: {{ .Values.namespace }}
labels:
app: pgadmin
version: v1
spec:
revisionHistoryLimit: 0
selector:
matchLabels:
app: pgadmin
replicas: 1
template:
metadata:
labels:
app: pgadmin
istio_version: {{ .Values.istioVersion }}
spec:
serviceAccountName: pgadmin-sa
automountServiceAccountToken: false
containers:
- name: pgadmin
image: {{ .Values.pgAdmin.imageRepository }}:{{ .Values.pgAdmin.imageVersion }}
imagePullPolicy: IfNotPresent
env:
- name: PGADMIN_DEFAULT_EMAIL
value: {{ .Values.pgAdmin.defaultEmail }}
- name: PGADMIN_DEFAULT_PASSWORD
valueFrom:
secretKeyRef:
name: pgadmin-credentials
key: password
- name: PGADMIN_LISTEN_PORT
value: "8080"
resources:
limits:
cpu: 100m
memory: 256Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- name: pgadmin-secret
mountPath: /mnt/secrets-store
- name: pgadmin-data
mountPath: /var/lib/pgadmin
volumes:
- name: pgadmin-secret
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "pgadmin-secret-spc"
- name: pgadmin-data
emptyDir: {}
The service account called out in deployment.spec.template.spec is associated to a role which has the required policy to fetch the secret from AWS secrets manager.
I’m trying to accomplish the same thing inside the operator, so that I can use the AWS secret as the user’s password that is setup as part of the operator. The operator deployment as it stands now looks like this:
apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
name: mongo-db
namespace: {{ .Values.namespace }}
spec:
members: 3
type: ReplicaSet
version: "6.4.0"
security:
authentication:
modes: ["SCRAM"]
users:
- name: my-admin
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: mongodb-credentials
key: password
roles:
- name: root
db: admin
scramCredentialsSecretName: my-admin-scram
- name: my-user
db: admin
passwordSecretRef: # a reference to the secret that will be used to generate the user's password
name: mongodb-credentials
key: password
roles:
- name: readWriteAnyDatabase
db: admin
scramCredentialsSecretName: my-user-scram
I think I will still need the volumes setup in each pod and the volumeMounts in each container because I think that is how the CSI driver creates the Kubernetes secrets objects (but I’m not sure). I’m sure I will need to be able to run each pod with the following, in order for the pod to be able to access the secret. Otherwise I will get an authorization error:
serviceAccountName: pgadmin-sa
automountServiceAccountToken: false