Change-the-OO-customer-managed-key-for-EFS-file-system-and-RDS_688982917

Note: This wiki is only used for OO

Prerequisite

Create 2 KMS customer-managed keys, one for EFS, and the other one for RDS.
Please refer to https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html
1. Sign in to the AWS Management Console and open the AWS Key Management Service (AWS KMS) console at https://console.aws.amazon.com/kms.
2. To change the AWS Region, use the Region selector in the upper-right corner of the page.
3. In the navigation pane, choose Customer managed keys.
4. Choose Create key.
5. To create a symmetric encryption KMS key, for Key type choose Symmetric.
6. In Key usage, the Encrypt and decrypt option is selected for you.
7. In Advanced options, you can import key material from you key management infratructure into AWS KMS. In Regionality, please choose Multi-Region key. https://docs.aws.amazon.com/kms/latest/developerguide/importing-keys-conceptual.html
8. Choose Next.
9. Type an alias for the KMS key,like "saas-efs-key". The alias name cannot begin with aws/. The aws/ prefix is reserved by Amazon Web Services to represent AWS managed keys in your account.
Repeat the above steps to create a key for rds, like "saas-rds-key".
1. Create the EFS replication
Please refer to https://docs.aws.amazon.com/efs/latest/ug/create-replication.html#create-replication-new
1. Sign in to the AWS Management Console and open the Amazon EFS console at https://console.aws.amazon.com/efs/.
2. Open the file system that you want to replicate: 1. In the left navigation pane, choose File systems. 2. In the File systems list, choose the file system that you want to replicate. The file system that you choose cannot be a source or destination file system in an existing replication configuration.
3. Choose the Replication tab.
4. In the Replication section, choose Create replication.
5. In the Replication settings section, define the replication settings: 1. For Replication configuration, choose Replicate to a new file system. 2. For Destination AWS Region, choose the AWS Region in which to replicate the file system.
6. In the Destination file system settings section, define the destination file system settings. 1. For File system type, choose choose Regional. 2. For Encryption,choose the KMS key like "saas-efs-key".
1. Mount the destination file system:https://docs.aws.amazon.com/efs/latest/ug/accessing-fs.html.
2. Sign in to the AWS Management Console and open the Amazon EFS console at https://console.aws.amazon.com/efs/. 2. In the left navigation pane, choose File systems. The File systems page displays the EFS file systems in your account. 3. Choose the file system that you want to manage mount targets for by choosing its Name or the File system ID to display the file system details page. 4. Choose Network, click Create mount target. 5. In Network, select your VPC. 6. In Mount targets, select the private subnet id and the EFS Security groups one by on
3. Deploy the Amazon EFS CSI driver to your Amazon EKS cluster. https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

Configure the bastion node. If you already have a bastion node, skip this step.

# Access to the bastion node.  such as 'i-0da6195baed41d3d8'.
# Optional. Make the EC2 as a real bastion node. You have to install OMT capabilities named 'Tools'.
./install --capabilities ClusterManagement=false,DeploymentManagement=false,LogCollection=false,Monitoring=false,MonitoringContent=false,NfsProvisioner=false,Tools=true,K8sBackup=false
source ~/.bashrc
# Make sure $CDF_HOME printing '/root/cdf'.
echo $CDF_HOME
# Install binary kubectl.
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
cp kubectl /usr/bin/
# update kubeconfig
export AWS_ACCESS_KEY_ID= xxx 
export AWS_SECRET_ACCESS_KEY= xxx
export AWS_SESSION_TOKEN= xxx
export AWS_DEFAULT_REGION="us-west-2"
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update
export PATH=/usr/local/bin:$PATH
aws sts get-caller-identity
aws eks update-kubeconfig --name encrypt-0-cluster
# Verify that we can access to the k8s cluster.
kubectl get ns
# Download OMT_External_K8s_24.4-270.zip on the bastion node.
wget https://orgartifactory.swinfra.net/artifactory/itom-buildoutput/cdf-daily-build/24.4-byok/OMT_External_K8s_24.4-270.zip
unzip 24.4-byok/OMT_External_K8s_24.4-270.zip

Maintain Window

On the bastion node, run the following command to stop OO. ( SMAX can either be stopped or running )

cd ${CDF_HOME}/scripts
./cdfctl.sh runlevel set -l DOWN -n ${OO_NAMESPACE}
# Make sure there is no Running pods under the namespaces.

Create a DB snapshot

Please refer to https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ManagingManualBackups.html
Copy the DB snapshot (from step 2) for Amazon RDS

Please refer to https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CopySnapshot.html

Master key: select your created customer key for RDS
Rename the source DB instance to a new one. Please refer to https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RenameInstance.html
Restore to a new DB instance using the same instance name as source DB

Please refer to https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html

Make sure that only the kms key is diffent, other configurations are the same. For example, option group，size， multi az and so on.
Ensure all the data has been synced from the source EFS to the target EFS. Then delete the EFS Replication.
1. Create a marker file on the source EFS. 2. Wait for the marker file to appear on the target EFS. 3. Delete the EFS Replication.

Recreate PVs and PVCs using the new EFS

# Recreate PV according to yaml files generated during Prerequisite
# Make sure there is no Running pods under the namespaces. # get modifyEFS.sh script, please check attachment.
chmod +x modifyEFS
.sh./modifyEFS.sh --help # to get all the available options for this script
# The result will look like this:
#Options:
#  modify       Change the EFS ID and recreate PersistentVolumes (PV) and PersistentVolumeClaims (PVC).
#  restore      Restore the PersistentVolumes (PV) and PersistentVolumeClaims (PVC) from backups.
#Examples:
#  ./modifyEFS.sh modify
#  ./modifyEFS.sh restore
./modifyEFS.sh modify# Once you run this, a menu will appear for all the required values to be filled in#for example:Please enter the namespace: oo-helmPlease enter the new EFS name/id: fs-07a0b7d3308a0dbdf

modifyEFS.sh

Running this script will do the following:

1. stop the OO pods ( if running ) 2. recreate PVs and PVCs with the new EFS id/DNS name Note If the current env is NOT encrypted in transit (NOT using the csi driver) then, for "new efs id" please insert the DNS name (like: fs-06d7d8ae861f5xxxx.efs.us-west-2.amazonaws.com)
  If the current env IS encrypted in transit (using the csi driver) then, for "new efs id" please insert only the EFS ID (like: fs-06d7d8ae861f5xxxx) 3. start the OO pods

The script returns these messages upon completion:

[INFO] OO started successfully with the original replica counts.

[INFO] EFS has been modified successfully.

8. Verify the k8s cluster is running.

1. Check all the pods are in a Running state.
```
kubectl get pods -n ${OO_NAMESPACE}
```
  1. Login to SMAX and access OO through the portal.

9. Modify all the ec2 instances/bastion mounts(/etc/fstab) which mount source EFS to the new one ( if not already done as part of SMAX PV/PVC recreation )

10. Wait a couple of days to make sure that the transition to customer managed key is working smoothly, and then delete AWS old EFS and old RDS.

Rollback

If you meet any issues during step '7. Recreate PVs and PVCs using the new EFS', you can rollback to the original EFS.

```
 ./modifyEFS.sh restore
```

9.3 KiB Raw Blame History Unescape Escape

Change-the-OO-customer-managed-key-for-EFS-file-system-and-RDS_688982917

Note: This wiki is only used for OO

Prerequisite

Maintain Window

Rollback

9.3 KiB

Raw Blame History