# Change-the-OO-customer-managed-key-for-EFS-file-system-and-RDS_688982917
### Note: This wiki is only used for OO

### Prerequisite

1. Create 2 KMS customer-managed keys, one for EFS, and the other one for RDS.
	> Please refer to [https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html](https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html)
	> 
	> 1. Sign in to the AWS Management Console and open the AWS Key Management Service (AWS KMS) console at [https://console.aws.amazon.com/kms](https://console.aws.amazon.com/kms).
	> 2. To change the AWS Region, use the Region selector in the upper-right corner of the page.
	> 3. In the navigation pane, choose **Customer managed keys**.
	> 4. Choose **Create key**.
	> 5. To create a symmetric encryption KMS key, for **Key type** choose **Symmetric**.
	> 6. In **Key usage**, the **Encrypt and decrypt** option is selected for you.
	> 7. In **Advanced options,** you can import key material from you key management infratructure into AWS KMS. In **Regionality,** please choose **Multi-Region key. [https://docs.aws.amazon.com/kms/latest/developerguide/importing-keys-conceptual.html](https://docs.aws.amazon.com/kms/latest/developerguide/importing-keys-conceptual.html)**
	> 8. Choose **Next**.
	> 9. Type an alias for the KMS key,like "saas-efs-key". The alias name cannot begin with `aws/`. The `aws/` prefix is reserved by Amazon Web Services to represent AWS managed keys in your account.
	> 
	> Repeat the above steps to create a key for rds, like "saas-rds-key".
	2. Create the EFS replication
	> Please refer to [https://docs.aws.amazon.com/efs/latest/ug/create-replication.html#create-replication-new](https://docs.aws.amazon.com/efs/latest/ug/create-replication.html#create-replication-new)
	> 
	> ![](attachments/688982917/688982914.png)
	> 
	> 1. Sign in to the AWS Management Console and open the Amazon EFS console at [https://console.aws.amazon.com/efs/](https://console.aws.amazon.com/efs/).
	> 2. Open the file system that you want to replicate:
	> 	1. In the left navigation pane, choose **File systems**.
	> 		2. In the **File systems** list, choose the file system that you want to replicate. The file system that you choose cannot be a source or destination file system in an existing replication configuration.
	> 3. Choose the **Replication** tab.
	> 4. In the **Replication** section, choose **Create replication**.
	> 5. In the **Replication settings** section, define the replication settings:
	> 	1. For **Replication configuration**, choose **Replicate to a new file system**.
	> 		2. For **Destination AWS Region**, choose the AWS Region in which to replicate the file system.
	> 6. In the **Destination file system settings** section, define the destination file system settings.
	> 	1. For **File system type**, choose choose **Regional**.
	> 		2. For **Encryption**,choose the KMS key like "saas-efs-key".
	3. Mount the destination file system:[https://docs.aws.amazon.com/efs/latest/ug/accessing-fs.html](https://docs.aws.amazon.com/efs/latest/ug/accessing-fs.html).
	1. Sign in to the AWS Management Console and open the Amazon EFS console at [https://console.aws.amazon.com/efs/](https://console.aws.amazon.com/efs/).
			2. In the left navigation pane, choose **File systems**. The **File systems** page displays the EFS file systems in your account.
			3. Choose the file system that you want to manage mount targets for by choosing its **Name** or the **File system ID** to display the file system details page.
			4. Choose **Network,** click **Create mount target.**
			5. In **Network**, select your VPC.
			6. In **Mount targets**, select the private subnet id and the EFS Security groups one by on
	4. Deploy the Amazon EFS CSI driver to your Amazon EKS cluster. [https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html](https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html)
1. Configure the bastion node. If you already have a bastion node, skip this step.
	```shell
	# Access to the bastion node.  such as 'i-0da6195baed41d3d8'.
	# Optional. Make the EC2 as a real bastion node. You have to install OMT capabilities named 'Tools'.
	./install --capabilities ClusterManagement=false,DeploymentManagement=false,LogCollection=false,Monitoring=false,MonitoringContent=false,NfsProvisioner=false,Tools=true,K8sBackup=false
	source ~/.bashrc
	# Make sure $CDF_HOME printing '/root/cdf'.
	echo $CDF_HOME
	# Install binary kubectl.
	curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
	chmod +x kubectl
	cp kubectl /usr/bin/
	# update kubeconfig
	export AWS_ACCESS_KEY_ID= xxx 
	export AWS_SECRET_ACCESS_KEY= xxx
	export AWS_SESSION_TOKEN= xxx
	export AWS_DEFAULT_REGION="us-west-2"
	curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
	unzip awscliv2.zip
	sudo ./aws/install --update
	export PATH=/usr/local/bin:$PATH
	aws sts get-caller-identity
	aws eks update-kubeconfig --name encrypt-0-cluster
	# Verify that we can access to the k8s cluster.
	kubectl get ns
	# Download OMT_External_K8s_24.4-270.zip on the bastion node.
	wget https://orgartifactory.swinfra.net/artifactory/itom-buildoutput/cdf-daily-build/24.4-byok/OMT_External_K8s_24.4-270.zip
	unzip 24.4-byok/OMT_External_K8s_24.4-270.zip
	```

### Maintain Window

1. On the bastion node, run the following command to stop OO. ( SMAX can either be stopped or running )
	```shell
	cd ${CDF_HOME}/scripts
	./cdfctl.sh runlevel set -l DOWN -n ${OO_NAMESPACE}
	# Make sure there is no Running pods under the namespaces.
	```
2. Create a DB snapshot
	> Please refer to [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_ManagingManualBackups.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ManagingManualBackups.html)
	> 
	> ![](attachments/688982917/688982915.png)
3. Copy the DB snapshot (from step 2) for Amazon RDS
	> Please refer to [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_CopySnapshot.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CopySnapshot.html)
	> 
	> **Master key: select your created customer key for RDS**
4. Rename the source DB instance to a new one. Please refer to [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_RenameInstance.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RenameInstance.html)
5. Restore to a new DB instance using the same instance name as source DB
	> Please refer to [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_RestoreFromSnapshot.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html)
	> 
	> **Make sure that only the kms key is diffent, other configurations are the same. For example, option group，size， multi az and so on.**
6. Ensure all the data has been synced from the source EFS to the target EFS. Then delete the EFS Replication.
	1. Create a marker file on the source EFS.
		2. Wait for the marker file to appear on the target EFS.
		3. Delete the EFS Replication.
7. Recreate PVs and PVCs using the new EFS
	```shell
	# Recreate PV according to yaml files generated during Prerequisite
	# Make sure there is no Running pods under the namespaces. # get modifyEFS.sh script, please check attachment.
	chmod +x modifyEFS
	.sh./modifyEFS.sh --help # to get all the available options for this script
	# The result will look like this:
	#Options:
	#  modify       Change the EFS ID and recreate PersistentVolumes (PV) and PersistentVolumeClaims (PVC).
	#  restore      Restore the PersistentVolumes (PV) and PersistentVolumeClaims (PVC) from backups.
	#Examples:
	#  ./modifyEFS.sh modify
	#  ./modifyEFS.sh restore
	./modifyEFS.sh modify# Once you run this, a menu will appear for all the required values to be filled in#for example:Please enter the namespace: oo-helmPlease enter the new EFS name/id: fs-07a0b7d3308a0dbdf
	```
	[modifyEFS.sh](attachments/688982917/688982913.sh)

Running this script will do the following:

1. 1. stop the OO pods ( if running )
		2. recreate PVs and PVCs with the new EFS id/DNS name
		Note
		If the current env is **NOT** encrypted in transit (NOT using the csi driver) then, for "new efs id" please insert the DNS name (like: [fs-06d7d8ae861f5xxxx.efs.us-west-2.amazonaws.com](http://fs-06d7d8ae861f5xxxx.efs.us-west-2.amazonaws.com/))  
		If the current env **IS** encrypted in transit (using the csi driver) then, for "new efs id" please insert only the EFS ID (like: fs-06d7d8ae861f5xxxx)
		3. start the OO pods

The script returns these messages upon completion:

\[INFO\] OO started successfully with the original replica counts.

\[INFO\] EFS has been modified successfully.

8\. Verify the k8s cluster is running.

1. 1. Check all the pods are in a Running state.
		```shell
		kubectl get pods -n ${OO_NAMESPACE}
		```
		2. Login to SMAX and access OO through the portal.

9\. Modify all the ec2 instances/bastion mounts(/etc/fstab) which mount source EFS to the new one ( if not already done as part of SMAX PV/PVC recreation )

10\. Wait a couple of days to make sure that the transition to customer managed key is working smoothly, and then delete AWS old EFS and old RDS.

### Rollback

If you meet any issues during step '7. Recreate PVs and PVCs using the new EFS', you can rollback to the original EFS.

1. ```shell
	./modifyEFS.sh restore
	```
