ishenwei/nexus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b40abbcd473a7093d8261e212e3d6de97c1e516a
nexus/knowledgebase/csd-wiki/ICSD/AWS-RDS-certificate-update--Helm-Fedramp-simulation-ENV_688983269.md

11 KiB
Raw Blame History

AWS-RDS-certificate-update--Helm-Fedramp-simulation-ENV_688983269

Tasks

Products

Steps

Duration

Downtime

Prepare:

Certificate File Preparation

Download the new AWS RDS certificate bundle PEM file

Download the new AWS RDS certificate bundles for specific AWS region from the Certificate bundles for specific AWS Regions.

For region of us-gov-west-1, download the below certificate:

Upload the certificate bundle to the bastion.

Prepare:

Update certificate configuration in application side

OMT

1. Acquire database info before running the script:

You may get the db user, db name and PASSWORD_KEY values from database configmap with below commands:

kubectl get cm default-database-configmap -n -o yaml

The result is like:

DEFAULT_DB_CDFIDM_PASSWORD_KEY: defaultdb_cdfidm_user_password

DEFAULT_DB_CDFIDM_USERNAME: cdfidm

DEFAULT_DB_HOST: xxxxxyyyyy.us-west-2.rds.amazonaws.com

DEFAULT_DB_NAME: itom-cdf-idm

2. Get the cdfidm db password:

kubectl get pod -n $CDF_NAMESPACE | grep "itom-idm" | head -1 | awk '{print $1}'

kubectl exec -it -n $CDF_NAMESPACE -c idm -- get_secret

For example:

kubectl exec -it $(kubectl get pod -n $CDF_NAMESPACE | grep "itom-idm" | head -1 | awk '{print $1}') -n $CDF_NAMESPACE -c idm -- get_secret defaultdb_cdfidm_user_password

Note: Record the database info and password, they will be used in execute command

https://docs.microfocus.com/doc/SMAX/24.2/ModifyExternalDBConfig

SMAX & HCMX

  1. [Optional] If you don't remember the path of your custom my-values.yaml file, run the following command to get your yaml file.
    helm get values -n >
  2. Confirm if tlsEnabled is set to true and replace the caCertificates under "database" with the new PEM content directly or base64 all the content (e.g cat | base 64)
    Example:
    global:
    database:
    tlsEnabled: true
    tlsMode: verify-full # or use verify-ca
    ......
    database:
    caCertificates:
    pg_ca.crt: or PEM

    Note:

    If you are using base64 encoded content, make sure the encoded content is formatted in one line, as below: If you are not using base64 encoded content, please refer to below format:
  4. Run the command to check if job exist in ENV. It should return nothing. If job exist, make sure it's in Completed status and delete it.
    kubectl get job -n |grep -v NAME | awk '{print $1}'
    Run below command to get the pod status - if they are "Completed", it means the jobs are done and you may delete the jobs with 'delete job' command:
    kubectl get pods -n |grep job
    If above pods are all completed status, delete related jobs:
    kubectl delete job -n

NOTE: The yaml file with new pem content replaced will be used in RDS certificate replacement.Reference: https://staging.docs.microfocus.com/doc/SMAX/Main/ChangeCertForPostgreSQL

https://docs.microfocus.com/doc/SMAX/24.2/ModifyExternalDBConfig

CMS

  1. Get the CMS values.yaml from current running deployment by running below command

helm get values -n > values.yaml

2. Replace the content of caCertificates.postgresql.crt in values.yaml with the content of AWS RDS certificate bundle got at above step.

Note:every line of certificate content starts with 4 indentations in values.yaml, for example:

Audit

  1. Get the Audit values.yaml

helm get values -n > values.yaml

2. Replace the content of caCertificates.RE_ca_dbcrt in values.yaml with the content of AWS RDS certificate bundle got at above step.

Note: every line of certificate content starts with 4 indentations in values.yaml, for example:

Execute certificate update in application side

Note: There is no dependency on each application.

OMT

Navigate to the $CDF_HOME/bin directory, run the updateExternalDbInfo.sh script with below parameters:

./updateExternalDbInfo.sh -H -p -d -u --dbpassword --component itom-idm --cacert

For example:

./updateExternalDbInfo.sh -H xxxxyyyy.us-west-2.rds.amazonaws.com -p 5432 -d cdfidmdb -u cdfidm --dbpassword --component itom-idm --cacert /home/ssm-user/us-gov-west-1-bundle.pem

Reference: https://docs.microfocus.com/doc/OMT/24.2/ModifyExternalDatabaseConfiguration

1min

0

SMAX & HCMX

  1. Run the following commands to apply DB setting change. The yaml file is the one with new pem content replaced in preparation steps.
    helm upgrade -n -f my-values.yaml
    For example:
    helm upgrade itsma ESM_Helm_Chart-2x.x/charts/esm-1.0.0+2x.x-xxx.tgz -n itsma-xxyy -f values.yaml

  2. Run the following command to make sure that all SMAX pods are ready before next step.
    kubectl get pod -n < ESM_NAMESPACE > |grep -v 1/1|grep -v 2/2|grep -v 3/3|grep -v 4/4|grep -v Completed

  3. Restart the suite (Another option is to do the restart later via one command along with other products)
    $CDF_HOME/bin/cdfctl runlevel set -l DOWN -n
    Wait till all pods are down, then run below command to bring the application up
    $CDF_HOME/bin/cdfctl runlevel set -l UP -n

4mins

0

CMS

  1. Update the deployment by running helm upgrade command. The yaml file is the one with new pem content replaced in preparation steps.
    helm upgrade -n -f values.yaml
  2. Restart CMS (Another option is to do the restart later via one command along with other products)
    $CDF_HOME/bin/cdfctl runlevel set -l DOWN -n
    wait till all pods are shut down
    $CDF_HOME/bin/cdfctl runlevel set -l UP -n

NOTE: You may do this in parallel with SMAX restart

1min

0

Audit

  1. Update the deployment by running helm upgrade command. The yaml file is the one with new pem content replaced in preparation steps.
    helm upgrade -n -f values.yaml
  2. Restart Audit (Another option is to do the restart later via one command along with other products)

$CDF_HOME/bin/cdfctl runlevel set -l DOWN -n
wait till all pods are shut down
$CDF_HOME/bin/cdfctl runlevel set -l UP -n

NOTE: You may do this in parallel with SMAX restart

1min

0

Restart pods (Alternative)

You may also do the helm upgrade for all products in parallel without restarting.

Then do the restart against all products whose RDS certificates were changed

For example:

$CDF_HOME/bin/cdfctl runlevel set -l DOWN -n ,,,
wait till all pods are shut down and start the pods
$CDF_HOME/bin/cdfctl runlevel set -l UP -n ,,,

14mins

14mins

Monitor the restart till all pods are started

kubectl get pod -n < ESM_NAMESPACE > |grep -v 1/1|grep -v 2/2|grep -v 3/3|grep -v 4/4|grep -v Completed

Update the certificates of AWS RDS DB instances.

Update the certificate on AWS RDS DB instances.

1.Login AWS console, go to the RDS instances that you want to update the certificates.

2.Select the RDS instance, click modify button

3.Change the Certificate authority.

If your primary certificate CA is rds-ca-2019, it's recommended to select the rds-ca-rsa4096-g1 CA as new value

4.Save the change and select immediate effect.

5.Repeat the steps for all your RDS instances

2mins

0

Reference in New Issue View Git Blame Copy Permalink