nexus/knowledgebase/csd-wiki/ICSD/Configure-Nginx-through-network-load-balancer_688996474.md

Configure-Nginx-through-network-load-balancer_688996474

Create customer managed SMAX/CMS/OO FQDNs and corresponding certificates

Note

Please follow the SaaS Ops procedure to work with the customer to create the customer-managed FQDNs and generate publicly signed certificates.

Typically for each customer tenant a set of 3 FQDNs will be required, which need to be DNS-mapped (CNAME) to 3 intermediate FQDNs (managed by SaaS), such as (just an example):

DNS name (customer)	CNAME (SaaS-managed)	Certificate/key
smax.esm-api.acme.com	smax.api..esm-saas.com	smax-acme.crt, smax-acme.key
cms.esm-api.acme.com	cms.api..esm-saas.com	cms-acme.crt, cms-acme.key
oo.esm-api.acme.com	oo.api..esm-saas.com	oo-acme.crt, oo-acme.key

The 3 CNAMEs will need to be created under the esm-saas.com domain (managed under Route53 by SaaS team) and provided to the customer for DNS mapping. As a convention, the is the ID of the Customer entity in PCS for that particular customer.

If a customer has multiple tenants that need to be enabled for zero trust, use a prefix for the DNS name. For example for a test tenant:

DNS name (customer)	CNAME (SaaS-managed)	Certificate/key
tst.smax.esm-api.acme.com	tst.smax.api..esm-saas.com	tst-smax-acme.crt, tst-smax-acme.key
tst.cms.esm-api.acme.com	tst.cms.api..esm-saas.com	tst-cms-acme.crt, tst-cms-acme.key
tst.oo.esm-api.acme.com	tst.oo.api..esm-saas.com	tst-oo-acme.crt, tst-oo-acme.key

The customer will also need to provide the SaaS team with publicly signed certificates for their FQDNs - these will be required by nginx as described below.

Note

Public certificates have to be generated by the customer. We cannot use AWS-generated certificates in this case.

Create and configure Nginx service machine

You'll need to create two Nginx service machines to achieve high availability. This section provides detailed steps on how to create and configure the Nginx service machines.

Create and Deploy EC2 instance

1. Sign in to AWS, and then navigate to EC2 > Instance.
2. Click Launch instances in the right corner.
3. Enter a name. For example, nginx-1.
4. For Application and OS Images (Amazon Machine Image), choose an Amazon Machine Image (CCOE AMI for SaaS Operation), and then select the 64-bit(x86) Architecture. See Nginx on AWS for more information.
5. Select t3.medium as the Instance type.
6. In Name and tags section at top of page enter tags necessary for SaaS deployment. Copy the tags similar to another instance in the same farm. If you don't do this the deployment will fail.
7. Select one key pair name in the Key pair section.
8. For Network settings, click the Edit button
   • Select the existing VPC of current farm where smax/cms/oo are running, and select one existing private subnet.
     • Select Disable for the Auto-assign public IP filed.
     • Select Create security group and then enter a name and description.
     • For Inbound Security Group Rules, add SSH and HTTPS rules. Note Set the source type of the HTTPS rule to the security group of the NLB created below (you will have to come back here to add this rule after you create the NLB and its security group).
       Set the source type of the SSH rule to the bastion security group to limit SSH access to nginx server from the bastion node only.
9. Set the Configure storage section to 100 GiB gp3.
10. Click the Launch instance button.

Install Nginx service

1. Use SSH to access the nginx server machine from the resource defined in the above SSH rule.
2. Install the Nginx service by running the following command.

   sudo yum -y install nginx
   

Configure the nginx.conf file

1. Go to nginx configuration file folder via running cd /etc/nginx command. Back up the original nginx.conf file
2. Create a /etc/nginx/ssl  folder and copy the customer-issued public certificates and keys into it
3. Create a file albCA.crt in /etc/nginx/ssl  containing the root CA and any intermediate CAs used to sign the SaaS farm certificate on ALB (e.g. eu18-smax.saas.microfocus.com)
4. Run the following command to modify the nginx.conf file.

   sudo vim nginx.conf
   

5. Edit the file as below.

   user nginx;
   worker_processes auto;
   error_log /var/log/nginx/error.log notice;
   pid /run/nginx.pid;
   # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
   include /usr/share/nginx/modules/*.conf;
   events {
       worker_connections 1024;
   }
   http {
       log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';
       access_log  /var/log/nginx/access.log  main;
       sendfile            on;
       tcp_nopush          on;
       keepalive_timeout   60;
       types_hash_max_size 4096;
       client_body_timeout 60s;           # maximum time for reading the body of a client request. This value can be set globally or in each server. You may use the same value as customer's client request body timeout;
       client_max_body_size 50m;          # maximum allowed size of the client request body. This value can be set globally or in each server. You may use the same value as customer's client request body size;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       underscores_in_headers on;
       include             /etc/nginx/mime.types;
       default_type        application/octet-stream;
       # Load modular configuration files from the /etc/nginx/conf.d directory.
       # See http://nginx.org/en/docs/ngx_core_module.html#include
       # for more information.
       include /etc/nginx/conf.d/*.conf;
       # Used to resolve DNS name of the proxy_pass server
       # 169.254.169.253 is the well known DNS server in AWS VPC
       resolver 169.254.169.253;
       # Repeat the 3 server sections below for each tenant that has zero trust enabled.
       # For each tenant/product combination, set the proper server_name, ssl_certificate and ssl_certificate_key
       server {
           listen       443 ssl;      # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port;
           server_name  smax.esm-api.acme.com;     # specify the server name, the value should be SMAX FQDN allocated for the customer, for example smax.esm-api.acme.com;
           ssl_certificate      ssl/smax-acme.crt;     # the location of the server certificate generated for the server specified in server_name;
           ssl_certificate_key  ssl/smax-acme.key;     # the location of the private key of the generated for the server specified in server_name;
   
           client_body_timeout 60s;           # maximum time for reading the body of a client request sent to SMAX 
           client_max_body_size 50m;          # maximum allowed size of the client request body sent to SMAX
   
           ssl_session_cache    shared:SSL:1m;
           ssl_session_timeout  5m;
           ssl_ciphers  'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
           ssl_prefer_server_ciphers  on;
           location / {
               set $backend_server           eu18-smax.saas.microfocus.com; # The SMAX FQDN of current farm, take eu18 as an example
               proxy_pass                    https://$backend_server;       # Use a variable so that DNS lookup is forced every time
               proxy_set_header              Host  $backend_server;         # Sets the request header Host to the specified backend server
               proxy_ssl_verify              on;                # Enable SSL authentication for proxy requests
               proxy_ssl_trusted_certificate ssl/albCA.crt;     # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of SMAX application load balancer.
           }
       }
       server {
           listen       443 ssl;   # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port 
           server_name  cms.esm-api.acme.com;   # specify the server name, the value should be CMS FQDN allocated for the customer, for example cms.esm-api.acme.com;
           ssl_certificate      ssl/cms-acme.crt;    # the location of the server certificate generated for the server specified in server_name
           ssl_certificate_key  ssl/cms-acme.key;    # the location of server key generated for the server specified in server_name
           client_body_timeout 60s;           # maximum time for reading the body of a client request sent to CMS
           client_max_body_size 50m;          # maximum allowed size of the client request body sent to CMS
           ssl_session_cache    shared:SSL:1m;
           ssl_session_timeout  5m;
           ssl_ciphers  'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
           ssl_prefer_server_ciphers  on;
           location / {
               set $backend_server           cms.eu18-smax.saas.microfocus.com; # The CMS FQDN of current farm, take eu18 as an example
               proxy_pass                    https://$backend_server;           # Use a variable so that DNS lookup is forced every time
               proxy_set_header              Host  $backend_server;             # Sets the request header Host to the specified backend server
               proxy_ssl_verify              on;                # Enable SSL authentication for proxy requests
               proxy_ssl_trusted_certificate ssl/albCA.crt;     # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of CMS application load balancer. 
           }
       }
       server {
           listen       443 ssl;     # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port
           server_name  oo.esm-api.acme.com;   # specify the server name, the value should be OO FQDN allocated for the customer, for example oo.esm-api.acme.com;
           ssl_certificate      ssl/oo-acme.crt;   # the location of the server certificate generated for the server specified in server_name
           ssl_certificate_key  ssl/oo-acme.key;   # the location of server key generated for the server specified in server_name
           client_body_timeout 60s;           # maximum time for reading the body of a client request sent to OO
           client_max_body_size 50m;          # maximum allowed size of the client request body sent to OO
           ssl_session_cache    shared:SSL:1m;
           ssl_session_timeout  5m;
           ssl_ciphers  'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
           ssl_prefer_server_ciphers  on;
           location / {
               set $backend_server           oo.eu18-smax.saas.microfocus.com; # The OO FQDN of current farm, take eu18 as an example
               proxy_pass                    https://$backend_server;          # Use a variable so that DNS lookup is forced every time
               proxy_set_header              Host  $backend_server;            # Sets the request header Host to the specified backend server
               proxy_ssl_verify              on;                # Enable SSL authentication for proxy requests
               proxy_ssl_trusted_certificate ssl/albCA.crt;     # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of OO application load balancer. 
           }
       }
   }
   

6. Enable and start the nginx service by running the following command.

   sudo systemctl enable nginx
   sudo systemctl start nginx
   

Create the second nginx service machine

Create the second nginx service machine with the same steps as above. However, it should be in a different availability zone of current VPC for high availability. For example, nginx-2.

Note

The certificates applied to NLB will need to be publicly signed by a public CA, so normally the customer gateway will accept them.

Create a target group

1. Navigate to EC2 > Target groups.
2. Click the Create target group button in the right corner.
3. Select Instance as the target type.
4. Enter a name for the target group. For example, nlb-tg.
5. Select TCP as the protocol, and then enter 443 as Port.
6. Select the existing VPC of your current farm where smax/cms/oo are running.
7. Select TCP as the Health check protocol.
8. Click Next.
9. From the Register target page, select the instance ID whose Name is displayed as the two nginx service machines that you created in previous steps.
10. Click the Include as pending below button. The two Nginx instances will be listed in the Targets section.
11. Click the Create target group button.
    The target group has been created successfully.

Create a Network Load Balancer

1. Sign in to AWS, and then navigate to EC2 > Load Balancers.
2. Click the Create load balancer button on the right corner.
3. Select the Network Load Balancer the balancer type, and then click the Create button.
4. Enter a proper name for Load balancer name. For example, NLB-Acme.
5. Use the default value Internet-facing for the Scheme section.
6. Use the default value IPv4 for the IP address type section.
7. In the Network mapping section, select the existing VPC of current farm where SMAX/CMS/OO are running, then map to the 3 public subnets of the VPC, use default values for others.
8. In Security groups section, click create a new security group. Give a security group name, description, select the same VPC in step 7, add one inboud rule whose type is HTTPS, source IP is the IP range for customer's API gateway. Delete the default security group. Refresh and select the newly created security group. For troubleshooting purpose you may add some additional IP ranges. For SaaS enter the tag: Owner: ESM
9. In the Listeners and routing section, select TCP as the protocol, and then set the Port to 443.
10. Select the target group you created above. For example, select nlb-tg.
11. For Saas, enter Tags by copying them from another sample LB.
12. Click the Create load balancer button.
13. Select this NLB and go to the detail page, you will see the listeners of target groups.
14. Go back to the security group of the EC2 instance(s) of nginx and set the HTTPS rule source type to the security group of the NLB (as described above).

Edit a Network Load Balancer Security Group

For the step #7 above, in the section Create a Network Load Balancer, operate, when requested, the change:

1. Sign in to AWS, and then navigate to EC2 > Security Groups.
2. Choose the right ZeroTrust Security group, e.g. sg-0e4a9f16dadd46485 - zerotrust-nlb-sg on EU18.
3. Check the Inbound Rules section and choose Edit inbound rules.
4. Remove and/or add the requested IP/IP range in a new rule. Save the changes.

Map CNAMEs to the NLB

Map the 3 CNAMEs created under Route53 to the NLB (use Alias to NLB DNS name), for example:

DNS CNAME	Alias
smax.api..esm-saas.com
cms.api..esm-saas.com
oo.api..esm-saas.com

Testing

Validate certificates on the customer managed FQDN's

From within the Zero Trust Nginx instance, you can use a curl command like this to confirm the certificate from customer is valid:

curl -v --resolve tst.smax.esm-api.acme.com:443:127.0.0.1 https://tst.smax.esm-api.achmea.nl

Use Postman to check network connectivity

You can perform a REST call on the customer FQDN to validate connectivity. For example:

POST: https://tst.smax.esm-api.acme.com/auth/authentication-endpoint/authenticate/token?TENANTID=<TENANT_ID >

Note

This requires that your proxy ip address is part of the IP allowlist.

If you don't configure mTLS, you will get an error: 400 No required SSL certificate was sent But at least this verifies network connectivity.

This requires that your proxy ip address is part of the IP allowlist

Related pages

• Page: ESM Cloud Farm Version Tracking
• Page: How to get an Opentext Confluence account
• Page: ITOM APM AppPluse Cloud Farm Information
• Page: ITOM Cloud Service Ops Doc Management Process
• Page: ITOM ESM Cloud Service Catalog
• Page: ITOM OpsB NOM Cloud Service Catalog
• Page: OpsB and NOM Cloud Deployments Version Tracking