276 lines
18 KiB
Markdown
276 lines
18 KiB
Markdown
# Configure-Nginx-through-network-load-balancer_688996474
|
||
## Create customer managed SMAX/CMS/OO FQDNs and corresponding certificates
|
||
|
||
Note
|
||
|
||
**Please follow the SaaS Ops procedure to work with the customer to create the customer-managed FQDNs and generate publicly signed certificates.**
|
||
|
||
Typically for each customer tenant a set of 3 FQDNs will be required, which need to be DNS-mapped (CNAME) to 3 intermediate FQDNs (managed by SaaS), such as (just an example):
|
||
|
||
| DNS name (customer) | CNAME (SaaS-managed) | Certificate/key |
|
||
| --- | --- | --- |
|
||
| [smax.esm-api.acme.com](http://smax.esm-api.acme.com/) | smax.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | smax-acme.crt, smax-acme.key |
|
||
| [cms.esm-api.acme.com](http://cms.esm-api.acme.com/) | cms.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | cms-acme.crt, cms-acme.key |
|
||
| [oo.esm-api.acme.com](http://oo.esm-api.acme.com/) | oo.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | oo-acme.crt, oo-acme.key |
|
||
|
||
The 3 CNAMEs will need to be created under the [esm-saas.com](http://esm-saas.com/) domain (managed under Route53 by SaaS team) and provided to the customer for DNS mapping. As a convention, the <customerid> is the ID of the Customer entity in PCS for that particular customer.
|
||
|
||
If a customer has multiple tenants that need to be enabled for zero trust, use a prefix for the DNS name. For example for a test tenant:
|
||
|
||
| DNS name (customer) | CNAME (SaaS-managed) | Certificate/key |
|
||
| --- | --- | --- |
|
||
| [tst.smax.esm-api.acme.com](http://tst.smax.esm-api.acme.com/) | tst.smax.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | tst-smax-acme.crt, tst-smax-acme.key |
|
||
| [tst.cms.esm-api.acme.com](http://tst.cms.esm-api.acme.com/) | tst.cms.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | tst-cms-acme.crt, tst-cms-acme.key |
|
||
| [tst.oo.esm-api.acme.com](http://tst.oo.esm-api.acme.com/) | tst.oo.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | tst-oo-acme.crt, tst-oo-acme.key |
|
||
|
||
The customer will also need to provide the SaaS team with publicly signed certificates for their FQDNs - these will be required by nginx as described below.
|
||
|
||
Note
|
||
|
||
Public certificates have to be generated by the customer. We cannot use AWS-generated certificates in this case.
|
||
|
||
## Create and configure Nginx service machine
|
||
|
||
You'll need to create two Nginx service machines to achieve high availability. This section provides detailed steps on how to create and configure the Nginx service machines.
|
||
|
||
### Create and Deploy EC2 instance
|
||
|
||
1. Sign in to AWS, and then navigate to **EC2 >** **Instance**.
|
||
2. Click **Launch instances** in the right corner.
|
||
3. Enter a name. For example, `nginx-1`.
|
||
4. For Application and OS Images (Amazon Machine Image), choose an Amazon Machine Image (CCOE AMI for SaaS Operation), and then select the **64-bit(x86)** Architecture. See [Nginx on AWS](https://docs.nginx.com/nginx/deployment-guides/amazon-web-services/ec2-instances-for-nginx/) for more information.
|
||
5. Select **t3.medium** as the Instance type.
|
||
6. In Name and tags section at top of page enter tags necessary for SaaS deployment. Copy the tags similar to another instance in the same farm. **If you don't do this the deployment will fail.**
|
||
7. Select one key pair name in the **Key pair** section.
|
||
8. For Network settings, click the **Edit** button
|
||
- Select the **existing VPC** of current farm where smax/cms/oo are running, and select one **existing private subnet**.
|
||
- Select **Disable** for the Auto-assign public IP filed.
|
||
- Select **Create security group** and then enter a name and description.
|
||
- For Inbound Security Group Rules, add **SSH** and **HTTPS** rules.
|
||
Note
|
||
Set the source type of the **HTTPS** rule to the security group of the NLB created below (you will have to come back here to add this rule after you create the NLB and its security group).
|
||
Set the source type of the **SSH** rule to the bastion security group to limit SSH access to nginx server from the bastion node only.
|
||
9. Set the **Configure storage section** to **100 GiB gp3**.
|
||
10. Click the **Launch instance** button.
|
||
|
||
### Install Nginx service
|
||
|
||
1. Use SSH to access the nginx server machine from the resource defined in the above SSH rule.
|
||
2. Install the Nginx service by running the following command.
|
||
```
|
||
sudo yum -y install nginx
|
||
```
|
||
|
||
### Configure the nginx.conf file
|
||
|
||
1. Go to nginx configuration file folder via running `cd /etc/nginx` command. Back up the original `nginx.conf` file
|
||
2. Create a `/etc/nginx/ssl ` folder and copy the customer-issued public certificates and keys into it
|
||
3. Create a file `albCA.crt` in `/etc/nginx/ssl ` containing the root CA and any intermediate CAs used to sign the SaaS farm certificate on ALB (e.g. `eu18-smax.saas.microfocus.com`)
|
||
4. Run the following command to modify the `nginx.conf` file.
|
||
```
|
||
sudo vim nginx.conf
|
||
```
|
||
5. Edit the file as below.
|
||
```
|
||
user nginx;
|
||
worker_processes auto;
|
||
error_log /var/log/nginx/error.log notice;
|
||
pid /run/nginx.pid;
|
||
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
|
||
include /usr/share/nginx/modules/*.conf;
|
||
events {
|
||
worker_connections 1024;
|
||
}
|
||
http {
|
||
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
|
||
'$status $body_bytes_sent "$http_referer" '
|
||
'"$http_user_agent" "$http_x_forwarded_for"';
|
||
access_log /var/log/nginx/access.log main;
|
||
sendfile on;
|
||
tcp_nopush on;
|
||
keepalive_timeout 60;
|
||
types_hash_max_size 4096;
|
||
client_body_timeout 60s; # maximum time for reading the body of a client request. This value can be set globally or in each server. You may use the same value as customer's client request body timeout;
|
||
client_max_body_size 50m; # maximum allowed size of the client request body. This value can be set globally or in each server. You may use the same value as customer's client request body size;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
underscores_in_headers on;
|
||
include /etc/nginx/mime.types;
|
||
default_type application/octet-stream;
|
||
# Load modular configuration files from the /etc/nginx/conf.d directory.
|
||
# See http://nginx.org/en/docs/ngx_core_module.html#include
|
||
# for more information.
|
||
include /etc/nginx/conf.d/*.conf;
|
||
# Used to resolve DNS name of the proxy_pass server
|
||
# 169.254.169.253 is the well known DNS server in AWS VPC
|
||
resolver 169.254.169.253;
|
||
# Repeat the 3 server sections below for each tenant that has zero trust enabled.
|
||
# For each tenant/product combination, set the proper server_name, ssl_certificate and ssl_certificate_key
|
||
server {
|
||
listen 443 ssl; # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port;
|
||
server_name smax.esm-api.acme.com; # specify the server name, the value should be SMAX FQDN allocated for the customer, for example smax.esm-api.acme.com;
|
||
ssl_certificate ssl/smax-acme.crt; # the location of the server certificate generated for the server specified in server_name;
|
||
ssl_certificate_key ssl/smax-acme.key; # the location of the private key of the generated for the server specified in server_name;
|
||
|
||
client_body_timeout 60s; # maximum time for reading the body of a client request sent to SMAX
|
||
client_max_body_size 50m; # maximum allowed size of the client request body sent to SMAX
|
||
|
||
ssl_session_cache shared:SSL:1m;
|
||
ssl_session_timeout 5m;
|
||
ssl_ciphers 'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
|
||
ssl_prefer_server_ciphers on;
|
||
location / {
|
||
set $backend_server eu18-smax.saas.microfocus.com; # The SMAX FQDN of current farm, take eu18 as an example
|
||
proxy_pass https://$backend_server; # Use a variable so that DNS lookup is forced every time
|
||
proxy_set_header Host $backend_server; # Sets the request header Host to the specified backend server
|
||
proxy_ssl_verify on; # Enable SSL authentication for proxy requests
|
||
proxy_ssl_trusted_certificate ssl/albCA.crt; # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of SMAX application load balancer.
|
||
}
|
||
}
|
||
server {
|
||
listen 443 ssl; # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port
|
||
server_name cms.esm-api.acme.com; # specify the server name, the value should be CMS FQDN allocated for the customer, for example cms.esm-api.acme.com;
|
||
ssl_certificate ssl/cms-acme.crt; # the location of the server certificate generated for the server specified in server_name
|
||
ssl_certificate_key ssl/cms-acme.key; # the location of server key generated for the server specified in server_name
|
||
client_body_timeout 60s; # maximum time for reading the body of a client request sent to CMS
|
||
client_max_body_size 50m; # maximum allowed size of the client request body sent to CMS
|
||
ssl_session_cache shared:SSL:1m;
|
||
ssl_session_timeout 5m;
|
||
ssl_ciphers 'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
|
||
ssl_prefer_server_ciphers on;
|
||
location / {
|
||
set $backend_server cms.eu18-smax.saas.microfocus.com; # The CMS FQDN of current farm, take eu18 as an example
|
||
proxy_pass https://$backend_server; # Use a variable so that DNS lookup is forced every time
|
||
proxy_set_header Host $backend_server; # Sets the request header Host to the specified backend server
|
||
proxy_ssl_verify on; # Enable SSL authentication for proxy requests
|
||
proxy_ssl_trusted_certificate ssl/albCA.crt; # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of CMS application load balancer.
|
||
}
|
||
}
|
||
server {
|
||
listen 443 ssl; # listen on port 443 and enable SSL/TLS secure connections, 443 is an example, you may use other port
|
||
server_name oo.esm-api.acme.com; # specify the server name, the value should be OO FQDN allocated for the customer, for example oo.esm-api.acme.com;
|
||
ssl_certificate ssl/oo-acme.crt; # the location of the server certificate generated for the server specified in server_name
|
||
ssl_certificate_key ssl/oo-acme.key; # the location of server key generated for the server specified in server_name
|
||
client_body_timeout 60s; # maximum time for reading the body of a client request sent to OO
|
||
client_max_body_size 50m; # maximum allowed size of the client request body sent to OO
|
||
ssl_session_cache shared:SSL:1m;
|
||
ssl_session_timeout 5m;
|
||
ssl_ciphers 'TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_8_SHA256:TLS_AES_128_CCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384';
|
||
ssl_prefer_server_ciphers on;
|
||
location / {
|
||
set $backend_server oo.eu18-smax.saas.microfocus.com; # The OO FQDN of current farm, take eu18 as an example
|
||
proxy_pass https://$backend_server; # Use a variable so that DNS lookup is forced every time
|
||
proxy_set_header Host $backend_server; # Sets the request header Host to the specified backend server
|
||
proxy_ssl_verify on; # Enable SSL authentication for proxy requests
|
||
proxy_ssl_trusted_certificate ssl/albCA.crt; # Specifies the location of CA certificate from which the proxy requests SSL security authentication, in this case, the CA cert of OO application load balancer.
|
||
}
|
||
}
|
||
}
|
||
```
|
||
6. Enable and start the nginx service by running the following command.
|
||
```
|
||
sudo systemctl enable nginx
|
||
sudo systemctl start nginx
|
||
```
|
||
|
||
### Create the second nginx service machine
|
||
|
||
Create the second nginx service machine with the same steps as above. However, it should be in a different availability zone of current VPC for high availability. For example, nginx-2.
|
||
|
||
Note
|
||
|
||
The certificates applied to NLB will need to be publicly signed by a public CA, so normally the customer gateway will accept them.
|
||
|
||
## Create a target group
|
||
|
||
1. Navigate to **EC2** > **Target groups**.
|
||
2. Click the **Create target group** button in the right corner.
|
||
3. Select **Instance** as the target type.
|
||
4. Enter a name for the target group. For example, `nlb-tg`.
|
||
5. Select **TCP** as the protocol, and then enter `443` as Port.
|
||
6. Select the existing **VPC** of your current farm where smax/cms/oo are running.
|
||
7. Select **TCP** as the Health check protocol.
|
||
8. Click **Next**.
|
||
9. From the Register target page, select the instance ID whose Name is displayed as the **two nginx service machines** that you created in previous steps.
|
||
10. Click the **Include as pending below** button. The two Nginx instances will be listed in the **Targets** section.
|
||
11. Click the **Create target group** button.
|
||
The target group has been created successfully.
|
||
|
||
## Create a Network Load Balancer
|
||
|
||
1. Sign in to AWS, and then navigate to **EC2** > **Load Balancers**.
|
||
2. Click the **Create load balancer** button on the right corner.
|
||
3. Select the **Network Load Balancer** the balancer type, and then click the **Create** button.
|
||
4. Enter a proper name for **Load balancer name**. For example, NLB-Acme.
|
||
5. Use the default value `Internet-facing` for the **Scheme** section.
|
||
6. Use the default value `IPv4` for the **IP address type** section.
|
||
7. In the **Network mapping** section, select the **existing VPC** of current farm where SMAX/CMS/OO are running, then map to the **3 public subnets** of the VPC, use default values for others.
|
||
8. In **Security groups** section, click create a new security group. Give a security group name, description, select the same VPC in step 7, add one inboud rule whose type is **HTTPS**, source IP is the **IP range for customer's API gateway**. Delete the default security group. Refresh and select the newly created security group. For troubleshooting purpose you may add some additional IP ranges. For SaaS enter the tag: Owner: ESM
|
||
9. In the **Listeners and routing** section, select **TCP** as the protocol, and then set the Port to `443`.
|
||
10. Select the target group you created above. For example, select **nlb-tg**.
|
||
11. For Saas, enter Tags by copying them from another sample LB.
|
||
12. Click the **Create load balancer** button.
|
||
13. Select this NLB and go to the detail page, you will see the listeners of target groups.
|
||
14. Go back to the security group of the EC2 instance(s) of nginx and set the HTTPS rule source type to the security group of the NLB (as described above).
|
||
|
||
## Edit a Network Load Balancer Security Group
|
||
|
||
For the step #7 above, in the section **Create a Network Load Balancer**, operate, when requested, the change:
|
||
|
||
1. Sign in to AWS, and then navigate to **EC2** > **Security Groups**.
|
||
2. Choose the right ZeroTrust Security group, e.g. sg-0e4a9f16dadd46485 - zerotrust-nlb-sg on EU18.
|
||
3. Check the **Inbound Rules** section and choose **Edit inbound rules**.
|
||
4. Remove and/or add the requested IP/IP range in a new rule. Save the changes.
|
||
|
||
## Map CNAMEs to the NLB
|
||
|
||
Map the 3 CNAMEs created under Route53 to the NLB (use Alias to NLB DNS name), for example:
|
||
|
||
| DNS CNAME | Alias |
|
||
| --- | --- |
|
||
| smax.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | <NLB DNS name> |
|
||
| cms.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | <NLB DNS name> |
|
||
| oo.api.<customerid>.[esm-saas.com](http://esm-saas.com/) | <NLB DNS name> |
|
||
|
||
## Testing
|
||
|
||
### Validate certificates on the customer managed FQDN's
|
||
|
||
From within the Zero Trust Nginx instance, you can use a curl command like this to confirm the certificate from customer is valid:
|
||
|
||
```
|
||
curl -v --resolve tst.smax.esm-api.acme.com:443:127.0.0.1 https://tst.smax.esm-api.achmea.nl
|
||
```
|
||
|
||
|
||
|
||
### Use Postman to check network connectivity
|
||
|
||
You can perform a REST call on the customer FQDN to validate connectivity. For example:
|
||
|
||
POST: https://tst.smax.esm-api.acme.com/auth/authentication-endpoint/authenticate/token?TENANTID=<TENANT\_ID >
|
||
|
||
Note
|
||
|
||
This requires that your proxy ip address is part of the IP allowlist.
|
||
|
||
If you don't configure mTLS, you will get an error: 400 No required SSL certificate was sent But at least this verifies network connectivity.
|
||
|
||
This requires that your proxy ip address is part of the IP allowlist
|
||
|
||
**Related pages**
|
||
|
||
- Page:
|
||
[ESM Cloud Farm Version Tracking](/display/ICSD/ESM+Cloud+Farm+Version+Tracking)
|
||
- Page:
|
||
[How to get an Opentext Confluence account](/display/ICSD/How+to+get+an+Opentext+Confluence+account)
|
||
- Page:
|
||
[ITOM APM AppPluse Cloud Farm Information](/display/ICSD/ITOM+APM+AppPluse+Cloud+Farm+Information)
|
||
- Page:
|
||
[ITOM Cloud Service Ops Doc Management Process](/display/ICSD/ITOM+Cloud+Service+Ops+Doc+Management+Process)
|
||
- Page:
|
||
[ITOM ESM Cloud Service Catalog](/display/ICSD/ITOM+ESM+Cloud+Service+Catalog)
|
||
- Page:
|
||
[ITOM OpsB NOM Cloud Service Catalog](/display/ICSD/ITOM+OpsB+NOM+Cloud+Service+Catalog)
|
||
- Page:
|
||
[OpsB and NOM Cloud Deployments Version Tracking](/display/ICSD/OpsB+and+NOM+Cloud+Deployments+Version+Tracking)
|