Procedure

1. Click the Get started button on Content from console.aws.amazon.com is not included.AWS console page:

   

   If you have activated Red Hat OpenShift Service on AWS before but did not complete the process, you can click the button and complete the account linking as described in the following steps.

2. Confirm that you want your contact information to be shared with Red Hat and enable the service:

   

   You are not charged by enabling the service in this step. The connection is made for billing and metering that takes place only after you deploy your first cluster. This could take a few minutes.

3. Verify that the process completed successfully by reviewing the confirmation message:

   

4. Check the status of additional prerequisites on this verification page. If any prerequisites are not met, a corresponding message is shown. The following is an example of insufficient quotas in the selected region:

   
   • Click the Increase service quotas button or use the Learn more link to get more information about how to manage service quotas. In the case of insufficient quotas, note that quotas are region-specific. You can use the region switcher to re-run the quota check for any region you are interested in and then submit service quota increase requests as needed.

5. Confirm that all the prerequisites are met:

   

   The Elastic Load Balancing (ELB) service-linked role is created for you automatically. You can click any of the small Info blue links to get context-sensitive help and resources.

Procedure

1. Click the Continue to Red Hat button to proceed with account linking:

   

2. Log in to your Red Hat account, if prompted.

   
   Note

   Your AWS account must be linked to a single Red Hat organization.

   • You can also register for a new Red Hat account or reset your password on this page.
   • Log in to the Red Hat account that you want to associate with the AWS account that has activated the Red Hat OpenShift Service on AWS product subscription.
   • The AWS account used for service billing can only be associated with a single Red Hat account. Typically, an AWS payer account is the one that is used to subscribe to Red Hat OpenShift Service on AWS and used for account linking and billing.
   • All team members within the same Red Hat organization can use the linked AWS account for service billing while creating Red Hat OpenShift Service on AWS clusters.

3. Complete the Red Hat account linking after reviewing the terms and conditions.

   Note

   This step is available only if the AWS account was not linked to any Red Hat account before.

   This step is skipped if the AWS account is already linked to the user’s logged in Red Hat account.

   If the AWS account is linked to a different Red Hat account, an error is displayed. See Correcting Billing Account Information for HCP clusters for troubleshooting.

4. On the Complete your account connection page, verify that the Red Hat and AWS account numbers are correct.

   

5. Click the Connect accounts button if you agree with the service terms.

   If this is the first time you are using the Red Hat Hybrid Cloud Console, you must agree with the general managed services terms and conditions before creating the first cluster:

   

   Additional terms that need to be reviewed and accepted are shown after clicking the View Terms and Conditions button:

   

   Submit your agreement once you have reviewed any additional terms when prompted at this time.

6. The Hybrid Cloud Console provides a confirmation that AWS account setup was completed and lists the prerequisites for cluster deployment:

   

   The last section of this page shows cluster deployment options, either using the ROSA command-line interface (CLI) (rosa) or through the web console:

   

Procedure

1. Initiate the cluster deployment using the rosa create cluster command. You can click the copy button on the This content is not included.Set up Red Hat OpenShift Service on AWS console page and paste the command in your terminal. This launches the cluster creation process in interactive mode.

   

   1. Optional: Add the --profile <profile_name> option to the rosa create cluster command to use a non-default profile from your ~/.aws/credentials file, for example:

      $ rosa create cluster --profile stage

      Copy code to clipboard

      If no profile is specified, the default AWS CLI profile determines the AWS infrastructure profile into which the cluster is deployed.

2. Specify the billing AWS account.

   
   • Only AWS accounts that are linked to the user’s logged in Red Hat account are shown.
   • The specified AWS account is charged for using the Red Hat OpenShift Service on AWS service.

   • An indicator shows if the Red Hat OpenShift Service on AWS contract is enabled or not enabled for a given AWS billing account.

     • If you select an AWS billing account that shows the Contract enabled label, on-demand consumption rates are charged only after the capacity of your pre-paid contract is consumed.
     • AWS accounts without the Contract enabled label are charged the applicable on-demand consumption rates.

Procedure

1. To start creating a cluster, select Create with web interface of the introductory Set up Red Hat OpenShift Service on AWS page.

   
   Note

   Complete the prerequisites before starting the web console deployment process.

   The ROSA command-line interface (CLI) (rosa) is required for certain tasks, such as creating the account roles. If you are deploying Red Hat OpenShift Service on AWS for the first time, follow the CLI steps until running the rosa whoami command, before starting the web console deployment steps.

2. Select the Hosted control plane option, then click Next.

   

3. In the next step Accounts and roles, specify the infrastructure AWS account, into which the Red Hat OpenShift Service on AWS cluster is deployed and where the resources are consumed and managed.

   
   • Click the How to associate a new AWS account, if you don’t see the account into which you want to deploy the Red Hat OpenShift Service on AWS cluster for detailed information on how to create or link account roles for this association.
   • The ROSA CLI is used for this.
   • If you are using multiple AWS accounts and have their profiles configured for the AWS CLI, you can use the --profile selector to specify the AWS profile when working with the ROSA CLI commands.

4. Select the AWS billing account.

   
   • Only AWS accounts that are linked to the user’s logged in Red Hat account are shown.
   • The specified AWS account is charged for using the Red Hat OpenShift Service on AWS service.

   • An indicator shows if the Red Hat OpenShift Service on AWS contract is enabled or not enabled for a given AWS billing account.

     • If you select an AWS billing account that shows the Contract enabled label, on-demand consumption rates are charged only after the capacity of your pre-paid contract is consumed.

     • AWS accounts without the Contract enabled label are charged the applicable on-demand consumption rates.

       Note

       The following steps past the billing AWS account selection are beyond the scope of this tutorial.

Procedure

1. Open the private offer URL provided by the seller. You can only access the URL from the AWS account ID that was specified by the seller.

2. On the Offer selection page, review Available offers. Options can include:

   1. A regular private offer that is pre-selected. This type of offer can be accepted only if Red Hat OpenShift Service on AWS was not activated before using the public offer or another private offer.

      

   2. A private offer that was created for an AWS account that previously activated Red Hat OpenShift Service on AWS using the public offer. The selected private offer is labeled as Upgrade and accepting this offer replaces the currently running contract for Red Hat OpenShift Service on AWS.

      

   3. If multiple offers are available, the public offer activated earlier is shown together with the newly provided agreement-based offer that is labeled as "Upgrade".

      

3. Verify that your offer configuration is selected and review the offer details.

   Note

   Review the contract end date, the number of units included with the offer, and the payment schedule. In this example, 1 cluster and up to 3 nodes utilizing 4 vCPUs are included.

   

4. Optional: Content from docs.aws.amazon.com is not included.Add your own purchase order (PO) number to the subscription that is being purchased, so it is included on your subsequent AWS invoices. Also, check the "Additional usage fees" that are charged for any usage above the scope of the "New offer configuration details".

   Note

   Private offers have several available configurations.

   • It is possible that the private offer you are accepting is set up with a fixed future start date.
   • If you do not have another active Red Hat OpenShift Service on AWS subscription at the time of accepting the private offer, a public offer or an older private offer entitlement, accept the private offer itself and continue with the account linking and cluster deployment steps after the specified service start date.

   You must have an active Red Hat OpenShift Service on AWS entitlement to complete these steps. Service start dates are always reported in the UTC time zone

5. Create or upgrade your contract.

   1. For private offers accepted by an AWS account that does not have Red Hat OpenShift Service on AWS activated yet and is creating the first contract for this service, click the Create contract button.

      

   2. For agreement-based offers, click the Upgrade current contract button shown in Figures 4 and 6.

      

6. Click Confirm.

   

7. If the accepted private offer service start date is set to be immediately following the offer acceptance, click the Set up your account button in the confirmation modal window.

   

8. If the accepted private offer has a future start date specified, return to the private offer page after the service start date, and click the Setup your account button to proceed with the Red Hat and AWS account linking.

   Note

   With no agreement active, the account linking described below is not triggered, the "Account setup" process can be done only after the "Service start date".

   These are always in UTC time zone.

Procedure

1. Click Set up your account to proceed to the AWS and Red Hat account linking step. At this time, you are already logged in with the AWS account that accepted the offer. Log in to a Red Hat account, if prompted.

   Note

   A Red Hat OpenShift Service on AWS private offer cannot be shared with AWS linked accounts through the AWS License Manager.

2. Add any users that you want to deploy Red Hat OpenShift Service on AWS clusters. Check Common User Management Questions for more details about Red Hat account user management tasks.
3. Verify that the already logged in Red Hat account includes all users that are meant to be Red Hat OpenShift Service on AWS cluster deployers benefiting from the accepted private offer.

4. Verify that the Red Hat account number and the AWS account ID are the desired accounts that are to be linked. This linking is unique and a Red Hat account can be connected only with a single AWS (billing) account.

   

5. To link the AWS account with another Red Hat account, log out from the Red Hat Hybrid Cloud Console before connecting the accounts. Repeat the step of setting the account by returning to the private offer URL that is already accepted.

   An AWS account can be connected with a single Red Hat account only. Once Red Hat and AWS accounts are connected, this cannot be changed by the user. If a change is needed, the user must create a support ticket.

6. Agree to the terms and conditions and then click Connect accounts.

Procedure

1. In your terminal, configure the following environment variables:

   $ export VPC_ID=<vpc_ID>
   $ export REGION=<region>
   $ export VPC_CIDR=<vpc_CIDR>

   Copy code to clipboard

   where:

   <vpc_ID>

   Replace with the ID of the VPC you want to install your cluster into.

   <region>

   Replace with the AWS region you want to install your cluster into.

   <vpc_CIDR>

   Replace with the CIDR range of your VPC.

2. Ensure all fields output correctly before moving to the next section:

   $ echo "VPC ID: ${VPC_ID}, VPC CIDR Range: ${VPC_CIDR}, Region: ${REGION}"

   Copy code to clipboard

Procedure

1. Create a security group and allow access to ports 53/tcp and 53/udp from the VPC:

   $ SG_ID=$(aws ec2 create-security-group --group-name rosa-inbound-resolver --description "Security group for ROSA inbound resolver" --vpc-id ${VPC_ID} --region ${REGION} --output text)
   $ aws ec2 authorize-security-group-ingress --group-id ${SG_ID} --protocol tcp --port 53 --cidr ${VPC_CIDR} --region ${REGION}
   $ aws ec2 authorize-security-group-ingress --group-id ${SG_ID} --protocol udp --port 53 --cidr ${VPC_CIDR} --region ${REGION}

   Copy code to clipboard

2. Create an Amazon Route 53 Inbound Resolver in your VPC:

   $ RESOLVER_ID=$(aws route53resolver create-resolver-endpoint \
     --name rosa-inbound-resolver \
     --creator-request-id rosa-$(date '+%Y-%m-%d') \
     --security-group-ids ${SG_ID} \
     --direction INBOUND \
     --ip-addresses $(aws ec2 describe-subnets --filter Name=vpc-id,Values=${VPC_ID} --region ${REGION} | jq -jr '.Subnets | map("SubnetId=\(.SubnetId) ") | .[]') \
     --region ${REGION} \
     --output text \
     --query 'ResolverEndpoint.Id')

   Copy code to clipboard
   Note

   The above command attaches Amazon Route 53 Inbound Resolver endpoints to all subnets in the provided VPC using dynamically allocated IP addresses. If you prefer to manually specify the subnets, the IP addresses, or both, run the following command instead:

   $ RESOLVER_ID=$(aws route53resolver create-resolver-endpoint \
     --name rosa-inbound-resolver \
     --creator-request-id rosa-$(date '+%Y-%m-%d') \
     --security-group-ids ${SG_ID} \
     --direction INBOUND \
     --ip-addresses SubnetId=<subnet_ID>,Ip=<endpoint_IP> SubnetId=<subnet_ID>,Ip=<endpoint_IP> \
     --region ${REGION} \
     --output text \
     --query 'ResolverEndpoint.Id')

   Copy code to clipboard

   + where:

   <subnet_ID>

   Specifies the subnet ID you want inbound resolver endpoints added to.

   <endpoint_IP>

   Specifies the static IP addresses you want inbound resolver endpoints added to.

3. Get the IP addresses of your inbound resolver endpoints to configure in your DNS server configuration:

   $ aws route53resolver list-resolver-endpoint-ip-addresses \
     --resolver-endpoint-id ${RESOLVER_ID} \
     --region=${REGION} \
     --query 'IpAddresses[*].Ip'

   Copy code to clipboard

   Example output

   [
       "10.0.45.253",
       "10.0.23.131",
       "10.0.148.159"
   ]

   Copy code to clipboard

Procedure

1. Before the cluster is created, configure your DNS server to forward all DNS requests for <cluster-name>.hypershift.local to your Amazon Route 53 Inbound Resolver endpoints.

   If you use a BIND DNS server, add a new zone to the /etc/named.conf file using the following example:

   zone "<cluster-name>.hypershift.local" {
     type forward;
     forward only;
     forwarders {
       10.0.45.253;
       10.0.23.131;
       10.0.148.159;
     };
   };

   Copy code to clipboard
   • Replace <cluster-name> with your Red Hat OpenShift Service on AWS cluster name.
   • Replace with the IP addresses of your inbound resolver endpoints collected above, ensuring that following each IP address there is a ;.
2. Create your cluster.

3. After your cluster has begun the creation process, locate the newly created private hosted zone:

   $ aws route53 list-hosted-zones-by-vpc \
     --vpc-id ${VPC_ID} \
     --vpc-region ${REGION} \
     --query 'HostedZoneSummaries[*].Name' \
     --output table

   Copy code to clipboard

   Example output

   --------------------------------------------------
   |             ListHostedZonesByVPC               |
   +------------------------------------------------+
   |  rosa.domain-prefix.lkmb.p3.openshiftapps.com. |
   |  cluster-name.hypershift.local.                |
   +------------------------------------------------+

   Copy code to clipboard

   Note

   It may take a few minutes for the cluster creation process to create the private hosted zones in Route 53. If you do not see a p3.openshiftapps.com domain, wait a few minutes and run the command again.

4. When you know the unique ID of the cluster domain, configure your DNS server to forward all DNS requests for rosa.<domain-prefix>.<unique-ID>.p3.openshiftapps.com to your Amazon Route 53 Inbound Resolver endpoints. If you use a BIND DNS server, add a new zone to the /etc/named.conf file using the following example:

   zone "rosa.<domain-prefix>.<unique-ID>.p3.openshiftapps.com" {
     type forward;
     forward only;
     forwarders {
       10.0.45.253;
       10.0.23.131;
       10.0.148.159;
     };
   };

   Copy code to clipboard
   • Replace <domain-prefix> with your cluster domain prefix and <unique-ID> with your unique ID collected above.
   • Replace with the IP addresses of your inbound resolver endpoints collected above, ensuring that following each IP address there is a ;.

Procedure

1. In your terminal, configure the following environment variables:

   $ export DOMAIN=apps.example.com
   $ export AWS_PAGER=""
   $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}"  | sed 's/-[a-z0-9]\{5\}$//')
   $ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
   $ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
   $ export SCRATCH="/tmp/${CLUSTER}/cloudfront-waf"
   $ mkdir -p ${SCRATCH}
   $ echo "Cluster: ${CLUSTER}, Region: ${REGION}, AWS Account ID: ${AWS_ACCOUNT_ID}"

   Copy code to clipboard

2. Replace the DOMAIN value apps.example.com with the custom domain you want to use for the IngressController.

   Note

   The "Cluster" output from the previous command might be the name of your cluster, the internal ID of your cluster, or the cluster’s domain prefix. If you prefer to use another identifier, you can manually set this value by running the following command:

   $ export CLUSTER=my-custom-value

   Copy code to clipboard

Procedure

1. Create a new Transport Layer Security (TLS) secret from a private key and a public certificate, where fullchain.pem is your full wildcard certificate chain, including any intermediaries, and privkey.pem is your wildcard certificate’s private key.

   Example

   $ oc -n openshift-ingress create secret tls waf-tls --cert=fullchain.pem --key=privkey.pem

   Copy code to clipboard

2. Create a new IngressController resource, for example, a waf-ingress-controller.yaml:

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: cloudfront-waf
     namespace: openshift-ingress-operator
   spec:
     domain: apps.example.com
     defaultCertificate:
       name: waf-tls
     endpointPublishingStrategy:
       loadBalancer:
         dnsManagementPolicy: Unmanaged
         providerParameters:
           aws:
             type: NLB
           type: AWS
         scope: External
       type: LoadBalancerService
     routeSelector:
       matchLabels:
        route: waf

   Copy code to clipboard

   where:

   domain: apps.example.com

   Specifies the custom domain you want to use for the IngressController.

   routeSelector

   Filters the set of routes serviced by the Ingress Controller. If you don’t provide a value, no filtering occurs. In this tutorial, you use the waf route selector.

3. Apply the IngressController:

   Example

   $ oc apply -f waf-ingress-controller.yaml

   Copy code to clipboard

Procedure

1. Create an AWS WAF rules file to apply to an ACL:

   $ cat << EOF > ${SCRATCH}/waf-rules.json
   [
       {
         "Name": "AWS-AWSManagedRulesCommonRuleSet",
         "Priority": 0,
         "Statement": {
           "ManagedRuleGroupStatement": {
             "VendorName": "AWS",
             "Name": "AWSManagedRulesCommonRuleSet"
           }
         },
         "OverrideAction": {
           "None": {}
         },
         "VisibilityConfig": {
           "SampledRequestsEnabled": true,
           "CloudWatchMetricsEnabled": true,
           "MetricName": "AWS-AWSManagedRulesCommonRuleSet"
         }
       },
       {
         "Name": "AWS-AWSManagedRulesSQLiRuleSet",
         "Priority": 1,
         "Statement": {
           "ManagedRuleGroupStatement": {
             "VendorName": "AWS",
             "Name": "AWSManagedRulesSQLiRuleSet"
           }
         },
         "OverrideAction": {
           "None": {}
         },
         "VisibilityConfig": {
           "SampledRequestsEnabled": true,
           "CloudWatchMetricsEnabled": true,
           "MetricName": "AWS-AWSManagedRulesSQLiRuleSet"
         }
       }
   ]
   EOF

   Copy code to clipboard

   This enables the Core (Common) and SQL AWS Managed Rule Sets.

2. Create an AWS WAF Web ACL using the rules you specified in the previous step:

   $ WAF_WACL=$(aws wafv2 create-web-acl \
     --name cloudfront-waf \
     --region ${REGION} \
     --default-action Allow={} \
     --scope CLOUDFRONT \
     --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=${CLUSTER}-waf-metrics \
     --rules file://${SCRATCH}/waf-rules.json \
     --query 'Summary.Name' \
     --output text)

   Copy code to clipboard

Procedure

1. Retrieve the newly created custom ingress controller’s Network Load Balancer (NLB) hostname:

   $ NLB=$(oc -n openshift-ingress get service router-cloudfront-waf \
     -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

   Copy code to clipboard

2. Import your certificate into Amazon Certificate Manager, where cert.pem is your wildcard certificate, fullchain.pem is your wildcard certificate’s chain and privkey.pem is your wildcard certificate’s private key.

   Note

   Regardless of what region your cluster is deployed, you must import this certificate to us-east-1 as Amazon CloudFront is a global AWS service.

   Example

   $ aws acm import-certificate --certificate file://cert.pem \
     --certificate-chain file://fullchain.pem \
     --private-key file://privkey.pem \
     --region us-east-1

   Copy code to clipboard

3. Log in to the Content from us-east-1.console.aws.amazon.com is not included.AWS console to create a CloudFront distribution.

4. Configure the CloudFront distribution by using the following information:

   Note

   If an option is not specified in the table below, leave the default, even if it is blank.

   Option	Value
   Origin domain	Output from the previous command [a]
   Name	rosa-waf-ingress [b]
   Viewer protocol policy	Redirect HTTP to HTTPS
   Allowed HTTP methods	GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE
   Cache policy	CachingDisabled
   Origin request policy	AllViewer
   Web Application Firewall (WAF)	Enable security protections
   Use existing WAF configuration	true
   Choose a web ACL	cloudfront-waf
   Alternate domain name (CNAME)	*.apps.example.com [c]
   Custom SSL certificate	Select the certificate you imported from the step above [d]

   1. Run echo ${NLB} to get the origin domain.
   2. If you have multiple clusters, ensure the origin name is unique.
   3. This should match the wildcard domain you used to create the custom ingress controller.
   4. This should match the alternate domain name entered above.

5. Retrieve the Amazon CloudFront Distribution endpoint:

   $ aws cloudfront list-distributions --query "DistributionList.Items[?Origins.Items[?DomainName=='${NLB}']].DomainName" --output text

   Copy code to clipboard

6. Update the DNS of your custom wildcard domain with a CNAME to the Amazon CloudFront Distribution endpoint from the step above.

   Example

   *.apps.example.com CNAME d1b2c3d4e5f6g7.cloudfront.net

   Copy code to clipboard

Procedure

1. Create a new project for your sample application by running the following command:

   $ oc new-project hello-world

   Copy code to clipboard

2. Deploy a hello-world application:

   $ oc -n hello-world new-app --image=docker.io/openshift/hello-openshift

   Copy code to clipboard

3. Create a route for the application specifying your custom domain name:

   Example

   $ oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls \
   --hostname hello-openshift.${DOMAIN}

   Copy code to clipboard

4. Label the route to admit it to your custom ingress controller:

   $ oc -n hello-world label route.route.openshift.io/hello-openshift-tls route=waf

   Copy code to clipboard

Procedure

1. Test that the application is accessible behind Amazon CloudFront:

   Example

   $ curl "https://hello-openshift.${DOMAIN}"

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

2. Test that the WAF denies a bad request:

   Example

   $ curl -X POST "https://hello-openshift.${DOMAIN}" \
     -F "user='<script><alert>Hello></alert></script>'"

   Copy code to clipboard

   Example output

   <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
   <TITLE>ERROR: The request could not be satisfied</TITLE>
   </HEAD><BODY>
   <H1>403 ERROR</H1>
   <H2>The request could not be satisfied.</H2>
   <HR noshade size="1px">
   Request blocked.
   We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
   <BR clear="all">
   If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
   <BR clear="all">
   <HR noshade size="1px">
   <PRE>
   Generated by cloudfront (CloudFront)
   Request ID: nFk9q2yB8jddI6FZOTjdliexzx-FwZtr8xUQUNT75HThPlrALDxbag==
   </PRE>
   <ADDRESS>
   </ADDRESS>
   </BODY></HTML>

   Copy code to clipboard

   The expected result is a 403 ERROR, which means the AWS WAF is protecting your application.

Procedure

1. Create a new project to deploy the AWS Load Balancer Operator into by running the following command:

   $ oc new-project aws-load-balancer-operator

   Copy code to clipboard

2. Create an AWS IAM policy for the AWS Load Balancer Controller if one does not already exist by running the following command:

   Note

   The policy is sourced from Content from raw.githubusercontent.com is not included.the upstream AWS Load Balancer Controller policy. This is required by the Operator to function.

   $ POLICY_ARN=$(aws iam list-policies --query \
        "Policies[?PolicyName=='aws-load-balancer-operator-policy'].{ARN:Arn}" \
        --output text)

   Copy code to clipboard

   $ if [[ -z "${POLICY_ARN}" ]]; then
       wget -O "${SCRATCH}/load-balancer-operator-policy.json" \
          https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json
        POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn \
        --output text iam create-policy \
        --policy-name aws-load-balancer-operator-policy \
        --policy-document "file://${SCRATCH}/load-balancer-operator-policy.json")
   fi

   Copy code to clipboard

3. Create an AWS IAM trust policy for AWS Load Balancer Operator:

   $ cat <<EOF > "${SCRATCH}/trust-policy.json"
   {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Condition": {
      "StringEquals" : {
        "${OIDC_ENDPOINT}:sub": ["system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-operator-controller-manager", "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster"]
      }
    },
    "Principal": {
      "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
    },
    "Action": "sts:AssumeRoleWithWebIdentity"
    }
    ]
   }
   EOF

   Copy code to clipboard

4. Create an AWS IAM role for the AWS Load Balancer Operator:

   $ ROLE_ARN=$(aws iam create-role --role-name "${CLUSTER}-alb-operator" \
      --assume-role-policy-document "file://${SCRATCH}/trust-policy.json" \
      --query Role.Arn --output text)

   Copy code to clipboard

5. Attach the AWS Load Balancer Operator policy to the IAM role we created previously by running the following command:

   $ aws iam attach-role-policy --role-name "${CLUSTER}-alb-operator" \
        --policy-arn ${POLICY_ARN}

   Copy code to clipboard

6. Create a secret for the AWS Load Balancer Operator to assume the newly created AWS IAM role:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Secret
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   stringData:
     credentials: |
       [default]
       role_arn = ${ROLE_ARN}
       web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
   EOF

   Copy code to clipboard

7. Install the AWS Load Balancer Operator:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   spec:
     upgradeStrategy: Default
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   spec:
     channel: stable-v1.0
     installPlanApproval: Automatic
     name: aws-load-balancer-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     startingCSV: aws-load-balancer-operator.v1.0.0
   EOF

   Copy code to clipboard

8. Deploy an instance of the AWS Load Balancer Controller using the Operator:

   $ cat << EOF | oc apply -f -
   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     credentials:
       name: aws-load-balancer-operator
     enabledAddons:
       - AWSWAFv2
   EOF

   Copy code to clipboard

   If you get an error, the Operator has not completed installing yet. Wait briefly and retry the step.

9. Check that the Operator and controller pods are both running:

   $ oc -n aws-load-balancer-operator get pods

   Copy code to clipboard

   Example output

   NAME                                                             READY   STATUS    RESTARTS   AGE
   aws-load-balancer-controller-cluster-6ddf658785-pdp5d            1/1     Running   0          99s
   aws-load-balancer-operator-controller-manager-577d9ffcb9-w6zqn   2/2     Running   0          2m4s

   Copy code to clipboard

   If the pods aren’t running yet, wait briefly and retry the step.

Procedure

1. Create a new project for the sample application:

   $ oc new-project hello-world

   Copy code to clipboard

2. Deploy a hello-world application:

   $ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift

   Copy code to clipboard

3. Convert the pre-created service resource to a NodePort service type:

   $ oc -n hello-world patch service hello-openshift -p '{"spec":{"type":"NodePort"}}'

   Copy code to clipboard

4. Deploy an AWS ALB using the AWS Load Balancer Operator:

   $ cat << EOF | oc apply -f -
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: hello-openshift-alb
     namespace: hello-world
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
   spec:
     ingressClassName: alb
     rules:
       - http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: hello-openshift
                   port:
                     number: 8080
   EOF

   Copy code to clipboard

5. Curl the AWS ALB Ingress endpoint to verify the hello-world application is accessible:

   Note

   AWS ALB provisioning takes a few minutes. If you receive an error that says curl: (6) Could not resolve host, please wait and try again.

   $ INGRESS=$(oc -n hello-world get ingress hello-openshift-alb -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   $ curl "http://${INGRESS}"

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

Procedure

1. Create an AWS WAF rules file to apply to your web access control list (ACL):

   $ cat << EOF > ${SCRATCH}/waf-rules.json
   [
       {
         "Name": "AWS-AWSManagedRulesCommonRuleSet",
         "Priority": 0,
         "Statement": {
           "ManagedRuleGroupStatement": {
             "VendorName": "AWS",
             "Name": "AWSManagedRulesCommonRuleSet"
           }
         },
         "OverrideAction": {
           "None": {}
         },
         "VisibilityConfig": {
           "SampledRequestsEnabled": true,
           "CloudWatchMetricsEnabled": true,
           "MetricName": "AWS-AWSManagedRulesCommonRuleSet"
         }
       },
       {
         "Name": "AWS-AWSManagedRulesSQLiRuleSet",
         "Priority": 1,
         "Statement": {
           "ManagedRuleGroupStatement": {
             "VendorName": "AWS",
             "Name": "AWSManagedRulesSQLiRuleSet"
           }
         },
         "OverrideAction": {
           "None": {}
         },
         "VisibilityConfig": {
           "SampledRequestsEnabled": true,
           "CloudWatchMetricsEnabled": true,
           "MetricName": "AWS-AWSManagedRulesSQLiRuleSet"
         }
       }
   ]
   EOF

   Copy code to clipboard

   This enables the Core (Common) and SQL AWS Managed Rule Sets.

2. Create an AWS WAF Web ACL using the rules you specified in the previous steps:

   $ WAF_ARN=$(aws wafv2 create-web-acl \
     --name ${CLUSTER}-waf \
     --region ${REGION} \
     --default-action Allow={} \
     --scope REGIONAL \
     --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=${CLUSTER}-waf-metrics \
     --rules file://${SCRATCH}/waf-rules.json \
     --query 'Summary.ARN' \
     --output text)

   Copy code to clipboard

3. Annotate the Ingress resource with the AWS WAF Web ACL (Amazon Resource Name) ARN:

   $ oc annotate -n hello-world ingress.networking.k8s.io/hello-openshift-alb \
     alb.ingress.kubernetes.io/wafv2-acl-arn=${WAF_ARN}

   Copy code to clipboard

Verification

1. Wait 10 seconds for the rules to propagate and test that the application still works:

   $ curl "http://${INGRESS}"

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

2. Test that the WAF denies a bad request:

   $ curl -X POST "http://${INGRESS}" \
     -F "user='<script><alert>Hello></alert></script>'"

   Copy code to clipboard

   Example output

   <html>
   <head><title>403 Forbidden</title></head>
   <body>
   <center><h1>403 Forbidden</h1></center>
   </body>
   </html

   Copy code to clipboard

   Note

   Activation of the AWS WAF integration can sometimes take several minutes. If you do not receive a 403 Forbidden error, please wait a few seconds and try again.

   The expected result is a 403 Forbidden error, which means the AWS WAF is protecting your application.

Procedure

1. Create an IAM policy to enable S3 access:

   $ POLICY_ARN=$(aws iam list-policies --query "Policies[?PolicyName=='RosaOadpVer1'].{ARN:Arn}" --output text)
   if [[ -z "${POLICY_ARN}" ]]; then
   $ cat << EOF > ${SCRATCH}/policy.json
   {
   "Version": "2012-10-17",
   "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:PutBucketTagging",
        "s3:GetBucketTagging",
        "s3:PutEncryptionConfiguration",
        "s3:GetEncryptionConfiguration",
        "s3:PutLifecycleConfiguration",
        "s3:GetLifecycleConfiguration",
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucketMultipartUploads",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts",
        "ec2:DescribeSnapshots",
        "ec2:DescribeVolumes",
        "ec2:DescribeVolumeAttribute",
        "ec2:DescribeVolumesModifications",
        "ec2:DescribeVolumeStatus",
        "ec2:CreateTags",
        "ec2:CreateVolume",
        "ec2:CreateSnapshot",
        "ec2:DeleteSnapshot"
      ],
      "Resource": "*"
    }
   ]}
   EOF
   $ POLICY_ARN=$(aws iam create-policy --policy-name "RosaOadpVer1" \
   --policy-document file:///${SCRATCH}/policy.json --query Policy.Arn \
   --tags Key=rosa_openshift_version,Value=${CLUSTER_VERSION} Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-oadp Key=operator_name,Value=openshift-oadp \
   --output text)
   fi
   $ echo ${POLICY_ARN}

   Copy code to clipboard

2. Create an IAM role trust policy for the cluster:

   $ cat <<EOF > ${SCRATCH}/trust-policy.json
   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Principal": {
         "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_ENDPOINT}"
       },
       "Action": "sts:AssumeRoleWithWebIdentity",
       "Condition": {
         "StringEquals": {
            "${OIDC_ENDPOINT}:sub": [
              "system:serviceaccount:openshift-adp:openshift-adp-controller-manager",
              "system:serviceaccount:openshift-adp:velero"]
         }
       }
     }]
   }
   EOF
   $ ROLE_ARN=$(aws iam create-role --role-name \
    "${ROLE_NAME}" \
     --assume-role-policy-document file://${SCRATCH}/trust-policy.json \
     --tags Key=rosa_cluster_id,Value=${ROSA_CLUSTER_ID} Key=rosa_openshift_version,Value=${CLUSTER_VERSION} Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-adp Key=operator_name,Value=openshift-oadp \
     --query Role.Arn --output text)
   
   $ echo ${ROLE_ARN}

   Copy code to clipboard

3. Attach the IAM policy to the IAM role:

   $ aws iam attach-role-policy --role-name "${ROLE_NAME}" \
    --policy-arn ${POLICY_ARN}

   Copy code to clipboard

Procedure

1. Create a namespace for OADP:

   $ oc create namespace openshift-adp

   Copy code to clipboard

2. Create a credentials secret:

   $ cat <<EOF > ${SCRATCH}/credentials
   [default]
   role_arn = ${ROLE_ARN}
   web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
   region=<aws_region>
   EOF
   $ oc -n openshift-adp create secret generic cloud-credentials \
    --from-file=${SCRATCH}/credentials

   Copy code to clipboard

   where:

   <aws_region>

   Replace with the AWS region to use for the Security Token Service (STS) endpoint.

3. Deploy the OADP Operator:

   Note

   There is currently an issue with version 1.1 of the Operator with backups that have a PartiallyFailed status. This does not seem to affect the backup and restore process, but it should be noted as there are issues with it.

   $ cat << EOF | oc create -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
    generateName: openshift-adp-
    namespace: openshift-adp
    name: oadp
   spec:
    targetNamespaces:
    - openshift-adp
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
    name: redhat-oadp-operator
    namespace: openshift-adp
   spec:
    channel: stable-1.2
    installPlanApproval: Automatic
    name: redhat-oadp-operator
    source: redhat-operators
    sourceNamespace: openshift-marketplace
   EOF

   Copy code to clipboard

4. Wait for the Operator to be ready:

   $ watch oc -n openshift-adp get pods

   Copy code to clipboard

   Example output

   NAME                                                READY   STATUS    RESTARTS   AGE
   openshift-adp-controller-manager-546684844f-qqjhn   1/1     Running   0          22s

   Copy code to clipboard

5. Create Cloud Storage:

   $ cat << EOF | oc create -f -
   apiVersion: oadp.openshift.io/v1alpha1
   kind: CloudStorage
   metadata:
    name: ${CLUSTER_NAME}-oadp
    namespace: openshift-adp
   spec:
    creationSecret:
      key: credentials
      name: cloud-credentials
    enableSharedConfig: true
    name: ${CLUSTER_NAME}-oadp
    provider: aws
    region: $REGION
   EOF

   Copy code to clipboard

6. Check your application’s storage default storage class by running the following command with your desired namespace:

   $ oc get pvc -n <namespace>

   Copy code to clipboard

   Example output

   NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
   applog   Bound    pvc-351791ae-b6ab-4e8b-88a4-30f73caf5ef8   1Gi        RWO            gp3-csi        4d19h
   mysql    Bound    pvc-16b8e009-a20a-4379-accc-bc81fedd0621   1Gi        RWO            gp3-csi        4d19h

   Copy code to clipboard

   $ oc get storageclass

   Copy code to clipboard

   Example output

   NAME                PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
   gp2                 kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   true                   4d21h
   gp2-csi             ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
   gp3                 ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
   gp3-csi (default)   ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h

   Copy code to clipboard

   Using either gp3-csi, gp2-csi, gp3 or gp2 will work. If the application(s) that are being backed up are all using Persistent Volumes (PVs) with the Container Storage Interface (CSI), include the CSI plugin in the OADP Data Protection Application (DPA) configuration.

7. CSI only: Deploy a Data Protection Application:

   $ cat << EOF | oc create -f -
   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionApplication
   metadata:
    name: ${CLUSTER_NAME}-dpa
    namespace: openshift-adp
   spec:
    backupImages: true
    features:
      dataMover:
        enable: false
    backupLocations:
    - bucket:
        cloudStorageRef:
          name: ${CLUSTER_NAME}-oadp
        credential:
          key: credentials
          name: cloud-credentials
        prefix: velero
        default: true
        config:
          region: ${REGION}
    configuration:
      velero:
        defaultPlugins:
        - openshift
        - aws
        - csi
      restic:
        enable: false
   EOF

   Copy code to clipboard
   Note

   If you run this command for CSI volumes, you can skip the next step.

8. Non-CSI volumes: Deploy a Data Protection Application:

   $ cat << EOF | oc create -f -
   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionApplication
   metadata:
    name: ${CLUSTER_NAME}-dpa
    namespace: openshift-adp
   spec:
    backupImages: true
    features:
      dataMover:
        enable: false
    backupLocations:
    - bucket:
        cloudStorageRef:
          name: ${CLUSTER_NAME}-oadp
        credential:
          key: credentials
          name: cloud-credentials
        prefix: velero
        default: true
        config:
          region: ${REGION}
    configuration:
      velero:
        defaultPlugins:
        - openshift
        - aws
      restic:
        enable: false
    snapshotLocations:
      - velero:
          config:
            credentialsFile: /tmp/credentials/openshift-adp/cloud-credentials-credentials
            enableSharedConfig: 'true'
            profile: default
            region: ${REGION}
          provider: aws
   EOF

   Copy code to clipboard
   Note

   • In OADP 1.1.x Red Hat OpenShift Service on AWS STS environments, the container image backup and restore (spec.backupImages) value must be set to false as it is not supported.
   • The Restic feature (restic.enable=false) is disabled and not supported in Red Hat OpenShift Service on AWS STS environments.
   • The DataMover feature (dataMover.enable=false) is disabled and not supported in Red Hat OpenShift Service on AWS STS environments.

Procedure

1. Create a workload to back up:

   1. Create a namespace by running:

      $ oc create namespace hello-world

      Copy code to clipboard

   2. Create an application by running:

      $ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift

      Copy code to clipboard

2. Expose the route:

   $ oc expose service/hello-openshift -n hello-world

   Copy code to clipboard

3. Check that the application is working:

   $ curl `oc get route/hello-openshift -n hello-world -o jsonpath='{.spec.host}'`

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

4. Back up the workload:

   $ cat << EOF | oc create -f -
   apiVersion: velero.io/v1
   kind: Backup
   metadata:
    name: hello-world
    namespace: openshift-adp
   spec:
    includedNamespaces:
    - hello-world
    storageLocation: ${CLUSTER_NAME}-dpa-1
    ttl: 720h0m0s
   EOF

   Copy code to clipboard

5. Wait until the backup is done:

   $ watch "oc -n openshift-adp get backup hello-world -o json | jq .status"

   Copy code to clipboard

   Example output

   {
    "completionTimestamp": "2022-09-07T22:20:44Z",
    "expiration": "2022-10-07T22:20:22Z",
    "formatVersion": "1.1.0",
    "phase": "Completed",
    "progress": {
      "itemsBackedUp": 58,
      "totalItems": 58
    },
    "startTimestamp": "2022-09-07T22:20:22Z",
    "version": 1
   }

   Copy code to clipboard

6. Delete the demo workload:

   $ oc delete ns hello-world

   Copy code to clipboard

7. Restore from the backup:

   $ cat << EOF | oc create -f -
   apiVersion: velero.io/v1
   kind: Restore
   metadata:
    name: hello-world
    namespace: openshift-adp
   spec:
    backupName: hello-world
   EOF

   Copy code to clipboard

8. Wait for the restore to finish:

   $ watch "oc -n openshift-adp get restore hello-world -o json | jq .status"

   Copy code to clipboard

   Example output

   {
    "completionTimestamp": "2022-09-07T22:25:47Z",
    "phase": "Completed",
    "progress": {
      "itemsRestored": 38,
      "totalItems": 38
    },
    "startTimestamp": "2022-09-07T22:25:28Z",
    "warnings": 9
   }

   Copy code to clipboard

9. Check that the workload is restored:

   $ oc -n hello-world get pods

   Copy code to clipboard

   Example output

   NAME                              READY   STATUS    RESTARTS   AGE
   hello-openshift-9f885f7c6-kdjpj   1/1     Running   0          90s

   Copy code to clipboard

   $ curl `oc get route/hello-openshift -n hello-world -o jsonpath='{.spec.host}'`

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

Procedure

1. Delete the workload:

   $ oc delete ns hello-world

   Copy code to clipboard

2. Remove the backup and restore resources from the cluster if they are no longer required:

   $ oc delete backups.velero.io hello-world
   $ oc delete restores.velero.io hello-world

   Copy code to clipboard

3. Delete the backup, restore, and remote objects in S3:

   $ velero backup delete hello-world
   $ velero restore delete hello-world

   Copy code to clipboard

4. Delete the Data Protection Application:

   $ oc -n openshift-adp delete dpa ${CLUSTER_NAME}-dpa

   Copy code to clipboard

5. Delete the Cloud Storage:

   $ oc -n openshift-adp delete cloudstorage ${CLUSTER_NAME}-oadp

   Copy code to clipboard
   Warning

   If this command hangs, you might need to delete the finalizer:

   $ oc -n openshift-adp patch cloudstorage ${CLUSTER_NAME}-oadp -p '{"metadata":{"finalizers":null}}' --type=merge

   Copy code to clipboard

6. Remove the Operator if it is no longer required:

   $ oc -n openshift-adp delete subscription oadp-operator

   Copy code to clipboard

7. Remove the namespace for the Operator:

   $ oc delete ns redhat-openshift-adp

   Copy code to clipboard

8. Remove the Custom Resource Definitions from the cluster if you no longer wish to have them:

   $ for CRD in `oc get crds | grep velero | awk '{print $1}'`; do oc delete crd $CRD; done
   $ for CRD in `oc get crds | grep -i oadp | awk '{print $1}'`; do oc delete crd $CRD; done

   Copy code to clipboard

9. Delete the AWS S3 Bucket:

   $ aws s3 rm s3://${CLUSTER_NAME}-oadp --recursive
   $ aws s3api delete-bucket --bucket ${CLUSTER_NAME}-oadp

   Copy code to clipboard

10. Detach the policy from the role:

    $ aws iam detach-role-policy --role-name "${ROLE_NAME}" \
     --policy-arn "${POLICY_ARN}"

    Copy code to clipboard

11. Delete the role:

    $ aws iam delete-role --role-name "${ROLE_NAME}"

    Copy code to clipboard

Procedure

1. Create an AWS Identity and Access Management (IAM) policy for the AWS Load Balancer Controller:

   Note

   The policy is sourced from Content from raw.githubusercontent.com is not included.the upstream AWS Load Balancer Controller policy plus permission to create tags on subnets. This is required by the Operator to function.

   $ oc new-project aws-load-balancer-operator
   $ POLICY_ARN=$(aws iam list-policies --query \
        "Policies[?PolicyName=='aws-load-balancer-operator-policy'].{ARN:Arn}" \
        --output text)
   $ if [[ -z "${POLICY_ARN}" ]]; then
       wget -O "${SCRATCH}/load-balancer-operator-policy.json" \
          https://raw.githubusercontent.com/rh-mobb/documentation/main/content/rosa/aws-load-balancer-operator/load-balancer-operator-policy.json
        POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn \
        --output text iam create-policy \
        --policy-name aws-load-balancer-operator-policy \
        --policy-document "file://${SCRATCH}/load-balancer-operator-policy.json")
   fi
   $ echo $POLICY_ARN

   Copy code to clipboard

2. Create an AWS IAM trust policy for AWS Load Balancer Operator:

   $ cat <<EOF > "${SCRATCH}/trust-policy.json"
   {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Condition": {
      "StringEquals" : {
        "${OIDC_ENDPOINT}:sub": ["system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-operator-controller-manager", "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster"]
      }
    },
    "Principal": {
      "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
    },
    "Action": "sts:AssumeRoleWithWebIdentity"
    }
    ]
   }
   EOF

   Copy code to clipboard

3. Create an AWS IAM role for the AWS Load Balancer Operator:

   $ ROLE_ARN=$(aws iam create-role --role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
      --assume-role-policy-document "file://${SCRATCH}/trust-policy.json" \
      --query Role.Arn --output text)
   $ echo $ROLE_ARN
   
   $ aws iam attach-role-policy --role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
        --policy-arn $POLICY_ARN

   Copy code to clipboard

4. Create a secret for the AWS Load Balancer Operator to assume the newly created AWS IAM role:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Secret
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   stringData:
     credentials: |
       [default]
       role_arn = $ROLE_ARN
       web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
   EOF

   Copy code to clipboard

5. Install the AWS Load Balancer Operator:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   spec:
     upgradeStrategy: Default
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: aws-load-balancer-operator
     namespace: aws-load-balancer-operator
   spec:
     channel: stable-v1.0
     installPlanApproval: Automatic
     name: aws-load-balancer-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     startingCSV: aws-load-balancer-operator.v1.0.0
   EOF

   Copy code to clipboard

6. Deploy an instance of the AWS Load Balancer Controller using the Operator:

   Note

   If you get an error, the Operator has not completed installing yet. Wait briefly and retry the step.

   $ cat << EOF | oc apply -f -
   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     credentials:
       name: aws-load-balancer-operator
   EOF

   Copy code to clipboard

7. Check that the Operator and controller pods are both running:

   $ oc -n aws-load-balancer-operator get pods

   Copy code to clipboard

   Example output

   NAME                                                             READY   STATUS    RESTARTS   AGE
   aws-load-balancer-controller-cluster-6ddf658785-pdp5d            1/1     Running   0          99s
   aws-load-balancer-operator-controller-manager-577d9ffcb9-w6zqn   2/2     Running   0          2m4s

   Copy code to clipboard

Procedure

1. Create a new project:

   $ oc new-project hello-world

   Copy code to clipboard

2. Deploy a hello-world application:

   $ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift

   Copy code to clipboard

3. Configure a NodePort service for the AWS Application Load Balancer (ALB) to connect to:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: hello-openshift-nodeport
     namespace: hello-world
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: NodePort
     selector:
       deployment: hello-openshift
   EOF

   Copy code to clipboard

4. Deploy an AWS ALB using the AWS Load Balancer Operator:

   $ cat << EOF | oc apply -f -
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: hello-openshift-alb
     namespace: hello-world
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
   spec:
     ingressClassName: alb
     rules:
       - http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: hello-openshift-nodeport
                   port:
                     number: 80
   EOF

   Copy code to clipboard

5. Curl the AWS ALB Ingress endpoint to verify the hello-world application is accessible:

   Note

   AWS ALB provisioning takes a few minutes. If you receive an error that says curl: (6) Could not resolve host, wait and try again.

   $ INGRESS=$(oc -n hello-world get ingress hello-openshift-alb \
       -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   $ curl "http://${INGRESS}"

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

6. Deploy an AWS Network Load Balancer (NLB) for your hello-world application:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: hello-openshift-nlb
     namespace: hello-world
     annotations:
       service.beta.kubernetes.io/aws-load-balancer-type: external
       service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
       service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: LoadBalancer
     selector:
       deployment: hello-openshift
   EOF

   Copy code to clipboard

7. Test the AWS NLB endpoint:

   Note

   NLB provisioning takes a few minutes. If you receive an error that says curl: (6) Could not resolve host, wait and try again.

   $ NLB=$(oc -n hello-world get service hello-openshift-nlb \
     -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   $ curl "http://${NLB}"

   Copy code to clipboard

   Example output

   Hello OpenShift!

   Copy code to clipboard

Procedure

1. Delete the hello-world application namespace and all the resources in the namespace:

   $ oc delete project hello-world

   Copy code to clipboard

2. Delete the AWS Load Balancer Operator and the AWS IAM roles:

   $ oc delete subscription aws-load-balancer-operator -n aws-load-balancer-operator
   $ aws iam detach-role-policy \
     --role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
     --policy-arn $POLICY_ARN
   $ aws iam delete-role \
     --role-name "${ROSA_CLUSTER_NAME}-alb-operator"

   Copy code to clipboard

3. Delete the AWS IAM policy:

   $ aws iam delete-policy --policy-arn $POLICY_ARN

   Copy code to clipboard

Procedure

1. Create the cluster’s OAuth callback URL by changing the specified variables and running the following command:

   $ domain=$(rosa describe cluster -c <cluster_name> | grep "DNS" | grep -oE '\S+.openshiftapps.com')
   echo "OAuth callback URL: https://oauth.${domain}/oauth2callback/AAD"

   Copy code to clipboard

   The AAD directory at the end of the OAuth callback URL must match the OAuth identity provider name that you will set up later in this process.

   Note

   Remember to save this callback URL; it will be required later in the process.

2. Create the Entra ID application by logging in to the Azure portal, and select the Content from portal.azure.com is not included.App registrations blade. Then, select New registration to create a new application.

   

3. Name the application, for example openshift-auth.
4. Select Web from the Redirect URI dropdown and enter the value of the OAuth callback URL you retrieved in the previous step.

5. After providing the required information, click Register to create the application.

   

6. Select the Certificates & secrets sub-blade and select New client secret.

   

7. Complete the requested details and store the generated client secret value. This secret is required later in this process.

   Important

   After initial setup, you cannot see the client secret. If you did not record the client secret, you must generate a new one.

   

8. Select the Overview sub-blade and note the Application (client) ID and Directory (tenant) ID. You will need these values in a future step.

   

Procedure

1. Create the variables necessary for the identity provider configuration by running the following command:

   $ CLUSTER_NAME=example-cluster
   $ IDP_NAME=AAD
   $ APP_ID=yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy
   $ CLIENT_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
   $ TENANT_ID=zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz

   Copy code to clipboard

   where:

   example-cluster

   Replace this with the name of your cluster.

   AAD

   Replace this value with the name you used in the OAuth callback URL that you generated earlier in this process.

   yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy

   Replace this with the Application (client) ID.

   xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

   Replace this with the Client Secret.

   zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz

   Replace this with the Directory (tenant) ID.

2. Configure the cluster’s OAuth provider by running the following command. If you enabled group claims, ensure that you use the --group-claims groups argument.

   • If you enabled group claims, run the following command:

     $ rosa create idp \
     --cluster ${CLUSTER_NAME} \
     --type openid \
     --name ${IDP_NAME} \
     --client-id ${APP_ID} \
     --client-secret ${CLIENT_SECRET} \
     --issuer-url https://login.microsoftonline.com/${TENANT_ID}/v2.0 \
     --email-claims email \
     --name-claims name \
     --username-claims preferred_username \
     --extra-scopes email,profile \
     --groups-claims groups

     Copy code to clipboard

   • If you did not enable group claims, run the following command:

     $ rosa create idp \
     --cluster ${CLUSTER_NAME} \
     --type openid \
     --name ${IDP_NAME} \
     --client-id ${APP_ID} \
     --client-secret ${CLIENT_SECRET} \
     --issuer-url https://login.microsoftonline.com/${TENANT_ID}/v2.0 \
     --email-claims email \
     --name-claims name \
     --username-claims preferred_username \
     --extra-scopes email,profile

     Copy code to clipboard

     After a few minutes, the cluster authentication Operator reconciles your changes, and you can log in to the cluster by using Entra ID.

Procedure

1. Log in to your Red Hat OpenShift Service on AWS cluster by running the following command:

   $ oc login --token=<your-token> --server=<your-server-url>

   Copy code to clipboard

   You can find your login token by accessing your cluster in This content is not included.pull secret from Red Hat OpenShift Cluster Manager.

2. Validate that your cluster has STS by running the following command:

   $ oc get authentication.config.openshift.io cluster -o json \
     | jq .spec.serviceAccountIssuer

   Copy code to clipboard

   Example output

   "https://xxxxx.cloudfront.net/xxxxx"

   Copy code to clipboard

   If your output is different, do not proceed. See Red Hat documentation on creating an STS cluster before continuing this process.

3. Set the SecurityContextConstraints permission to allow the Container Storage Interface (CSI) driver to run by running the following command:

   $ oc new-project csi-secrets-store
   $ oc adm policy add-scc-to-user privileged \
       system:serviceaccount:csi-secrets-store:secrets-store-csi-driver
   $ oc adm policy add-scc-to-user privileged \
       system:serviceaccount:csi-secrets-store:csi-secrets-store-provider-aws

   Copy code to clipboard

4. Create environment variables to use later in this process by running the following command:

   $ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
   $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster \
      -o jsonpath='{.spec.serviceAccountIssuer}' | sed  's|^https://||')
   $ export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
   $ export AWS_PAGER=""

   Copy code to clipboard

Procedure

1. Use Helm to register the Secrets Store CSI Driver by running the following command:

   $ helm repo add secrets-store-csi-driver \
       https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts

   Copy code to clipboard

2. Update your Helm repositories by running the following command:

   $ helm repo update

   Copy code to clipboard

3. Install the Secrets Store CSI Driver by running the following command:

   $ helm upgrade --install -n csi-secrets-store \
       csi-secrets-store-driver secrets-store-csi-driver/secrets-store-csi-driver

   Copy code to clipboard

4. Deploy the AWS provider by running the following command:

   $ oc -n csi-secrets-store apply -f \
       https://raw.githubusercontent.com/rh-mobb/documentation/main/content/misc/secrets-store-csi/aws-provider-installer.yaml

   Copy code to clipboard

5. Check that both Daemonsets are running by running the following command:

   $ oc -n csi-secrets-store get ds \
       csi-secrets-store-provider-aws \
       csi-secrets-store-driver-secrets-store-csi-driver

   Copy code to clipboard

6. Label the Secrets Store CSI Driver to allow use with the restricted pod security profile by running the following command:

   $ oc label csidriver.storage.k8s.io/secrets-store.csi.k8s.io security.openshift.io/csi-ephemeral-volume-profile=restricted

   Copy code to clipboard

Procedure

1. Create a secret in AWS Secrets Manager by running the following command:

   $ SECRET_ARN=$(aws --region "$REGION" secretsmanager create-secret \
       --name MySecret --secret-string \
       '{"username":"shadowman", "password":"hunter2"}' \
       --query ARN --output text); echo $SECRET_ARN

   Copy code to clipboard

2. Create an IAM Access Policy document by running the following command:

   $ cat << EOF > policy.json
   {
      "Version": "2012-10-17",
      "Statement": [{
         "Effect": "Allow",
         "Action": [
           "secretsmanager:GetSecretValue",
           "secretsmanager:DescribeSecret"
         ],
         "Resource": ["$SECRET_ARN"]
         }]
   }
   EOF

   Copy code to clipboard

3. Create an IAM Access Policy by running the following command:

   $ POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn \
   --output text iam create-policy \
   --policy-name openshift-access-to-mysecret-policy \
   --policy-document file://policy.json); echo $POLICY_ARN

   Copy code to clipboard

4. Create an IAM Role trust policy document by running the following command:

   Note

   The trust policy is locked down to the default service account of a namespace you create later in this process.

   $ cat <<EOF > trust-policy.json
   {
      "Version": "2012-10-17",
      "Statement": [
      {
      "Effect": "Allow",
      "Condition": {
        "StringEquals" : {
          "${OIDC_ENDPOINT}:sub": ["system:serviceaccount:my-application:default"]
         }
       },
       "Principal": {
          "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
       },
       "Action": "sts:AssumeRoleWithWebIdentity"
       }
       ]
   }
   EOF

   Copy code to clipboard

5. Create an IAM role by running the following command:

   $ ROLE_ARN=$(aws iam create-role --role-name openshift-access-to-mysecret \
   --assume-role-policy-document file://trust-policy.json \
   --query Role.Arn --output text); echo $ROLE_ARN

   Copy code to clipboard

6. Attach the role to the policy by running the following command:

   $ aws iam attach-role-policy --role-name openshift-access-to-mysecret \
       --policy-arn $POLICY_ARN

   Copy code to clipboard

Procedure

1. Create an OpenShift project by running the following command:

   $ oc new-project my-application

   Copy code to clipboard

2. Annotate the default service account to use the Security Token Service (STS) role by running the following command:

   $ oc annotate -n my-application serviceaccount default \
       eks.amazonaws.com/role-arn=$ROLE_ARN

   Copy code to clipboard

3. Create a secret provider class to access our secret by running the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: my-application-aws-secrets
   spec:
     provider: aws
     parameters:
       objects: |
         - objectName: "MySecret"
           objectType: "secretsmanager"
   EOF

   Copy code to clipboard

4. Create a deployment by using our secret in the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: my-application
     labels:
       app: my-application
   spec:
     volumes:
     - name: secrets-store-inline
       csi:
         driver: secrets-store.csi.k8s.io
         readOnly: true
         volumeAttributes:
           secretProviderClass: "my-application-aws-secrets"
     containers:
     - name: my-application-deployment
       image: k8s.gcr.io/e2e-test-images/busybox:1.29
       command:
         - "/bin/sleep"
         - "10000"
       volumeMounts:
       - name: secrets-store-inline
         mountPath: "/mnt/secrets-store"
         readOnly: true
   EOF

   Copy code to clipboard

Procedure

1. Delete the application by running the following command:

   $ oc delete project my-application

   Copy code to clipboard

2. Delete the secrets store Container Storage Interface (CSI) driver by running the following command:

   $ helm delete -n csi-secrets-store csi-secrets-store-driver

   Copy code to clipboard

3. Delete the security context constraints by running the following command:

   $ oc adm policy remove-scc-from-user privileged \
       system:serviceaccount:csi-secrets-store:secrets-store-csi-driver; oc adm policy remove-scc-from-user privileged \
       system:serviceaccount:csi-secrets-store:csi-secrets-store-provider-aws

   Copy code to clipboard

4. Delete the AWS provider by running the following command:

   $ oc -n csi-secrets-store delete -f \
   https://raw.githubusercontent.com/rh-mobb/documentation/main/content/misc/secrets-store-csi/aws-provider-installer.yaml

   Copy code to clipboard

5. Delete AWS roles and policies by running the following command:

   $ aws iam detach-role-policy --role-name openshift-access-to-mysecret \
       --policy-arn $POLICY_ARN; aws iam delete-role --role-name openshift-access-to-mysecret; aws iam delete-policy --policy-arn $POLICY_ARN

   Copy code to clipboard

6. Delete the Secrets Manager secret by running the following command:

   $ aws secretsmanager --region $REGION delete-secret --secret-id $SECRET_ARN

   Copy code to clipboard

Procedure

1. Configure the following environment variables, changing the cluster name to suit your cluster:

   $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}"  | sed 's/-[a-z0-9]\{5\}$//')
   $ export REGION=$(rosa describe cluster -c ${ROSA_CLUSTER_NAME} --output json | jq -r .region.id)
   $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o json | jq -r .spec.serviceAccountIssuer | sed  's|^https://||')
   $ export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
   $ export ACK_SERVICE=s3
   $ export ACK_SERVICE_ACCOUNT=ack-${ACK_SERVICE}-controller
   $ export POLICY_ARN=arn:aws:iam::aws:policy/AmazonS3FullAccess
   $ export AWS_PAGER=""
   $ export SCRATCH="/tmp/${ROSA_CLUSTER_NAME}/ack"
   $ mkdir -p ${SCRATCH}

   Copy code to clipboard

2. Ensure all fields output correctly:

   $ echo "Cluster: ${ROSA_CLUSTER_NAME}, Region: ${REGION}, OIDC Endpoint: ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"

   Copy code to clipboard

Procedure

1. Create an AWS IAM trust policy for the ACK Operator:

   $ cat <<EOF > "${SCRATCH}/trust-policy.json"
   {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Condition": {
      "StringEquals" : {
        "${OIDC_ENDPOINT}:sub": "system:serviceaccount:ack-system:${ACK_SERVICE_ACCOUNT}"
      }
    },
    "Principal": {
      "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
    },
    "Action": "sts:AssumeRoleWithWebIdentity"
    }
    ]
   }
   EOF

   Copy code to clipboard

2. Create an AWS IAM role for the ACK Operator to assume with the AmazonS3FullAccess policy attached:

   Note

   You can find the recommended policy in each project’s GitHub repository, for example Content from github.com is not included.https://github.com/aws-controllers-k8s/s3-controller/blob/main/config/iam/recommended-policy-arn.

   $ ROLE_ARN=$(aws iam create-role --role-name "ack-${ACK_SERVICE}-controller" \
      --assume-role-policy-document "file://${SCRATCH}/trust-policy.json" \
      --query Role.Arn --output text)

   Copy code to clipboard

   $ echo $ROLE_ARN

   Copy code to clipboard

   $ aws iam attach-role-policy --role-name "ack-${ACK_SERVICE}-controller" \
        --policy-arn ${POLICY_ARN}

   Copy code to clipboard

Procedure

1. Create a project to install the ACK S3 Operator into:

   $ oc new-project ack-system

   Copy code to clipboard

2. Create a file with the ACK S3 Operator configuration:

   Note

   ACK_WATCH_NAMESPACE is purposely left blank so the controller can properly watch all namespaces in the cluster.

   $ cat << EOF  "${SCRATCH}/config.txt"
   ACK_ENABLE_DEVELOPMENT_LOGGING=true
   ACK_LOG_LEVEL=debug
   ACK_WATCH_NAMESPACE=
   AWS_REGION=${REGION}
   AWS_ENDPOINT_URL=
   ACK_RESOURCE_TAGS=${CLUSTER_NAME}
   ENABLE_LEADER_ELECTION=true
   LEADER_ELECTION_NAMESPACE=
   RECONCILE_DEFAULT_MAX_CONCURRENT_SYNCS=1
   FEATURE_FLAGS=
   FEATURE_GATES=
   EOF

   Copy code to clipboard

3. Use the file from the previous step to create a ConfigMap:

   $ oc -n ack-system create configmap \
     --from-env-file=${SCRATCH}/config.txt ack-${ACK_SERVICE}-user-config

   Copy code to clipboard

4. Install the ACK S3 Operator from the software catalog:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: ack-${ACK_SERVICE}-controller
     namespace: ack-system
   spec:
     upgradeStrategy: Default
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: ack-${ACK_SERVICE}-controller
     namespace: ack-system
   spec:
     channel: alpha
     installPlanApproval: Automatic
     name: ack-${ACK_SERVICE}-controller
     source: community-operators
     sourceNamespace: openshift-marketplace
   EOF

   Copy code to clipboard

5. Annotate the ACK S3 Operator service account with the AWS IAM role to assume and restart the deployment:

   $ oc -n ack-system annotate serviceaccount ${ACK_SERVICE_ACCOUNT} \
     eks.amazonaws.com/role-arn=${ROLE_ARN} && \
     oc -n ack-system rollout restart deployment ack-${ACK_SERVICE}-controller

   Copy code to clipboard

Verification

1. Verify that the ACK S3 Operator is running:

   $ oc -n ack-system get pods

   Copy code to clipboard

   Example output

   NAME                                 READY   STATUS    RESTARTS   AGE
   ack-s3-controller-585f6775db-s4lfz   1/1     Running   0          51s

   Copy code to clipboard

Procedure

1. Deploy an S3 bucket resource:

   $ cat << EOF | oc apply -f -
   apiVersion: s3.services.k8s.aws/v1alpha1
   kind: Bucket
   metadata:
      name: ${CLUSTER-NAME}-bucket
      namespace: ack-system
   spec:
      name: ${CLUSTER-NAME}-bucket
   EOF

   Copy code to clipboard

2. Verify the S3 bucket was created in AWS:

   $ aws s3 ls | grep ${CLUSTER_NAME}-bucket

   Copy code to clipboard

   Example output

   2023-10-04 14:51:45 mrmc-test-maz-bucket

   Copy code to clipboard

Procedure

1. Delete the S3 bucket resource:

   $ oc -n ack-system delete bucket.s3.services.k8s.aws/${CLUSTER-NAME}-bucket

   Copy code to clipboard

2. Delete the AWS Controller for Kubernetes (ACK) S3 Operator and the AWS Identity and Access Management (IAM) roles:

   $ oc -n ack-system delete subscription ack-${ACK_SERVICE}-controller

   Copy code to clipboard

   $ aws iam detach-role-policy \
     --role-name "ack-${ACK_SERVICE}-controller" \
     --policy-arn ${POLICY_ARN}

   Copy code to clipboard

   $ aws iam delete-role \
     --role-name "ack-${ACK_SERVICE}-controller"

   Copy code to clipboard

3. Delete the ack-system project:

   $ oc delete project ack-system

   Copy code to clipboard

Procedure

1. Create a new project by running the following command:

   $ oc new-project demo-egress-ns

   Copy code to clipboard

2. Create the egress rule for all pods within the namespace by running the following command:

   $ cat <<EOF | oc apply -f -
   apiVersion: k8s.ovn.org/v1
   kind: EgressIP
   metadata:
     name: demo-egress-ns
   spec:
     # NOTE: these egress IPs are within the subnet range(s) in which my worker nodes
     #       are deployed.
     egressIPs:
       - 10.10.100.253
       - 10.10.150.253
       - 10.10.200.253
     namespaceSelector:
       matchLabels:
         kubernetes.io/metadata.name: demo-egress-ns
   EOF

   Copy code to clipboard

Procedure

1. Create a new project by running the following command:

   $ oc new-project demo-egress-pod

   Copy code to clipboard

2. Create the egress rule for the pod by running the following command:

   Note

   spec.namespaceSelector is a mandatory field.

   $ cat <<EOF | oc apply -f -
   apiVersion: k8s.ovn.org/v1
   kind: EgressIP
   metadata:
     name: demo-egress-pod
   spec:
     # NOTE: these egress IPs are within the subnet range(s) in which my worker nodes
     #       are deployed.
     egressIPs:
       - 10.10.100.254
       - 10.10.150.254
       - 10.10.200.254
     namespaceSelector:
       matchLabels:
         kubernetes.io/metadata.name: demo-egress-pod
     podSelector:
       matchLabels:
         run: demo-egress-pod
   EOF

   Copy code to clipboard

Procedure

1. Clean up your cluster by running the following commands:

   $ oc delete svc demo-service -n default
   $ oc delete pod demo-service -n default
   $ oc delete project demo-egress-ns
   $ oc delete project demo-egress-pod
   $ oc delete egressip demo-egress-ns
   $ oc delete egressip demo-egress-pod

   Copy code to clipboard

2. Clean up the assigned node labels by running the following command:

   Warning

   If you rely on node labels for your machine pool, this command replaces those labels. Input your desired labels into the --labels field to ensure your node labels remain.

   $ rosa update machinepool ${ROSA_MACHINE_POOL_NAME} \
     --cluster="${ROSA_CLUSTER_NAME}" \
     --labels ""

   Copy code to clipboard