Procedure

1. Optional: Set the following environment variables:

   $ export KUADRANT_GATEWAY_NS=api-gateway \
     export KUADRANT_GATEWAY_NAME=ingress-gateway \
     export KUADRANT_DEVELOPER_NS=toystore \
     export KUADRANT_AWS_ACCESS_KEY_ID=xxxx \
     export KUADRANT_AWS_SECRET_ACCESS_KEY=xxxx \
     export KUADRANT_AWS_DNS_PUBLIC_ZONE_ID=xxxx \
     export KUADRANT_ZONE_ROOT_DOMAIN=example.com
     export KUADRANT_CLUSTER_ISSUER_NAME=self-signed

   Copy code to clipboard

   These environment variables are described as follows:

   • KUADRANT_GATEWAY_NS: Namespace for your gateway in OpenShift Container Platform.
   • KUADRANT_GATEWAY_NAME: Name of your ingress gateway in OpenShift Container Platform.
   • KUADRANT_DEVELOPER_NS: Namespace for the example Toystore app in OpenShift Container Platform.
   • KUADRANT_AWS_ACCESS_KEY_ID: AWS key ID with access to manage your DNS zone.
   • KUADRANT_AWS_SECRET_ACCESS_KEY: AWS secret access key with permissions to manage your DNS zone.
   • KUADRANT_AWS_DNS_PUBLIC_ZONE_ID: AWS Route 53 zone ID for the Gateway. This is the ID of the hosted zone that is displayed in the AWS Route 53 console.
   • KUADRANT_ZONE_ROOT_DOMAIN: Root domain in AWS Route 53 associated with your DNS zone ID.

   • KUADRANT_CLUSTER_ISSUER_NAME: Name of the certificate authority or issuer TLS certificates.

     Note

     If you know your environment variable values, you can set up the required YAML files to suit your environment.

1. Create the namespace for the Toystore app as follows:

   $ oc create ns ${KUADRANT_DEVELOPER_NS}

   Copy code to clipboard

1. Deploy the Toystore app to the developer namespace:

   $ oc apply -f https://raw.githubusercontent.com/Kuadrant/Kuadrant-operator/main/examples/toystore/toystore.yaml -n ${KUADRANT_DEVELOPER_NS}

   Copy code to clipboard

Procedure

1. Create the namespace that the Gateway will be deployed in as follows:

   $ oc create ns ${KUADRANT_GATEWAY_NS}

   Copy code to clipboard

2. Create the secret credentials in the same namespace as the Gateway as follows:

   $ oc -n ${KUADRANT_GATEWAY_NS} create secret generic aws-credentials \
     --type=kuadrant.io/aws \
     --from-literal=AWS_ACCESS_KEY_ID=$KUADRANT_AWS_ACCESS_KEY_ID \
     --from-literal=AWS_SECRET_ACCESS_KEY=$KUADRANT_AWS_SECRET_ACCESS_KEY

   Copy code to clipboard

3. Before adding a TLS certificate issuer, create the secret credentials in the cert-manager namespace as follows:

   $ oc -n cert-manager create secret generic aws-credentials \
     --type=kuadrant.io/aws \
     --from-literal=AWS_ACCESS_KEY_ID=$KUADRANT_AWS_ACCESS_KEY_ID \
     --from-literal=AWS_SECRET_ACCESS_KEY=$KUADRANT_AWS_SECRET_ACCESS_KEY

   Copy code to clipboard

Procedure

1. Enter the following command to define a TLS certificate issuer:

   $ oc apply -f - <<EOF
   apiVersion: cert-manager.io/v1
   kind: ClusterIssuer
   metadata:
     name: ${KUADRANT_CLUSTER_ISSUER_NAME}
   spec:
     selfSigned: {}
   EOF

   Copy code to clipboard

2. Wait for the ClusterIssuer to become ready as follows:

   $ oc wait clusterissuer/${KUADRANT_CLUSTER_ISSUER_NAME} --for=condition=ready=true

   Copy code to clipboard

Verification

1. Check the status of your gateway by running the following command:

   $ oc get gateway ${KUADRANT_GATEWAY_NAME} -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Programmed")].message}'

   Copy code to clipboard

   The statuses Accepted and Programmed mean that your gateway is valid and assigned an external address.

2. Check the status of your HTTPS listener by running the following command:

   $ oc get gateway ${KUADRANT_GATEWAY_NAME} -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.listeners[0].conditions[?(@.type=="Programmed")].message}'

   Copy code to clipboard

Procedure

1. Set the TLSPolicy for your Gateway as follows:

   $ oc apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: TLSPolicy
   metadata:
     name: ${KUADRANT_GATEWAY_NAME}-tls
     namespace: ${KUADRANT_GATEWAY_NS}
   spec:
     targetRef:
       name: ${KUADRANT_GATEWAY_NAME}
       group: gateway.networking.k8s.io
       kind: Gateway
     issuerRef:
       group: cert-manager.io
       kind: ClusterIssuer
       name: ${KUADRANT_CLUSTER_ISSUER_NAME}
   EOF

   Copy code to clipboard

2. Check that your TLS policy has an Accepted and Enforced status as follows:

   $ oc get tlspolicy ${KUADRANT_GATEWAY_NAME}-tls -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy code to clipboard

   This may take a few minutes depending on the TLS provider, for example, Let’s Encrypt.

Procedure

1. Set up the primary cluster. Set the following environment variables for your primary cluster context:

   $ export CTX_PRIMARY=<primary_cluster_context_name> # e.g., kind-primary \
     export KUBECONFIG=~/.kube/config # Adjust path if necessary \
     export PRIMARY_CLUSTER_NAME=<primary_cluster_name> # e.g., primary \
     export ONPREM_DOMAIN=<your_onprem_domain> # e.g., example.local \
     export KUADRANT_SUBDOMAIN=kuadrant # Subdomain to delegate

   Copy code to clipboard

2. Install CoreDNS using the Connectivity Link kustomization, which includes the required kuadrant plugin. Apply the following configuration to the primary cluster, replacing <kuadrant_coredns_kustomize_url> with the actual URL for the Kuadrant CoreDNS kustomization.

   $ kustomize build --enable-helm github.com/kuadrant/dns-operator/config/coredns?ref=v0.15.0 | kubectl apply --context ${CTX_PRIMARY} -f -

   Copy code to clipboard
   Note

   The default CoreDNS Helm chart does not include the kuadrant plugin. You must use the Connectivity Link-provided kustomization which bundles a customized CoreDNS build.

3. Wait for the CoreDNS service to get an external IP address and store it:

   $ export COREDNS_IP_PRIMARY=$(kubectl --context $CTX_PRIMARY -n kuadrant-system get service <coredns-service-name> -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
   echo "CoreDNS Primary IP: ${COREDNS_IP_PRIMARY}"

   Copy code to clipboard

   You need this IP address later to configure delegation on your authoritative on-premises DNS server.

4. Create a ConfigMap to define the authoritative zone for CoreDNS on the primary cluster. This minimal configuration enables the kuadrant plugin and GeoIP features.

   cat | kubectl --context $CTX_PRIMARY apply -f -
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: coredns-kuadrant-config
     namespace: kuadrant-system
   data:
     Corefile: |
       ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}:53 {
           debug
           errors
           health {
               lameduck 5s
           }
           ready
           log
           geoip GeoLite2-City-demo.mmdb {
               edns-subnet
           }
           metadata
           kuadrant
       }

   Copy code to clipboard
   Note

   The geoip plugin in this example uses the GeoLite2-City-demo.mmdb database included for demonstration purposes. For production or accurate GeoIP routing, mount your licensed MaxMind GeoIP database into the CoreDNS pod and update the filename in the Corefile.

5. Update the CoreDNS deployment to use the new configuration:

   $ kubectl --context $CTX_PRIMARY -n kuadrant-system patch deployment <coredns-deployment-name> --patch '{"spec":{"template":{"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"coredns-kuadrant-config","items":[{"key":"Corefile","path":"Corefile"}]}}]}}}}'

   Copy code to clipboard

6. Wait for the deployment rollout to complete:

   $ kubectl --context $CTX_PRIMARY -n kuadrant-system rollout status deployment/<coredns-deployment-name>

   Copy code to clipboard

7. Create the Kubernetes Secret that Connectivity Link uses to interact with CoreDNS. This secret specifies the zones this provider instance is authoritative for.

   $ kubectl create secret generic coredns-credentials \
     --namespace=kuadrant-system \
     --type=kuadrant.io/coredns \
     --from-literal=ZONES="${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}" \
     --context ${CTX_PRIMARY}

   Copy code to clipboard

8. On your authoritative on-premises DNS server, configure delegation for the ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN} subdomain to the external IP addresses of the CoreDNS services running on your primary and secondary clusters ($COREDNS_IP_PRIMARY and $COREDNS_IP_SECONDARY). The specific steps depend on your DNS server software (for example, BIND, Windows DNS Server). You typically need to add NS (Name Server) records pointing the subdomain to the CoreDNS IP addresses. For example:

   ; Delegate kuadrant.example.local to CoreDNS instances
   $ORIGIN ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}.
   @       IN      SOA     ns1.${ONPREM_DOMAIN}. hostmaster.${ONPREM_DOMAIN}. (
                           2023102601 ; serial
                           7200       ; refresh (2 hours)
                           3600       ; retry (1 hour)
                           1209600    ; expire (2 weeks)
                           3600       ; minimum (1 hour)
                           )
           IN      NS      coredns-primary.${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}.
   
   coredns-primary   IN A ${COREDNS_IP_PRIMARY}

   Copy code to clipboard

Procedure

1. Ensure that your Connectivity Link system namespace is set correctly by running the following command:

   $ export KUADRANT_SYSTEM_NS=$(oc get kuadrant -A -o jsonpath="{.items[0].metadata.namespace}")

   Copy code to clipboard

2. Define API keys for bob and alice users as follows:

   $ oc apply -f - <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: bob-key
     namespace: ${KUADRANT_SYSTEM_NS}
     labels:
       authorino.kuadrant.io/managed-by: authorino
       app: toystore
     annotations:
       secret.kuadrant.io/user-id: bob
   stringData:
     api_key: IAMBOB
   type: Opaque
   ---
   apiVersion: v1
   kind: Secret
   metadata:
     name: alice-key
     namespace: ${KUADRANT_SYSTEM_NS}
     labels:
       authorino.kuadrant.io/managed-by: authorino
       app: toystore
     annotations:
       secret.kuadrant.io/user-id: alice
   stringData:
     api_key: IAMALICE
   type: Opaque
   EOF

   Copy code to clipboard

3. Create a new AuthPolicy in a different namespace that overrides the deny-all policy created earlier and accepts the API keys as follows:

   $ oc apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: AuthPolicy
   metadata:
     name: toystore-auth
     namespace: ${KUADRANT_DEVELOPER_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: toystore
     defaults:
      when:
        - predicate: "request.path != '/health'"
      rules:
       authentication:
         "api-key-users":
           apiKey:
             selector:
               matchLabels:
                 app: toystore
           credentials:
             authorizationHeader:
               prefix: APIKEY
       response:
         success:
           filters:
             "identity":
               json:
                 properties:
                   "userid":
                     selector: auth.identity.metadata.annotations.secret\.kuadrant\.io/user-id
   EOF

   Copy code to clipboard

Procedure

1. Create a new RateLimitPolicy in a different namespace to override the default low-limit policy created previously and set rate limits for specific users as follows:

   $ oc apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: RateLimitPolicy
   metadata:
     name: toystore-rlp
     namespace: ${KUADRANT_DEVELOPER_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: toystore
     limits:
       "general-user":
         rates:
   
         - limit: 5
           window: 10s
         counters:
         - expression: auth.identity.userid
         when:
         - predicate: "auth.identity.userid != 'bob'"
       "bob-limit":
         rates:
         - limit: 2
           window: 10s
         when:
         - predicate: "auth.identity.userid == 'bob'"
   EOF

   Copy code to clipboard

   It might take a few minutes for the RateLimitPolicy to be applied, depending on your cluster.

2. Check that the RateLimitPolicy has a status of Accepted and Enforced as follows:

   $ oc get ratelimitpolicy -n ${KUADRANT_DEVELOPER_NS} toystore-rlp -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy code to clipboard

3. Check that the status of the HTTPRoute is now affected by the RateLimitPolicy in the same namespace:

   $ oc get httproute toystore -n ${KUADRANT_DEVELOPER_NS} -o=jsonpath='{.status.parents[0].conditions[?(@.type=="kuadrant.io/RateLimitPolicyAffected")].message}'

   Copy code to clipboard

Verification

1. Send requests as user alice as follows:

   $ while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMALICE' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; done

   Copy code to clipboard

   You should see HTTP status 200 every second for 5 seconds, followed by HTTP status 429 every second for 5 seconds.

2. Send requests as user bob as follows:

   $ while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMBOB' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; done

   Copy code to clipboard

   You should see HTTP status 200 every second for 2 seconds, followed by HTTP status 429 every second for 8 seconds.