Cross-Namespace API Routing Fails with 404 Error in Multi-Namespace Configuration

[steveliem]

Description:

I am experiencing an issue where APIs deployed in a namespace other than the one where the WSO2 APK dataplane is running do not respond as expected. The setup involves deploying the dataplane in the apk namespace, and APIs using WSO2 APK CRDs in the tenant namespace. Despite configuring the Helm chart to recognize APIs across both namespaces, API calls to services deployed in the tenant namespace return a 404 error, indicating the resource is not found. However, deploying and calling the API in the apk namespace works as expected. This issue suggests a problem with cross-namespace routing or API recognition in WSO2 APK.

Suggested Labels:

bug, helm, cross-namespace, routing, crd

OS, DB, other environment details and versions:

• Kubernetes version: 1.27.2
• Helm version: 3.7.2
• WSO2 APK version: 1.0.0 (app version 1.16.0)
• Environment details: Docker Desktop setup on MAcOS

Steps to reproduce:

1. Created namespaces apk and tenant.
2. In the Helm chart values under adapter and commonController, added the configuration:

   configs: 
     apiNamespaces: 
       - "apk"
       - "tenant"
   

3. Deployed the dataplane part of APK in the apk namespace and ensured all components were running.
4. Deployed my example API using WSO2 APK CRDs to the tenant namespace with:

   kubectl apply -f example/ -n tenant
   

   The command output was:

   httproute.gateway.networking.k8s.io/custom-route-httpbin created
   api.dp.wso2.com/http-bin-api created
   backend.dp.wso2.com/http-bin-backend created
   

5. Attempted to call the API with:

   curl -k --location 'https://default.gw.wso2.com:9095/http-bin-api/1.0.0/get' --header 'Host: default.gw.wso2.com' --header 'Authorization: bearer [Token]'
   

   The response was:

   {"code":"404","description":"The requested resource is not available.","message":"Not Found"}
   

6. Deploying the API definition in the apk namespace and calling the API returned a successful response.

Console Outputs and Commands:

• Deployment command in tenant namespace:

  kubectl apply -f example/ -n tenant
  

• API call resulting in 404:

  curl -k --location 'https://default.gw.wso2.com:9095/http-bin-api/1.0.0/get' --header 'Host: default.gw.wso2.com' --header 'Authorization: bearer [Token]'
  {"code":"404","description":"The requested resource is not available.","message":"Not Found"}
  

• Successful API call in apk namespace:

  curl -k --location 'https://default.gw.wso2.com:9095/http-bin-api/1.0.0/get' --header 'Host: default.gw.wso2.com' --header 'Authorization: bearer [Token]'
  

  Returned a valid JSON response from httpbin.org.

API-CRDs.zip

[steveliem]

Feature Suggestion:
I would like to suggest an enhancement for future versions of WSO2 APK. Implementing label selectors for namespaces could offer a more dynamic and flexible way to configure dataplane recognition of APIs across namespaces. Instead of using hardcoded namespace names in the dataplane configuration, label selectors would allow the dataplane to automatically recognize and route traffic to APIs in any namespace tagged with a specific label. This approach could simplify configuration management, especially in environments with numerous or dynamically created namespaces, and align with Kubernetes best practices for resource selection and management.

[pubudu538]

Hi @steveliem ,

We have the API deploy support in the cluster level and the namespace level. In order to get this working can you add the namespace value to the HttpRoute as below?

  parentRefs:
  - group: "gateway.networking.k8s.io"
    kind: "Gateway"
    name: "default"
    namespace: "apk"
    sectionName: "httpslistener"

We value your suggestion and we will consider this for future releases.

Thank you!
Pubudu.

[steveliem]

Hi @pubudu538 ,

Thanks for your prompt feedback. Adding the extra **namespace** field is the solution. It works! 👍 Also I've managed to make it work in combination with an own TokenIssuer pointing to Okta, because also a TokenIssuer includes a reference to the Gateway in the apk namespace.

I haven't verified yet if the above was documented in the WSO2 APK Documentation, but I think this is a valuable piece of information.

Thank you for taking my feature suggestion.

Kind regards,
Steve

[pubudu538]

Hi @steveliem,

We haven't documented these sections. Will include this as well. We are mainly using the cluster level and namespace level deployments.

Thank you!
Pubudu.