Portal and argo-cd login both (intermittently) lead to blank pages.

[davedavemckay]

1. On clicking the "Portal" button on the RSP homepage, the user is sometimes taken to a blank page.

• The user must navigate back, then forward again to see the Portal.

2. On login to argo-cd, the user is sometimes taken to a blank page.

• The user must close their browser tab, then open a new one and navigate to argo-cd for it to load correctly.

It has been suggested this is a LoadBalancer issue, which may correlate with it being intermittent.

[stvoutsin]

To see if the load-balancer tempfix works you can:

• Navigate to Network / Load Balancers in the Openstack Dashboard
• Find and click on the lb that is used for the cluster we want to fix (e.g. ..something-prod..)
• For each of the pools in the lb, delete the members that aren't running the nginx controller pod. Normally only 1 ingress controller pod is run, unless you run it as a daemonset or have multiple replicas, so you'd only leave the master node & the 1 node running the ingress pod.

I think we had also experimented with Amphora and a health-checker, which handled the above, which I recall working though I'm not sure about it.
(see: https://github.com/lsst-uk/rsp-uk-docs/wiki/RSP-Deployment-instructions-on-Openstack-with-Magnum#setup-amphora-loadbalancer-healthchecker)

In any case if this is the cause the load balancer configuration probably needs to be revisited.

[davedavemckay]

Thanks @stvoutsin
I only see two pools:

pool_0_kube_service_rsp-prod-mint-7j4bka3i6afe_ingress-nginx_ingress-nginx-controller
pool_1_kube_service_rsp-prod-mint-7j4bka3i6afe_ingress-nginx_ingress-nginx-controller

@gpfrancis has mentioned issues with Amphora. We're currently using OVN.

[stvoutsin]

Actually I think we recently found a better workaround which was to deploy the ingress nginx as a Daemonset, which Amanda deployed on test, though I don't know if that fixed made it to main/prod:

See: lsst-sqre/phalanx@main...lsst-uk:phalanx:rsptest

Assuming this is deployed I would expect the LB issue to be resolved hopefully

[stvoutsin]

I tried logging into argo-cd and get an incorrect redirect to:
https://rsp-test.lsst.ac.uk/argo-cd/argo-cd/applications
instead of:
https://rsp-test.lsst.ac.uk/argo-cd/applications
So wherever the redirect is defined is broken somehow. Could you try syncing argo-cd to the latest version and trying again?