feat: Migrate vwa workflows clean

[henschwartz]

migrate VWA test workflows from kubeflow/kubeflow to notebooks-v1

• Add vwa_integration_test.yaml workflow
• Add vwa_multi_arch_test.yaml workflow
• Add vwa_frontend_tests.yaml workflow
  [TASK] Migrate VWA test-related workflows from kubeflow/kubeflow to notebooks-v1 branch #586

[andyatmiami]

@henschwartz - I would encourage you to reach out to @liavweiss and/or @yehudit1987 to get a better sense of expected changes for this work.

I'm not yet going to do a deep review on the code - as they should be able to help out to get this is better shape for an initial review.

In general - the following areas need improved:

• ensure kubeflow/notebooks (not kubeflow/kubeflow listed as part of the container registry
• ensure the branches: config lists the following:
  • main (not master)
  • notebooks-v1
  • 'v*-branch'
• ensure files added to this PR have valid content
  • I'm seeing numerous "empty files" which I am guessing was unintentional

Getting clean GHA check results for these workloads on your fork would likely highlight these shortcomings - so I encourage you to look into that. I am happy to help if you want to talk about how to accomplish that.

[andyatmiami]

Thank you for your continued work on this PR... getting closer to having something in desired shape to get this merged! 💯

In addition to the comments I have raised on specific lines throughout the PR - please rebase your changes off the latest notebooks-v1 branch so the testing/gh-actions files can be "picked up" from the notebooks-v1 branch now that they are available.

[henschwartz]

Hi @andyatmiami, one issue that I saw during the tests was "Error: Cannot access 'CookieJar.SetCookieOptions' because 'CookieJar' is a type, but not a namespace", it caused by a third-party type definition package (@types/request version 2.48.12) that contains incorrect TypeScript definitions . This package is pulled in as a transitive dependency through our project's dependency chain, even though we don't directly use the request library. I tried also to downgrade to 2.48.5 version but the issue was still there... I added the "skipLibCheck" into tsconfig.json file that it will skip type checking in declaration files in node_modules which passes the issue from above... I am still trying to find an elegant way to handle this issue but i would love if you can assist as well...

[andyatmiami]

/ok-to-test

[andyatmiami]

Now, please keep in mind.. I am NOT any type of "Typescript expert" - so I can understand/appreciate why this "skipLibCheck": true was added to avoid the following error:

Cannot access 'CookieJar.SetCookieOptions' because 'CookieJar' is a type, but not a namespace. Did you mean to retrieve the type of the property 'SetCookieOptions' in 'CookieJar' with 'CookieJar["SetCookieOptions"]'?

The update to [email protected] seems to cause this deviant behavior - and I got intrigued to understand why the similar #627 PR did not encounter this issue.

As it turns out - for reasons unclear - at some point in the past - an explicit dependency was added to TWA frontend/package.json for tough-cookie: ~4.1.3... and it seems that bypasses the issue . So that could (maybe) be another alternative here (?)

Other options like upgrading "@kubernetes/client-node" to a later version concern me w.r.t. potentially dealing with myriad of other upgrade issues 😒

$ npm ls @types/request
[email protected] /Users/astonebe/Development/Code/GitHub/kubeflow-notebooks/components/crud-web-apps/volumes/frontend
└─┬ @kubernetes/[email protected]
  └── @types/[email protected]

• latest version of @types/request is only 2.48.13 so unclear if it would make a difference.. but apparently requires TypeScript 5.1+ (we are using TS 4.7)

But the solution implemented here (based on my limited understanding) also seems reasonable to be clear!

@kimwnasptd - thoughts?

[kimwnasptd]

My Plan A would be to actually upgrade @kubernetes/client-node, as this package is only used for importing K8s type definitions:

cd components/crud-web-apps/volumes/frontend/src
grep -ri kubernetes/client-node
app/pages/volume-details-page/yaml/yaml.component.ts:import { V1PersistentVolumeClaim } from '@kubernetes/client-node';
app/pages/volume-details-page/events/events.component.ts:import { V1PersistentVolumeClaim } from '@kubernetes/client-node';
app/pages/volume-details-page/volume-details-page.component.ts:import { V1PersistentVolumeClaim } from '@kubernetes/client-node';
app/pages/volume-details-page/overview/pods-mock.ts:import { V1Pod } from '@kubernetes/client-node';
app/pages/volume-details-page/overview/overview.component.ts:import { V1PersistentVolumeClaim, V1Pod } from '@kubernetes/client-node';
app/pages/volume-details-page/pvc-mock.ts:import { V1PersistentVolumeClaim } from '@kubernetes/client-node';
app/types/backend.ts:import { V1PersistentVolumeClaim, V1Pod } from '@kubernetes/client-node';
app/types/event.ts:import { EventsV1Event } from '@kubernetes/client-node';
app/services/backend.service.ts:import { V1PersistentVolumeClaim, V1Pod } from '@kubernetes/client-node';

That being said, I've tried updating it to 0.17.0, 0.18.0 and 0.19.0 and it always complains for some declaration or another. Will try a bit more combinations and come back

[kimwnasptd]

So in the end I gave up with bumping the @kubernetes/client-node package as I was getting the above complaints.

The interesting part though is that afterwards I tried to add "tough-cookie": "~4.1.3", to VWA, but then I'm getting the same error with bumping the K8s package 🤔

 => [frontend 11/12] COPY --from=frontend-kubeflow-lib /src/dist/kubeflow/ ./node_modules/kubeflow/                                     0.1s
 => ERROR [frontend 12/12] RUN npm run build -- --output-path=./dist/default --configuration=production                                22.4s
------
 > [frontend 12/12] RUN npm run build -- --output-path=./dist/default --configuration=production:
0.325
0.325 > [email protected] build
0.325 > npm run copyLibAssets && ng build --base-href /volumes/ --deploy-url static/ --output-path=./dist/default --configuration=production
0.325
0.463
0.463 > [email protected] copyLibAssets
0.463 > cp -r ./node_modules/kubeflow/assets/* ./src/assets/
0.463
1.247 Option "deployUrl" is deprecated: Use "baseHref" option, "APP_BASE_HREF" DI token or a combination of both instead. For more information, see https://angular.io/guide/deployment#the-deploy-url.
1.504 - Generating browser application bundles (phase: setup)...
20.90 ✔ Browser application bundle generation complete.
21.78 ✔ Browser application bundle generation complete.
21.78
21.78 Warning: '/src/node_modules/kubeflow/styles/styles.scss' imports '~material-icons/iconfont/material-icons.scss' with a tilde. Usage of '~' in imports is deprecated.
21.78
21.78
21.78
21.78 Error: node_modules/@types/ws/index.d.ts:334:18 - error TS2315: Type 'Server' is not generic.
21.78
21.78 334         server?: HTTPServer<V> | HTTPSServer<V> | undefined;
21.78                      ~~~~~~~~~~~~~
21.78
21.78
21.78 Error: node_modules/@types/ws/index.d.ts:334:34 - error TS2315: Type 'Server' is not generic.
21.78
21.78 334         server?: HTTPServer<V> | HTTPSServer<V> | undefined;
21.78                                      ~~~~~~~~~~~~~~
21.78
21.78
21.78
------

[kimwnasptd]

So, I managed to update the version of the @kubernetes/client-node to v0.19.0 and also had to add "@types/ws": "8.5.4", to avoid the above error with the HTTPSServer<V>.

But then I hit again the issue of the CookieJar.SetCookieOptions.

So my proposal would be to:

1. Pin "cypress": "14.1.0",
2. Add "tough-cookie": "4.1.3",
3. Not to mess with the skipLibCheck setting in tsconfig.json

In general I'm not much in favour of pinning packages, but for this one, considering that we need to do a bump in a lot of other dependencies (node, Angular) I'd be OK in pinning to ensure reproducible builds as much as possible.

[andyatmiami]

I think this makes sense @kimwnasptd ... the focus HERE with these PRs is just getting notebooks-v1 branch in a build-able state.

I would expect a FAST FOLLOW once we hit that milestone to then turn attention to getting up compliant and in good standing for CNCF graduation... and addressing outstanding CVEs in notebooks-v1 is unavoidably going to require I'm guessing SIGNIFICANT version upgrades - so your proposal here (which I agree with) then becomes only a transient state .

[andyatmiami]

/lgtm

Thanks for all this work @henschwartz - appreciate your responsiveness and diligence in working through issues in this PR as it was way more of a head-scratcher than originally intended/assumed!

I'm comfortable with the changes and have verified the workflows behave as expected.

Tested on my fork:

• andyatmiami#64

[kimwnasptd]

I second every word from @andyatmiami above, thanks for the amazing work @henschwartz!

/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kimwnasptd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kimwnasptd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[henschwartz]

thank you @andyatmiami @kimwnasptd !