Cypress UI testing & verify template tests #153

jmulcahey-twilio · 2020-11-20T00:08:23Z

Description

Adds the Cypress automated browser testing framework, an npm start command that starts a development server to host the templates with Cypress tests, and includes UI tests for the verify function template.

Checklist

I ran npm test locally and it passed without errors.
I acknowledge that all my contributions will be made under the project's license.

Related issues

myfancypants · 2020-11-20T00:43:10Z

cypress/fixtures/users.json

@@ -0,0 +1,232 @@
+[


Are these fixture files being used in the tests? They seem like generic files part of some auto configuration process.

These are example fixtures I left in as a placeholder for when we have mocks for integration tests, but it is a good idea to remove them. I do want to keep the directory around if possible, since it does have special meaning to Cypress (the data for API mocks go in .json files in the fixtures directory). Should I keep a .gitignore in this directory to keep it in git, or perhaps a README.md or Fixtures.md explaining how Cypress uses this directory?

I think having a README.md or a .gitkeep file would make sense. I'd definitely get rid of the bigger fixtures

I removed the example files, and wrote up a quick README.md that links to the Cypress documentation on how the files that were in the fixtures directory should be used.

myfancypants · 2020-11-20T00:45:11Z

cypress/integration/verify/ui.spec.js

+
+// `verify` function template UI-only tests
+
+const testNumber = "5555555555";


Its a good habit to get used to formatting phone numbers with e.164 formatting. Here's probably more info than you need on the subject https://support.twilio.com/hc/en-us/articles/223183008-Formatting-International-Phone-Numbers

TLDR: Make sure to add the country code to the number "+15555555555"

I'd go one further and use specifically a valid phone number (since the one you listed is not valid). The typical one we have in examples is: +18448144627

I'm using it in this test for example: https://github.com/twilio-labs/configure-env/blob/main/src/__tests__/validators.test.ts#L22

I switched it to Dom's suggestion of +18448144627! I did notice that the interface has a dropdown for the country code with +1 filled in, and the testNumber string only controls what's typed into the textbox next to the country code dropdown, so it's possible we may have to revisit this slightly if our future integration tests receive +1+18448144627 from the UI.

myfancypants · 2020-11-20T00:47:45Z

docs/CONTRIBUTING.md

+If you want to run Cypress tests for all of the function templates in your terminal, use:
+
+```bash
+npm cypress


is there a command for running individual tests?

Excellent question! I took a look through the cypress run docs, and it looks like the --spec flag allows the user to specify a test to run. Additionally, --record effectively lets you remote control the Cypress UI from the command line, which could come in handy. I'm currently writing up a couple of lines for CONTRIBUTING.md describing these flags, with a link to the documentation for the rest of the flags cypress run accepts.

myfancypants · 2020-11-20T01:10:26Z

cypress/integration/verify/ui.spec.js

+
+        // NOTE: This response will change when Cypress integration tests are incorporated
+        cy.contains('Error');
+    });


Something I think i failed to go over with you, there are  or  comments in most of the index.html files. They are triggers that we use on the backend to inject the links to edit the deployed code directly in the console.

It would be good to make sure that is captured in our tests

I wrote up a quick test for the verify template that uses a DOM TreeWalker to ensure index.html only contains an EDIT_CODE comment with no APP_INFO. Right now it accepts more than one EDIT_CODE; is there any case in which there will be more than one EDIT_CODE tag?

myfancypants · 2020-11-20T01:18:32Z

src/index.js

+const templates = ['verify'];
+
+templates.forEach((template) => {
+    app.use(`/${template}`, express.static(path.resolve(__dirname, '..', template, 'assets')));


I wonder if it would be worth building a mini document tree walker to automatically just set each directories asset folder as available to remover the need to manually update the templates list as more tests are written. Will probably need some light error handling to ignore possible situations where an assets directory isn't found

I definitely want the development server to automatically pull in templates and make them available for tests, very likely via the method you've outlined here. I'll take a swing at it in this PR, since it will pay off very soon for future tests.

dkundel · 2020-11-20T17:11:28Z

.gitignore

@@ -4,3 +4,5 @@ coverage/
 package-lock.json
 .env
 .DS_Store
+cypress/screenshots/
+cypress/videos/


Since we don't actually track those videos, I wonder if by default we should just turn those off?

That sounds like a good idea -- I'll add "video": false to cypress.json. I would still like to keep cypress/videos/ in .gitignore just in case a contributor enables recording video by hand, so that video files don't end up getting accidentally committed.

dkundel · 2020-11-20T17:15:49Z

cypress/fixtures/users.json

@@ -0,0 +1,232 @@
+[


I think having a README.md or a .gitkeep file would make sense. I'd definitely get rid of the bigger fixtures

dkundel · 2020-11-20T18:20:30Z

cypress/integration/verify/ui.spec.js

+
+// `verify` function template UI-only tests
+
+const testNumber = "5555555555";


I'd go one further and use specifically a valid phone number (since the one you listed is not valid). The typical one we have in examples is: +18448144627

I'm using it in this test for example: https://github.com/twilio-labs/configure-env/blob/main/src/__tests__/validators.test.ts#L22

The development server now automatically crawls function template directories and automatically creates routes for all of the asset directories it finds.

andreasonny83 · 2021-12-16T09:26:54Z

It looks promising. Does this mean all the Cypress tests will live inside the root level Cypress folder rather than inside each individual project?

andreasonny83 · 2022-01-06T09:53:09Z

@jmulcahey-twilio, @dkundel is there any progress on this PR?

jmulcahey-twilio added 4 commits November 19, 2020 13:45

Add support for Cypress UI testing

b07c3cf

Update CONTRIBUTING.md with Cypress test commands

95e0cd2

UI-only Cypress tests for the verify template

b338bf8

Exclude the /cypress directory from Jest testing

2815c70

myfancypants reviewed Nov 20, 2020

View reviewed changes

dkundel suggested changes Nov 20, 2020

View reviewed changes

dkundel changed the base branch from main to ui-tests November 20, 2020 18:22

jmulcahey-twilio added 3 commits November 23, 2020 13:32

Various Cypress test improvements

3bb8583

Test the verify template for an EDIT_CODE tag

e114946

Automatic asset hosting for development server

6dd6456

The development server now automatically crawls function template directories and automatically creates routes for all of the asset directories it finds.

jmulcahey-twilio requested review from myfancypants and dkundel November 24, 2020 21:42

myfancypants approved these changes Nov 30, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cypress UI testing & verify template tests #153

Cypress UI testing & verify template tests #153

jmulcahey-twilio commented Nov 20, 2020

myfancypants Nov 20, 2020

jmulcahey-twilio Nov 20, 2020

dkundel Nov 20, 2020

jmulcahey-twilio Nov 21, 2020

myfancypants Nov 20, 2020

dkundel Nov 20, 2020

jmulcahey-twilio Nov 21, 2020

myfancypants Nov 20, 2020

jmulcahey-twilio Nov 21, 2020

myfancypants Nov 20, 2020

jmulcahey-twilio Nov 24, 2020

myfancypants Nov 20, 2020

jmulcahey-twilio Nov 20, 2020

dkundel Nov 20, 2020

jmulcahey-twilio Nov 21, 2020

dkundel Nov 20, 2020

dkundel Nov 20, 2020

andreasonny83 commented Dec 16, 2021

andreasonny83 commented Jan 6, 2022 •

edited

Loading


		// `verify` function template UI-only tests

		const testNumber = "5555555555";

Cypress UI testing & verify template tests #153

Are you sure you want to change the base?

Cypress UI testing & verify template tests #153

Conversation

jmulcahey-twilio commented Nov 20, 2020

Description

Checklist

Related issues

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andreasonny83 commented Dec 16, 2021

andreasonny83 commented Jan 6, 2022 • edited Loading

andreasonny83 commented Jan 6, 2022 •

edited

Loading