Skip to content

Add script to update list of failing generated fourslash tests #1361

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 7, 2025

Conversation

gabritto
Copy link
Member

@gabritto gabritto commented Jul 4, 2025

Now you can do npm run updatefailing and the script will delete the current list of failing generated fourslash tests, re-generate the tests again (so that none is skipped), run the tests, and write the new list of failing tests.

@gabritto gabritto marked this pull request as ready for review July 4, 2025 17:24
@Copilot Copilot AI review requested due to automatic review settings July 4, 2025 17:24
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds a convenience script to keep the autogenerated fourslash test failures up to date by regenerating tests, running them, and writing out the new failure list.

  • Defines an updatefailing npm task that clears the old list, regenerates all tests, runs them, and writes the new failures.
  • Introduces updateFailing.mts to orchestrate test regeneration and failure capture.
  • Updates convertFourslash.mts to accept a set of failing tests and apply skips when generating Go tests.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
package.json Adds updatefailing script alongside existing convertfourslash
internal/fourslash/_scripts/updateFailing.mts New script to clear, rerun, and record failing tests
internal/fourslash/_scripts/tsconfig.json Enables importing TypeScript extensions in scripts
internal/fourslash/_scripts/failingTests.txt Removes out-of-date entries to be regenerated
internal/fourslash/_scripts/convertFourslash.mts Refactors to read updated failures and skip tests accordingly
Comments suppressed due to low confidence (2)

internal/fourslash/_scripts/updateFailing.mts:7

  • import.meta.dirname is not defined in ES modules; derive the directory with something like:
const __filename = url.fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const failingTestsPath = path.join(__dirname, "failingTests.txt");
const failingTestsPath = path.join(import.meta.dirname, "failingTests.txt");

internal/fourslash/_scripts/convertFourslash.mts:12

  • import.meta.dirname isn’t supported in ESM; use the imported url module to compute __dirname, for example:
const __filename = url.fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const failingTestsPath = path.join(__dirname, "failingTests.txt");
const failingTestsPath = path.join(import.meta.dirname, "failingTests.txt");

@gabritto
Copy link
Member Author

gabritto commented Jul 4, 2025

@Andarist would this help with the process of fixing generated fourslash tests or do you need something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great 👍 it's a similar flow I've used manually myself

@gabritto gabritto requested a review from jakebailey July 4, 2025 20:27
@gabritto gabritto added this pull request to the merge queue Jul 7, 2025
Merged via the queue into main with commit 2fe47a5 Jul 7, 2025
22 checks passed
@gabritto gabritto deleted the gabritto/failingscript branch July 7, 2025 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants