Question: flow control across multiple files

[3cp]

Migrating my projects from ava/tape to zora. Love the speed and size!

Within one test file, zora offered grouped tests with await syntax to enable serial mode.

However when I use pta to run multiple files, there is currently no way to enable serial mode across files. I need to use serial mode for all tests because of the usage of mock-fs.

For reference, ava has a global config "concurrency": 1.

I could not find any workaround for this. Should we add some option to pta?

[3cp]

It seems easy to add an option to pta to turn the Promise.all to a chained await.

  await Promise.all(
    files.map((file) => {
      const filePath = resolve(process.cwd(), file);
      return import(pathToFileURL(filePath));
    })
  );

  for (const file of files) {
    const filePath = resolve(process.cwd(), file);
    await import(pathToFileURL(filePath));
  }

Update: I realised this could not work because all tests in every single file are asynchronously executed.

[3cp]

The other option is to add some env support in zora itself, something like ZORA_SERIAL=true.

That's probably a better option than changing pta. Because the env can make sure all test() calls are in serial. The pta change only ensures serialisation between files.

[lorenzofox3]

HI,

The other option is to add some env support in zora itself, something like ZORA_SERIAL=true.

That's probably a better option than changing pta. Because the env can make sure all test() calls are in serial. The pta change only ensures serialisation between files.

That would be quite challenging with the current design and probably not something that I want. And by the way, if you run all the tests serially it won't make things faster than AvA/tape.

I am not sure to see why you'd need to run the tests serially though? are you restoring the fs at some point ? could you share some code please ?

If you need to setup the mock before you run the test you can have a setup file:

const mock = require('mock-fs');

mock({
  'path/to/fake/dir': {
    'some-file.txt': 'file content here',
    'empty-dir': {/** empty directory */}
  },
  'path/to/some.png': Buffer.from([8, 6, 7, 5, 3, 0, 9]),
  'some/other/path': {/** another empty directory */}
});

and then run node with a require hook: node -r test/setup.js node_modules/.bin/pta.

If you really want to run multiple files serially in isolated environments you can spawn multiple processes with either a basic test runner or a simple bash command (with some caveats on the reporting I assume)

for f in test/*.spec.js; do pta $f; done

Or write your test files in some way they accept the assert object and control yourself how the files get loaded within an entry point:

// some.spec.js

export default (t) => {
    t.ok(true)
}

and entrypoint (or you can build a basic test-runner at this point: pta is only few lines of code if you need inspiration)

// test/index.js
import {test} from 'zora'

// this is a dummy test whose only purpose is to load the test suites and make them run serially
test(`my serial test`, async t => {
    for (const f of files) { // files coming from shell args, config, env var (up to you) 
        const { deafult: suite } = await import(f); // or require depending on your project
        await suite(t); // the await make sure the test suites run serially
    }
})

Hope this gives you some idea to proceed. Again, the easier would be to share with me a reproduction of your current test setup

[3cp]

Every test (test()) has it's own mocked set of files, uses mockfs.restore() to clean up at end of every test. It's not a global mock for all tests.

Since mock-fs stubs fs calls, it would not work well when two different mock in two tests (from two test.js files in my case) running in parallel by zora.

Within one test file, I can use await to queue the tests, but currently no solution across files.

[3cp]

I found out I can use top level await (no commonjs), plus the suggested change in pta to queue up all tests.

pta is using import() which works well with both commonjs and esm test files.

[3cp]

I think the suggested change in pta could simply replace the current Promise.all without introducing an option.

for (const file of files) {
    const filePath = resolve(process.cwd(), file);
    await import(pathToFileURL(filePath));
}

It would not cause noticeable performance lost for test files not using any top level await. I tested with another project (no use of mock-fs), there is no noticeable time change (all around 670ms to 680ms for full unit tests in 34 files).

With the change, the disk IO is definitely queued. But I doubt Nodejs parses all read-in files from Promise.all with more than one process/thread.

With this change, global queued tests across files are possible.

[lorenzofox3]

oh you mean if you write your test file as so:

// test.js
import {test} from 'zora';

await test('foo', fn); // with top level await

and we change pta with the suggested changes you gain the ability to run your tests in serie ?

Sounds good 👍

[Alexander-Taran]

❤️

[3cp]

Latest pta works really well! Has to use --module-loader es to load my tests. That also means I cannot mix cjs and es test files in pta, that's understandable.

Wrong module-loader only messes up the reporter, because of different zora instances (one loaded by pta to hold and report, another one in real tests).