Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The new timeout feature doesn't work correctly for serial tests (await) #183

Closed
3cp opened this issue Feb 26, 2023 · 1 comment
Closed

Comments

@3cp
Copy link
Contributor

3cp commented Feb 26, 2023

Demo code test.js
There are 2 serial tests: both needs 3 seconds to run, both well below the default 5 seconds timeout limit.

const { test } = require('zora');

function delay(ms) {
  return new Promise(resolve => {
    setTimeout(resolve, ms);
  });
}

test('group', async t => {
  await t.test('one', async t => {
    await delay(3000);
    t.ok(1);
  });
  await t.test('two', async t => {
    await delay(3000);
    t.ok(2);
  });
});

But the test "two" failed on the 5 secs limit check.

> node test.js
TAP version 13
# group
# one
ok 1 - should be truthy
# two
ok 2 - should be truthy
not ok 3 - The test did no complete on time. refer to https://github.com/lorenzofox3/zora/tree/master/zora#test-timeout for more info
  ---
    operator: "timeout"
    actual: "test takes longer than 5000ms to complete"
    expected: "test takes less than 5000ms to complete"
  ...

1..3
# tests 3
# pass  2
# fail  1
# skip  0

The reason is the timeout check started at same time for both tests: "one" and "two", the timeout check should only start for each test at the time it really starts.

The bug only appears if users use await to force serial tests.

@3cp
Copy link
Contributor Author

3cp commented Feb 26, 2023

My bad, the timeout failure is not on test "two", but the parent test.

test('group', async t => {
  await t.test...
  await t.test...
}, { timeout: 10000 });

The outer test timeout is needed.

@3cp 3cp closed this as completed Feb 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant