Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make tests easier to run and filter with build tags #215

Closed
wants to merge 6 commits into from
Closed

Make tests easier to run and filter with build tags #215

wants to merge 6 commits into from

Conversation

ijt
Copy link
Contributor

@ijt ijt commented Jul 21, 2018

Fixes #214

The tests can now be run as follows:

# Run fast tests with no dependencies.
vgo ./...  		

# Run fast tests and replay tests.
vgo -tags=replay ./...  

# Run fast tests and recording tests.
vgo -tags=record ./...  

# Run fast tests and slow, local tests (wire).
vgo -tags=over1s ./...

@ijt ijt changed the title Make tests easier to run and filter with build tags (fixes #214) Make tests easier to run and filter with build tags Jul 21, 2018
Copy link
Contributor

@zombiezen zombiezen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the issue; I do not agree with this solution. Build tags are great if the bottleneck is compilation time or dependency size, but that's not the issue. The issue is test execution time. The cost to build tags is complexity of build: IDEs and other tooling may not pick up compile errors because the tests will not be even built under all circumstances. The cost here does not outweigh benefit. Command-line flags and environment variables are much lower cost ways to accomplish the same thing.

There are two different test categories in Go Cloud that are slow: hitting cloud backends and the Wire functional tests. The cloud backend/RPC replay tests are easy: they should be run in replay mode by default and then an explicit flag is required to run in record mode. It's less clear to me what to do for the Wire tests: if you turn them off by default, then Wire is not tested by default. I also don't see a good way to trim down the number of tests run. Perhaps we can skip the determinism tests by default, but I'd rather that we spend time there in making Wire's performance better.

@ijt
Copy link
Contributor Author

ijt commented Jul 23, 2018

All right, how about an environment variable named something like $CLOUD_RECORD that turns on recording?

We could also have a $NO_WIRE_TESTS environment variable that would exclude those tests if set.

@zombiezen
Copy link
Contributor

Let's carry on discussion on the issue, I've already left a comment.

@zombiezen zombiezen closed this Jul 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants