Prefer combinatorial test

[bcollamore]

MS suggests using combinatorial testing via a package maintained by a MS employee, but not officially part of MSTest.

The package looks interesting.

It probably only makes sense to bother suggesting if there are numerous [DataRows] (perhaps > 3?). However, if the result is that the single DataRow line is longer than say 120 characters, it seemingly wouldn't be worth it.

An example might be this:

		[TestMethod]
		[DataRow("true", true),
		 DataRow("false", true),
		 DataRow("typeof(bool)", false),
		 DataRow("typeof(System.DateTime)", false),
		 DataRow("null", false),
		 DataRow("a", false),
		 DataRow("int.MaxValue", false),
		 DataRow("1d", false)]

or perhaps this:

		[DataRow(true, "Foo.Bar(new object())", true, "new object().Bar()")]
		[DataRow(true, "new object().Bar()", false, "")]
		[DataRow(false, "Foo.Bar(new object())", false, "")]
		[DataRow(true, "Foo.Bar", false, "")]
		[DataRow(false, "Foo.Bar", false, "")]
		[TestMethod]

or perhaps this:

		[TestMethod]
		[DataRow("1.0.1", "1.0.2", true)]
		[DataRow("1.1.0", "1.2.0", true)]
		[DataRow("1.0.0", "2.0.0", true)]
		[DataRow("1.0.0.1", "1.0.0.0", true)]
		[DataRow("1.0.0", "1.0.1-ci.1", true)]
		[DataRow("1.0.0", "1.0.1+417ce", true)]
		[DataRow("1.0.0", "1.0.1-beta+417ce", true)]
		[DataRow("1.0.0.0", "1.0.0", false)]
		[DataRow("1.0.0", "1.0.0.0", false)]
		[DataRow("1.0.0", "1.0.0", false)]
		[DataRow("1.0.0", "1.0.0-prerelease", false)]
		[DataRow("1.0.0", "1.0.0-ci.1", false)]
		[DataRow("1.1.2", "1.1.2+417ce", false)]
		[DataRow("1.1.2", "1.1.2-beta+417ce", false)]
		[DataRow("1.1", "1.1", false)]
		[TestCategory(TestDefinitions.UnitTests)]
		public async Task FileVersionMustBeSameAsPackageVersion(string fileVersion, string packageVersion, bool hasDiagnostic)
		{

or this:

		[DataRow("protected", true)]
		[DataRow("public", false)]
		[DataRow("private", false)]
		[DataRow("internal", false)]
		[TestMethod]

or this:

		[DataRow("_isFoo", true)]
		[DataRow("_areFoo", true)]
		[DataRow("_shouldFoo", true)]
		[DataRow("_hasFoo", true)]
		[DataRow("_doesFoo", true)]
		[DataRow("_wasFoo", true)]
		[DataRow("_is12Foo", true)]
		[DataRow("_foo", false)]
		[DataRow("_isfoo", false)]
		[DataRow("_arefoo", false)]
		[DataRow("_shouldfoo", false)]
		[DataRow("_hasfoo", false)]
		[DataRow("_doesfoo", false)]
		[DataRow("_wasfoo", false)]
		[DataRow("IsFoo", false)]
		[DataRow("AreFoo", false)]
		[DataRow("ShouldFoo", false)]
		[DataRow("HasFoo", false)]
		[DataRow("DoesFoo", false)]
		[DataRow("WasFoo", false)]
		[DataRow("isFoo", false)]
		[DataRow("areFoo", false)]
		[DataRow("shouldFoo", false)]
		[DataRow("hasFoo", false)]
		[DataRow("doesFoo", false)]
		[DataRow("wasFoo", false)]
		[TestMethod]
		[TestCategory(TestDefinitions.UnitTests)]
		public async Task FieldVariableNameIsCorrect(string content, bool isGood)

The best use cases would seemingly be when there are multiple CombinatorialValues for the same TestMethod; however, they may be tricky to implement a CodeFixer for in a way that guarantees we don't change the test cases.

Hmm... although, it appears that this package may not work in conjunction with non-Combinatorial parameters. This could limit the surface area of applicability. I.e., I don't think any of the above examples would actually work; perhaps if they were split into multiple TestMethods? For the following example, perhaps 1 non-Combinatorial test for "true" and 1 CombinatorialTest for the 3 falses?

		[DataRow("protected", true)]
		[DataRow("public", false)]
		[DataRow("private", false)]
		[DataRow("internal", false)]
		[TestMethod]

Determine if reasonable to create an analyzer that detects appropriate use cases for it (by looking at the TestMethods in this project), and if reasonable, create an analyzer and auto-fixer. (It's OK if the use cases the analyzer catches is very narrow.)