Testing

Testing

This page covers testing your app's integration with the Convert Android SDK — making experiment decisions deterministic in tests, capturing SDK logs, and keeping tracking quiet. For testing the SDK itself, see its repository.

Make decisions deterministic with direct-data mode

The biggest source of test flakiness is the asynchronous config fetch. Avoid it entirely by building the SDK in direct-data mode with a known config fixture, so decisions are available synchronously and never depend on the network:

import com.convert.sdk.android.ConvertSDK
import com.convert.sdk.core.model.generated.ConfigResponseData

val config: ConfigResponseData = loadTestConfig() // deserialize a fixture JSON
val sdk = ConvertSDK.builder(context)
    .data(config)
    .build()

sdk.onReady {
    val ctx = sdk.createContext("test-visitor")
    val variation = ctx.runExperience("homepage-redesign")
    // assert on variation?.key
}

With data(...) set, the SDK seeds its config without an HTTP call. Use a stable explicit visitor id (e.g. "test-visitor") so bucketing is reproducible run to run — the same id always maps to the same variation for a given experience.

Keep tracking quiet in tests

Disable outbound tracking so tests do not emit network events:

val sdk = ConvertSDK.builder(context)
    .data(config)
    .trackingEnabled(false)
    .build()

Bucketing, rule evaluation, and sticky persistence still work with tracking disabled — only the network side is silenced. See Tracking Control. For a single call, use the per-call override: ctx.runExperience("key", enableTracking = false).

Run on the JVM with Robolectric

The SDK uses Android APIs (SharedPreferences, ConnectivityManager, WorkManager), so integration tests that build a real ConvertSDK run under Robolectric rather than as plain JVM unit tests.

@RunWith(RobolectricTestRunner::class)
class MyConvertIntegrationTest {
    @Test
    fun bucketsVisitorDeterministically() {
        val sdk = ConvertSDK.builder(ApplicationProvider.getApplicationContext())
            .data(loadTestConfig())
            .trackingEnabled(false)
            .build()
        // ...assert on decisions
    }
}

Isolate test classes from each other

Robolectric's singletons — ShadowLog, the WorkManager test registry — are JVM-static, so two test classes running in the same JVM can see each other's state. The SDK's own suite forks a fresh JVM per test class (forkEvery = 1 on the Gradle Test tasks) precisely to avoid that. If your tests build a real ConvertSDK and you see failures that only appear when the whole suite runs, do the same:

// app/build.gradle.kts
tasks.withType<Test>().configureEach {
    forkEvery = 1
}

Raise the value (5–10) if JVM startup starts to dominate your run time.

Force a specific variation with preview

When you need a test to exercise one specific variation — including a draft one, or one the visitor would not normally be bucketed into — setPreview is more direct than reverse-engineering a visitor id that hashes into the arm you want:

@Test
fun rendersTreatment() = runTest {
    val ctx = sdk.createContext("test-visitor")
    ctx.setPreview("12345", "67890") // numeric experienceId, variationId
    val variation = ctx.runExperience("homepage-redesign")
    // variation.key is the previewed arm, whatever bucketing would have chosen
}

Two properties make this test-friendly: it is exact rather than probabilistic, and it is zero-trace — the context writes no tracking events, no sticky decision, and no goal-tracked marks, so a preview test cannot leak state into the next one. Remember setPreview is a suspend function (hence runTest), and that it is one-way for that context's lifetime: build a fresh context per previewed variation rather than trying to un-set it. See Code Examples.

Capturing logs

The SDK logs through android.util.Log under the tag ConvertSDK. In Robolectric tests you can assert on emitted log lines via org.robolectric.shadows.ShadowLog.getLogs(). DEBUG is already the default level, so those lines are there without any configuration — set it explicitly when your production build pins a quieter level and your tests share that setup:

ConvertSDK.builder(context)
    .data(config)
    .logLevel(LogLevel.DEBUG)
    .build()

Tips

• One SDK per test — build a fresh ConvertSDK per test (or per class) so visitor and queue state do not leak between tests.
• Use explicit visitor ids — never rely on the auto-UUID in tests; an explicit id makes bucketing assertions stable.
• Prefer fixtures over live keys — direct-data mode with a checked-in config fixture keeps tests hermetic and fast.

Related pages

• Initialization — direct-data mode and onReady
• Tracking Control — disabling tracking
• Troubleshooting — diagnosing null decisions