Skip to content

Set temp_store_directory on Android #173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Set temp_store_directory on Android #173

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
40cdd10
Set temp_store_directory on Android
simolus3 Apr 22, 2025
7056c35
Try inserting data
simolus3 Apr 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions core-tests-android/src/androidTest/java/com/powersync/AndroidDatabaseTest.kt
View file
Original file line number Diff line number Diff line change
@@ -222,4 +222,13 @@ class AndroidDatabaseTest {
// The exception messages differ slightly between drivers
assertEquals(exception.message!!.contains("write a readonly database"), true)
}

@Test
fun canCreateTemporaryTable() = runTest {
database.execute("PRAGMA temp_store = 1;") // Store temporary data as files
database.execute("CREATE TEMP TABLE my_tbl (content ANY) STRICT;")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth noting that this test passes even without the pragma, but I'm not sure why. My understanding is that SQLite should use temporary files here, but it apparently doesn't.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe SQLite always keeps it in memory first, until it reaches a certain size threshold. I can't find any docs confirming this though.

Copy link
Contributor Author

@simolus3 simolus3 Apr 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried inserting a 1MiB randomblob 128 times, but it still seems to pass without setting the temp directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another thing to check is the Android version it is being tested on - I remember this potentially only affecting newer Android versions. Although I can't find where the test emulator is specified here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just tested this manually with an SDK 35 emulator. An SDK-specific issue would surprise me though, we ship our own SQLite library along with the application. Or are you saying that some Android versions give apps their own /tmp that would be writable?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think I'm remembering anecdotes of temporary storage issues only being present on some Android versions, but I don't think we ever confirmed that.

Either way it doesn't quite explain how these tests worked, unless the emulator uses different filesystem permission structures from actual devices?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless the emulator uses different filesystem permission structures from actual devices?

That sounds unlikely, the emulator is a VM running a real Android system.

I think I'm remembering anecdotes of temporary storage issues only being present on some Android versions

I've asked the user reporting this to share the Android version they're using. Maybe it's not a problem on newer Android versions and that's why we didn't catch this.

for (i in 0..128) {
database.execute("INSERT INTO my_tbl VALUES (randomblob(1024 * 1024))")
}
}
}
19 changes: 18 additions & 1 deletion core/src/androidMain/kotlin/com/powersync/DatabaseDriverFactory.android.kt
View file
Original file line number Diff line number Diff line change
@@ -7,6 +7,7 @@ import com.powersync.db.internal.InternalSchema
import com.powersync.db.migrateDriver
import kotlinx.coroutines.CoroutineScope
import org.sqlite.SQLiteCommitListener
import java.util.concurrent.atomic.AtomicBoolean

@Suppress("EXPECT_ACTUAL_CLASSIFIERS_ARE_IN_BETA_WARNING")
public actual class DatabaseDriverFactory(
@@ -27,10 +28,22 @@ public actual class DatabaseDriverFactory(
context.getDatabasePath(dbFilename)
}

val properties = buildDefaultWalProperties(readOnly = readOnly)
val isFirst = IS_FIRST_CONNECTION.getAndSet(false)
if (isFirst) {
// Make sure the temp_store_directory points towards a temporary directory we actually
// have access to. Due to sandboxing, the default /tmp/ is inaccessible.
// The temp_store_directory pragma is deprecated and not thread-safe, so we only set it
// on the first connection (it sets a global field and will affect every connection
// opened).
val escapedPath = context.cacheDir.absolutePath.replace("\"", "\"\"")
properties.setProperty("temp_store_directory", "\"$escapedPath\"")
}

val driver =
JdbcSqliteDriver(
url = "jdbc:sqlite:$dbPath",
properties = buildDefaultWalProperties(readOnly = readOnly),
properties = properties,
)

migrateDriver(driver, schema)
@@ -59,4 +72,8 @@ public actual class DatabaseDriverFactory(

return mappedDriver
}

private companion object {
val IS_FIRST_CONNECTION = AtomicBoolean(true)
}
}