Add multi-device synchronization support #2149
Conversation
|
Translations are handled in the way you've done - add English in all files until someone contributes the translations (although I usually use the standardize.js script to do this automatically (ensures consistency) and automatically add AI translations from DeepL as placeholders. You can do this if you want to and have an API key, otherwise I'll do it later. |
|
What is this core file? |
I don't have one, but I'm happy to get one together if it's a pain for you to do.
Doh, that was a core dump from a failed run of the flutter emulator that must have been left over from the |
Nah it's no big deal |
|
Can't seem to get this to work yet. I tried setting up an SxncD server but couldn't get OIDC auth to work due to Fmstrat/SxncD#2. Then I tried GitHub auth and that worked on the webapp side - I signed in and got an API key for Obtainium. When I try syncing from my phone, I get an error about an "Unexpected character" in " Then I tried using the Android emulator (only 10 apps on it). This time the payload was small enough, but then I got this error: Overall it seems like it needs more testing around edge cases and failure modes. |
|
Also, what happens if I have, for example, version |
Interesting, I've used it with Keycloak and a custom Node server implementation and both worked for me. Happy to help debug config if you'd like.
Odd, let me try adding a bunch of apps to mine and see what happens.
I'm wondering if this has to do with the above error. I realize I haven't included each interaction in a transaction, so it's like the DB just got into a bad state (amateur hour on that one). Will take care of that next.
Oddly I tried a bunch of things to break it, and have a full e2e test running. I think in the above you may have triggered a cascading issue off the doctype item. In any event, I'll find a way to reproduce and make sure that's covered in the test cases. Thanks for helping out.
If this is an issue, it would be an issue during import/export, too, as the SxncD function is just a wrapper for those calls (using whichever device has the newest settings change or app change). As I go through the above I'll take a look by making sure I have different installed apps on each device and see how it treats it, but for clarity if a change is needed in those functions I'll probably do it in a separate PR so it's easier to review independently. |
It is an issue with import/export for apps that have the "reconcile OS version..." toggle disabled - those app entries appear with an incorrect install status and the user has to manually correct it. But import/export is a different use case - usually infrequent so it's mostly fine as is. Syncing would happen all the time so we can't have that. My suggestion is to modify the export JSON (before upload) to remove any data that's supposed to be device-specific (mainly the installedVersion for now, maybe there are others that make sense to also exclude later). Same for import/download - merge the incoming JSON with any existing on-device data. |
|
How much size does this add to the app? If this is added and it makes the app noticeably bigger then an Obtainium flavor should be released for those who don't want integrations with third-party services (SxncD in this case). Or fork Obtainium and maintain a fork since this is a lot of changes the developer needs to maintain so sync users get sync by getting your fork and people who don't want sync can use non-forked Obtainium. |
Basically nothing. As mentioned in the issue being tracked the libraries for HTTP connection are already included in Obtainium's pub list. |
|
We also need to trust the owners of the server. |
It is self-hosted, and e2e encrypted. |
Ah, I follow now, I'll investigate parsing those values on sync.
I will say, I have a few friends using Obtainium (4 including me), and all but one of us use Import/Export for syncing between multiple phones or a tablet. So I think the use-case is happening anyway. If you want, I can look at reconciling that option period, even during import (to accompany the below)?
Good call. Will do a quick audit to see if any other options make sense, too. |
|
Yeah, modifying the original import/export function is a good idea. |
This PR incorporates synchronization functionality with SxncD. SxncD is a self-hostable, FOSS synchronization system built for apps that currently use file import/export with a plugin already written for Obtainium. As you can see from the screenshots below, Apps each get their own API key per user, and each App has a card representing it. The selection of apps and authentication methods can be made when setting up the server.
The best way to think of it is SxncD is for synchronization what https://ntfy.sh/ is for notifications.
Covers issue #2013.
I've been running this on multiple devices for a couple of weeks now and it's running smoothly. I'm labeling this PR for draft as I'm not sure how translations are handled for new content yet.
Overall, this PR: