Add additional swipe mode - actions on swipe (i.e. Fleksy mode)

[ndom91]

• Closes Add More Swiping Gestures to Keyboard #1065
• Modify's swipe settings to be an enum with 2 options (mutually exclusive) + Disabled
  • Pre-existing "Swipe Typing" mode
  • New "Swipe Actions" mode
• The new "Swipe Actions" mode basically copies the Fleksy behaviour which means:
  • Swipe Up: Undo an auto-suggestion
  • Swipe Down: Move auto-suggestion forward 1
  • Swipe Back: Backspace character / word
  • Swipe Forward: Apply suggestion / space / punctuation
• Works best with / emulates Flesky feeling best with:
  • Swipe mode: "Swipe Actions (alpha)" (required to enable this at all)
  • "Automatic Spaces Mode" disabled
  • "Long press keys" -> delay reduced from 300ms -> 100ms and disable all of those long press key options if you're not using them.
    Not strictly necessary, but significantly helped me feel like I was getting closer to the typing speed I had with Fleksy
  • "Undo autocorrect on backspace" disabled
  • "Double space full stop" enabled
• Refactored the automatic spaces mode dropdown a tiny bit to use a native dropdown instead of the "inline" dropdown that would cause large layout shifts with a slightly awkward animation when opening/closing - hope thats alright!
• Added support for subtitles to the SettingsRadio component for this new settings section.
• I've signed the CLA
• This PR was created with the help of AI

Question:

1. What's the best way update the translations for the new strings? (I'm happy to update these for my new strings, just don't know where to make the change - via https://i18n-keyboard.futo.org/?)
   • I see there are auto-generated translations in the submodule, should I put those changes in this PR?

[magvag]

Really excited to see this merged.

A UI suggestion: it's best to use radio buttons for mutually exclusive options. Maybe something like

Swipe input

• Swipe typing
• Swipe actions
• Disabled

[ndom91]

Really excited to see this merged.

A UI suggestion: it's best to use radio buttons for mutually exclusive options. Maybe something like

Swipe input

• Swipe typing
• Swipe actions
• Disabled

Yeah good shout - let me update this

[ndom91]

@magvag okay I think I've got the emoji and up/down different word bug fixed here 🤞

https://drive.google.com/file/d/1GJ-KsSTSW6FhoO-WO_om8tR3sInW0Zgq/view?usp=sharing

Appreciate all the testing!!

[magvag]

@ndom91 daily driving it for a few hours and it feels great! Totally understand your desire to not overload this PR with features. The swiping tolerance is just a matter of muscle memory for me, no big deal to relearn.

Have found a rather rare bug: if you move a cursor in between space and the first letter of the word, swipe left will remove this word and not the word before it. E. g. "real |deal" to "real |"

Otherwise it's only the suggestion bugs I mentioned in PR 1923. Thank you so much for your contribution, typing with this feature is a blast!

[ndom91]

Updated this branch with all the latest upstream changes and improved the reliability of the swipe up / swipe down history. It now also works across initial auto-corrects that split a word into two, for example.

https://drive.google.com/file/d/1GJ-KsSTSW6FhoO-WO_om8tR3sInW0Zgq/view?usp=sharing

@ maintainers - I would still love to get this in 😊

[abb128]

@ndom91 Thank you for the pull request, I would like to add this functionality but during a patent search recently for something unrelated I happened to stumble upon a patent for this exact thing at https://patents.google.com/patent/US20080316183A1/en :(

So I'm not certain we could ship such a feature or if we would get sued over it. Apple does litigate over software patent violations from what I've seen, so it might be too risky.

[magvag]

@abb128 seems like it should not be a problem. Fleksy was in App Store till 2024(?) and now another keyboard with gestures, Finale, is taking its place: https://apps.apple.com/us/app/finale-keyboard/id1613395700

If anything, Apple would ban them for patent violation first.

[ndom91]

That patent would also seem to cover any swipe-based input (like any SwiftKey-like input methods), such as the preexisting swipe type mode in android-keyboard, no?

[magvag]

@ndom91 I forgot to report that the latest build indeed feels extremely solid when it comes to up/down swipe history. Thank you!

P.S. Last time I checked this patent was never mentioned in any Apple case ever.

[abb128]

That patent would also seem to cover any swipe-based input (like any SwiftKey-like input methods), such as the preexisting swipe type mode in android-keyboard, no?

No, the claim is specifically for detecting the direction and performing an action based on the direction without regard to the initial touchdown point. My reading is that this covers swipe actions but not swipe typing, because swipe typing is much more involved than detecting the direction and absolutely does depend on the initial touchdown point.

detecting a swipe gesture across the virtual keyboard;
determining a direction of the swipe gesture; and
performing a predetermined function determined by the direction of the swipe gesture without regard to an initial touchdown point of the swipe gesture.

Plus, swipe typing was covered in a different patent that predates this one, which has since expired, so based on that alone it shouldn't hold any valid claims on swipe typing anyway. https://patents.google.com/patent/US7098896B2

[ndom91]

Hmm okay yeah I think you're right, under "Claims":

1. A method of interpreting swipe gesture input across a virtual keyboard area of a touch-sensitive device wherein taps of a touch object in the virtual keyboard area generate text input, the method comprising:

• detecting a swipe gesture across the virtual keyboard;
• determining a direction of the swipe gesture; and
• performing a predetermined function determined by the direction of the swipe gesture without regard to an initial touchdown point of the swipe gesture.

That being said, this fear of that patent feels like grasping a bit. This isn't nearly the first project who would theoretically be in violation of that, and this is an open source project not selling anything for profit, there's not really any money to come after right

It's obviously your guys' project though, I'm happy to just run my own fork privately going forward. I know there any many other people who have asked for this Fleksy-like functionality in many other issues though, I'd love to see it released for them as well.

Anyway, I'll be available here if you guys decide you're interested in merging it at any point :)

[Quickmarble]

@ndom91
Thank you for your work! It's an essential feature and for me it's the difference between having to run ancient Fleksy builds and being able to use Futo.
The patent in question expires on 2030-03-22. In the worst case of Futo project owners deciding to avoid risk for now, are you ready to keep your fork & PR in sync with releases until then? If you also set up an F-droid repo with your fork, it'd be amazing.

@abb128
Could you contact, say, EFF and get their opinion on what could be done in this situation?

[Felix14-v2]

I want to put in two cents here. I think that instead of choosing between these two swipe modes, it is possible to make them compatible by triggering actions with two fingers when the swipe typing is enabled. I'm not sure how hard is that to implement, but at least in theory this could work.

[ndom91]

I want to put in two cents here. I think that instead of choosing between these two swipe modes, it is possible to make them compatible by triggering actions with two fingers when the swipe typing is enabled. I'm not sure how hard is that to implement, but at least in theory this could work.

Are you picturing 1 toggle that enables a swipe mode generally, where swipe typing is done classically with 1 finger and actions with 2?

While theoretically possible, the whole goal of this swipe actions mode is to be fast. Using two thumbs often doesn't sync up perfectly so it's hard to trigger 1 action with it, so you'd end up having to lift your hand up to use your index and middle finger or similar. Either way, that'd be a major slow down.

I don't think having a swipe mode toggle is an issue anyone has raised

[ndom91]

Updated the branch with the latest master changes and cleaned up the merge conflicts. Tested working on my device for a few hours now.

If anyone wants it: https://drive.google.com/file/d/1TfV-SpO25X4XyZuUKghAOqAAfNQlzB1Y/view?usp=sharing

[magvag]

@ndom91 thanks, it works great!

P.S. Thought I’d come around this behaviour, but never got used to swipe left on "before |word" removing "word", not "before". I believe I-beam is only "inside" a word when it's between the letters. Otherwise everything is great, thanks for your effort on maintaining this branch.

[ndom91]

@ndom91 thanks, it works great!

P.S. Thought I’d come around this behaviour, but never got used to swipe left on "before |word" removing "word", not "before". I believe I-beam is only "inside" a word when it's between the letters. Otherwise everything is great, thanks for your effort on maintaining this branch.

Hmm yeah that does feel wrong 🤔 I'll have a look