Skip to content

fix(agent): make chained gestures responsive and action-paced - #1761

Open
oliver-mee wants to merge 7 commits into
AprilNEA:masterfrom
oliver-mee:feat/held-gesture-parity
Open

oliver-mee wants to merge 7 commits into
AprilNEA:masterfrom
oliver-mee:feat/held-gesture-parity

Conversation

@oliver-mee

@oliver-mee oliver-mee commented Oct 13, 2026 •

Copy link
Copy Markdown

Summary

Make held directional gestures respond to deliberate movement immediately and pace repeated actions without delaying reversals or axis changes. This builds on the chained-swipe recognizer merged in #1652.

Rebase status

  • Rebased onto 98d60ca7b34b7e9b3fe7975ae11dff223b634c8e, which was also upstream master at validation (previous base: 5f0435b5c73a1e43ef63707af7867bbc074113eb). Validated head: 50dd3ef386e8d6207b291d8e5d50273526a1eea7.
  • Conflict resolution preserves upstream's HoldModifier, Gestures2 thumb-wheel, forced-reapply, DPI-write announcement, pointer-mode shortcut, Flow handoff and button-lifecycle work alongside this PR's gesture behavior.
  • The rebase leaves the PR's existing treatment of the three reserved default behaviors unchanged: GESTURE_HOLD_FOR_SWIPE, 400 ms / 200 ms same-direction pacing, and unbound swipes consuming the click. Upstream had not independently changed those behaviors since the previous base; no gate-removal work was added.

Changes

  • openlogi-core: retain signed-distance, backtracking and drift handling; resolve per-direction runtime policy centrally; bound pending movement without changing its slope. Keep each direction's enabled state and minimum interval in one DirectionPacing array, use GESTURE_REPEAT_DISTANCE directly, and remove the unused hold timestamp and test seam. Remove the initial hold timer and add deterministic-time processing.
  • openlogi-device: carry policy through HID++ capture and reconnects while preserving stale-report suppression and wheel-hold support. Group capture routes in CaptureRoutes and drop stale initial reports without a zero-motion accumulator call.
  • openlogi-agent-core and openlogi-agent: retain the physical hold across unchanged contexts, refresh action targets, cancel changed bindings and retiring gesture capture, and release earlier held shortcuts before a different action. Ordinary presses, pending long presses and wheel holds now survive app changes when effective bindings are unchanged, matching the behavior since fix(core,agent): chain swipes within one gesture-button hold #1652.
  • Unbound swipes consume the gesture without falling back to its click binding. Long stationary holds still click on release. Recognition uses raw distance rather than per-report acceleration.

Testing

Validated on Linux aarch64, Rust 1.99.1, with RUSTFLAGS=-D warnings. Tests, rustdoc and the final WASM checks also used CARGO_INCREMENTAL=0:

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace (4,696 passed, 5 ignored, 0 failed across 44 result groups; isolated test home)
  • RUSTDOCFLAGS=-D warnings cargo doc --workspace --no-deps --document-private-items --exclude openlogi-ui --exclude openlogi-desktop --exclude openlogi-overlay --exclude openlogi-agent
  • ast-grep 0.45.1: ast-grep scan and ast-grep scan .github
  • cargo check -p openlogi-device-registry -p openlogi-hidpp -p openlogi-device --target wasm32-unknown-unknown
  • cargo check -p openlogi-core --no-default-features --target wasm32-unknown-unknown

Focused checks also passed: 36 swipe tests, 26 device accumulator tests, and the app-switch lifecycle regression test.

Native macOS and Windows checks were not run locally. This head is not runtime-tested on hardware. The remaining hardware checklist is:

  1. Bind left/right to desktop switching, then hold and reverse direction repeatedly.
  2. Pause, resume, and release during a held gesture.
  3. Check a stationary click and an unbound swipe.
  4. Change profiles during a hold and disconnect during a hold.
  5. Hold a push-to-talk shortcut, a pending long press, and a wheel hold while switching between apps without per-app overrides. Confirm they survive, then confirm a per-app binding change invalidates old presses.

Continuous volume response is the separate follow-up.

Related: #1634, #969, #365, #1048 and #1698.

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 2/5 Tier: plus

[Medium impact] The changes affect held gesture actions across both desktop input routes.

The PR needs fixes for both app-switch cancellation gaps and the required user-guide update before merging.

Findings

  1. P1 Held actions reach another app ▶
  2. P1 Other mice keep old actions ▶
  3. P1 Main guides omit gesture changes ▶
  4. P2 Gesture phases use dependent fields ▶

Summary

The PR removes the initial gesture hold timer and adds paced repeats shared by HID++ capture and OS hooks.

  • Gesture holds act on deliberate movement and pace repeated actions.
  • Held actions follow unchanged bindings and stop when those bindings change.

oliver-mee explicitly identifies immediate distance-based recognition, current targets for gesture actions, and preserved independent wheel holds as intentional. Continuous volume response is deferred to a separate follow-up.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Config[Effective bindings] --> Policy[Direction pacing policy]
  Policy --> HID[HID++ capture]
  Policy --> Hook[OS hook]
  HID --> Swipe[Shared SwipeAccumulator]
  Hook --> Swipe
  Swipe --> Action[Gesture action with current target]
  Action --> Worker[Button worker checks live press]
  Worker --> Output[Run action]
  App[App changes] --> Compare[Compare effective press context]
  Compare -->|Changed| Cancel[Cancel old presses]
  Compare -->|Unchanged| Keep[Keep physical hold]
Loading

Reviews (10) · Last reviewed commit: "refactor(core): simplify held gesture ru..." · Reviewed by Greptile

@davidbudnick davidbudnick added area: hid HID device discovery, permissions, reads, or writes platform: all Cross-platform issue type: bug Something is broken or behaves incorrectly labels Oct 14, 2026
@oliver-mee
oliver-mee force-pushed the feat/held-gesture-parity branch from 1651955 to b58806f Compare October 14, 2026 04:26
@oliver-mee

Copy link
Copy Markdown
Author

Rebased onto 3e52db1a64a5 on master. The conflict resolution keeps both gesture timing policies across reconnects and upstream's thumbwheel rotation reset. The new upstream thumbwheel fixture now passes its capture policies to the updated constructor.

Validated with Rust 1.99.0: workspace formatting and Clippy, workspace tests (2,230 passed, four ignored), documentation with warnings denied, ast-grep guards, and the portable-crate WASM checks all pass. This is local macOS and WASM validation, not a claim that the upstream Windows/Linux matrix has finished.

The updated head is b58806f6.

@oliver-mee
oliver-mee force-pushed the feat/held-gesture-parity branch 2 times, most recently from bd560c0 to e4c31c5 Compare October 15, 2026 05:09
@oliver-mee
oliver-mee force-pushed the feat/held-gesture-parity branch from e4c31c5 to 371ed2b Compare October 20, 2026 12:43
@oliver-mee oliver-mee changed the title fix(agent): keep directional gestures active while held fix(agent): make chained gestures responsive and action-paced Oct 20, 2026
@oliver-mee

Copy link
Copy Markdown
Author

@greptileai please review the latest head, 371ed2bebc58cc047b0ae4627b10359115fa8819, again. A fresh pass over recognition, repeat pacing, direction changes and held-output cleanup would be helpful. Thanks!

@AprilNEA AprilNEA left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase. The security read found no issues, and I approved the CI and Build runs for fe8c51e1. All required checks pass.

Changes requested:

  1. crates/openlogi-agent/src/lifecycle.rs:536 and crates/openlogi-agent-core/src/runtime/button.rs:425: ForegroundChange::ContextOnly now cancels every non-gesture press on any front-app change. On master (since #1652) these presses survive unless a per-app override is involved. With this head, a held HoldShortcut (for example push-to-talk) is released when another app takes focus, and a pending long press or wheel hold is dropped. This is outside the gesture scope. Keep ordinary presses alive on ContextOnly, or show the failing case and move the change to its own PR. The test name at orchestrator/tests/capture_plans.rs:223 no longer matches the behavior either way.
  2. crates/openlogi-device/src/session/gesture/accum.rs:287: swipe.accumulate_at(now, 0, 0) is a no-op, because accumulate_at returns before it touches state on zero motion (swipe.rs:206). Remove the call and its comment.
  3. crates/openlogi-core/src/binding/swipe.rs:81-84: GesturePolicy keeps three parallel [_; 4] arrays indexed through direction_index() -> Option<usize>. Keep each direction's settings in one struct (for example DirectionPacing { enabled: bool, min_interval: Duration }) in a single array. repeat_distances is always GESTURE_REPEAT_DISTANCE in this PR; add it in #1909, its first user.
  4. swipe.rs:377: end_at ignores _now. held_since no longer gates anything, so backdate_hold_for_test (swipe.rs:391) and its 9 test calls have no effect. Drop the unused parameter and the test seam, and store the hold as a bool or a state enum instead of Option<Instant>.
  5. accum.rs on_event_at: 8 parameters behind #[expect(clippy::too_many_arguments)]. Group the four route slices (gesture_cids, dpi_cids, gesture_button_cids, button_cids) into one struct (.agents/rules/rust.md: "Replace long parameter lists with Change/Params structs").
  6. crates/openlogi-core/.ast-grep/rules/core-gesture-timing-owner.yml:15: the pattern GesturePolicy { $$$FIELDS } only matches a struct literal, and the private fields already reject that outside swipe.rs at compile time. Remove the rule.
  7. Hardware: the PR body says this head is not tested on a device. Run the checklist from the PR body on hardware and post the result.

Not for you to change yet: removing GESTURE_HOLD_FOR_SWIPE, the 400 ms / 200 ms same-direction pacing, and unbound swipes consuming the click are default-behavior changes. The gate removal also overlaps #1996 and #2022. These need a maintainer decision; hold further work on the gate part until it is made.

Comment on lines 145 to +161
@@ -110,6 +157,8 @@ pub struct SwipeAccumulator {
/// zero after each commit, so one hold can chain swipes (left, right,
/// left...) the way Options+ does.
last: Option<GestureDirection>,
/// Recognized stroke waiting for pacing; only fresh continuation may emit it.
pending: Option<GestureDirection>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Gesture phases use dependent fields

SwipeAccumulator adds last_commit and pending beside holding, fired, last, repeat, and moved_at. Together they describe whether a hold is idle, collecting movement, continuing a stroke, or waiting to repeat. The fields permit invalid combinations, such as holding == false with last_commit.is_some(), or both pending and last being set.

This violates the owner's requirement against extending implicit state machines and must be addressed before merging. Replace these phase fields with one GestureState enum whose variants carry only valid data. Keep all writes in one transition implementation, and test each refused transition, including movement after cancellation and a pending stroke becoming a continuation without first committing. No reachable invalid combination was established, so the concern is non-blocking in behavior.

Context Used: Implicit state machines Flag any change that introduces or extends an implicit state machine: a lifecycle, phase, or mode that the code tracks through a combination of independent fields instead of one enum with one transition function. Report it as... (source)

@AprilNEA AprilNEA added the needs: decision Needs a maintainer/product decision label Nov 10, 2026 — with Manus Connector
@oliver-mee
oliver-mee force-pushed the feat/held-gesture-parity branch from f9a3fd3 to 50dd3ef Compare November 10, 2026 14:02
Comment on lines +2160 to +2168
previous.bindings != next.bindings
|| previous.gestures != next.gestures
|| previous_keys
!= self
.shared
.keyboard_bindings
.read()
.ok()
.map(|keys| keys.clone())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Held actions reach another app

Blocking. set_current_app ignores changes to pointer. On macOS, hold Back configured as LongPress(Copy, Paste) over focused window A, then focus B with the keyboard without moving the pointer. If B overrides Back with Undo, the maps still select A's actions, but PointerSelection changes from Shared to Distinct.

This comparison returns false, so apply_foreground skips cancellation. The pending press keeps Shared and can send Paste to B.

Include changes to focus admission when deciding which presses to cancel. Add a regression test that changes focus without moving the pointer during a pending long press.

Knowledge Base Used: Background agent service

return false;
}
let previous = std::mem::replace(&mut self.current_app, id);
let previous = self.hook_maps_for(self.current_device());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Other mice keep old actions

Blocking. set_current_app now compares only current_device() and the global keyboard map. In focused-profile mode, hold Back on a nonselected MX Master 3S over Bluetooth with a pending Paste long action, then enter an app overriding Back with native MouseBack. The selected mouse's maps stay unchanged, so global cancellation is skipped.

Removing the other mouse's diversion retires its capture session. That branch in watchers/gesture.rs cancels only gesture presses. The ordinary long press remains live until restoration finishes and can fire Paste in the new app.

Cancel changed bindings on every live device before retirement, while preserving unrelated holds. Add a secondary-device app-switch test with restoration delayed past the long-press deadline.

Knowledge Base Used:

Comment thread docs/gesture-response.md
Comment on lines +3 to +8
Directional gestures share one raw-distance recognizer across HID++ and OS hooks.
The first deliberate swipe and changes of direction qualify at 50 raw units;
there is no initial hold timer. Same-direction ordinary repeats require another
600 units and at least 400 ms on macOS/Linux or 200 ms on Windows. A 250 ms rest
rearms the distance threshold without bypassing action pacing. Reversals and
axis changes bypass the previous direction's repeat cooldown.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Main guides omit gesture changes

Blocking. This page describes changed gesture timing and click behavior, but none of docs/CONFIGURATION.md, docs/USAGE.md, or docs/DECISIONS.md changes or points here. The repository's documentation rule requires user-visible behavior changes in at least one of those guides.

Add a short summary and link from the relevant guide so users can find the new behavior.

Rule Used: Severity: medium. A user-visible behaviour change or a known limitation left by a fix (state lost on restart, a race window not covered, Ctrl-C skipping a hand-back) is stated in docs/CONFIGURATION.md, docs/USAGE.md, or docs/DECISIONS.md in the... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@oliver-mee

Copy link
Copy Markdown
Author

Rebased onto current main (through 98d60ca7), head 50dd3ef3. All six code points from your last review are addressed:

  1. The ContextOnly change is out of this PR entirely. Ordinary presses survive a front-app change as they do on master, so held HoldShortcut, pending long presses and wheel holds are no longer dropped.
  2. The zero-motion accumulate_at(now, 0, 0) call and its comment are removed.
  3. GesturePolicy now holds a single [DirectionPacing; 4], one struct per direction. repeat_distances is dropped here and left for fix(agent): continue volume gestures through the shared policy #1909.
  4. end_at no longer takes an unused now, and held_since plus backdate_hold_for_test are gone. The hold is a state enum rather than an Option<Instant>.
  5. The four route slices are grouped into one CaptureRoutes struct, so the too_many_arguments expect is no longer needed.
  6. core-gesture-timing-owner.yml is deleted.

Two things I am not claiming as done. Item 7, the hardware checklist, has not been run on a device yet, and I have not re-checked the test name at orchestrator/tests/capture_plans.rs:223 against the behaviour it now describes. Both are on my list.

The gate questions stay parked until the #1996 / #2022 decision is made.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: hid HID device discovery, permissions, reads, or writes needs: decision Needs a maintainer/product decision platform: all Cross-platform issue type: bug Something is broken or behaves incorrectly

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants