docs: spec for mouse post-processing rework + Heun sampling
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,95 @@
|
|||||||
|
# Mouse trajectory quality: post-processing rework + Heun sampling
|
||||||
|
|
||||||
|
**Date:** 2026-07-09
|
||||||
|
**Status:** approved
|
||||||
|
**Scope:** inference-side only (`src/ai_mouse/`). No retraining, no ONNX re-export.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Generated mouse trajectories look unnatural in two ways (confirmed by
|
||||||
|
diagnostic plots, 4 cases × 6 seeds, bundled weights):
|
||||||
|
|
||||||
|
1. **Endpoint artifacts** — trajectories hit the target at near-right
|
||||||
|
angles ("vertical wall" of points stacked at the target x), or hook
|
||||||
|
back after overshooting. Start segments show abrupt kinks.
|
||||||
|
2. **Exaggerated curvature** — large dome arcs on straight moves, loops
|
||||||
|
on short moves. Up to 8 direction changes >45° per trace (max 135°).
|
||||||
|
|
||||||
|
## Diagnosis
|
||||||
|
|
||||||
|
Symptom 1 is manufactured by post-processing in `_postprocess.py`:
|
||||||
|
|
||||||
|
- `enforce_forward_monotonic` hard-clips forward to [0, 1]. Natural
|
||||||
|
overshoot past the target becomes a stack of points at forward=1 with
|
||||||
|
varying lateral → the vertical wall.
|
||||||
|
- `snap_endpoints` drags the last 6 points toward (1, 0) with quadratic
|
||||||
|
easing. When the raw sample ends off-target, the drag direction fights
|
||||||
|
the trajectory's own direction → hooks.
|
||||||
|
- `smooth_start` multiplies `lateral[1]` by 1/5 and releases abruptly
|
||||||
|
after point n → start kinks.
|
||||||
|
|
||||||
|
Symptom 2 is mostly learned from data (Balabit fixed-window click-anchored
|
||||||
|
segmentation includes mid-gesture starts and composite move+hover
|
||||||
|
gestures) and is **out of scope** here — deferred to a possible follow-up
|
||||||
|
(gesture re-segmentation + retrain). Coarse 10-step Euler sampling
|
||||||
|
contributes secondary jitter and IS in scope.
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### 1. Post-processing pipeline rework (`_postprocess.py`, `mouse.py`)
|
||||||
|
|
||||||
|
Current order: `snap_endpoints → smooth_start → enforce_forward_monotonic
|
||||||
|
→ gaussian_smooth(lateral)`.
|
||||||
|
|
||||||
|
New order (steps run in this sequence):
|
||||||
|
|
||||||
|
1. **Soft monotonic** (replaces `enforce_forward_monotonic`):
|
||||||
|
- No `clip(0, 1)`.
|
||||||
|
- Tolerate small backtracking: enforce `forward[i] >= forward[i-1] - 0.02`.
|
||||||
|
- Allow overshoot past 1.0; soft-compress extremes beyond ~1.08 with
|
||||||
|
tanh so the path never flies far past the target.
|
||||||
|
2. **Continuous start damping** (replaces `smooth_start`):
|
||||||
|
- Smoothstep-ramped lateral damping over the first n points; no
|
||||||
|
abrupt release, no local `max()` monotonic fix (step 1 owns that).
|
||||||
|
3. **Smoothing** — `gaussian_smooth` applied to both forward and lateral
|
||||||
|
(currently lateral only).
|
||||||
|
4. **Global residual correction** (replaces `snap_endpoints`, runs last
|
||||||
|
so endpoints stay exact after smoothing):
|
||||||
|
- Compute residuals of first/last points vs (0,0)/(1,0).
|
||||||
|
- Distribute the correction over the whole curve with smoothstep
|
||||||
|
weights (weight → 1 at the corrected end, → 0 at the opposite end).
|
||||||
|
- Endpoints land exactly; approach direction stays natural.
|
||||||
|
|
||||||
|
Function signatures, the `generate()` API, and the exact-endpoint
|
||||||
|
guarantee are preserved.
|
||||||
|
|
||||||
|
### 2. Sampling: Euler → Heun (`mouse.py`)
|
||||||
|
|
||||||
|
Replace the 10-step first-order Euler loop with 10-step Heun
|
||||||
|
(predictor-corrector): per step, evaluate v at x and at the Euler
|
||||||
|
prediction, advance with the average. NFE 10 → 20; each call is a
|
||||||
|
d_model=128 transformer (~1-2 ms CPU), total latency stays ~40 ms.
|
||||||
|
Seed reproducibility unaffected (randomness is only in the init noise
|
||||||
|
and duration sampling, both unchanged).
|
||||||
|
|
||||||
|
### 3. Tests and acceptance
|
||||||
|
|
||||||
|
1. **Golden regression re-capture** — `tests/unit/data/golden_mouse.npz`
|
||||||
|
is re-captured with the new pipeline (expected, intentional behavior
|
||||||
|
change; scroll golden untouched). CHANGELOG entry.
|
||||||
|
2. **Unit tests** (`tests/unit/test_postprocess.py`) — backtrack
|
||||||
|
tolerance, overshoot compression, exact endpoint hit after global
|
||||||
|
correction, correction weights 0/1 at the ends, no turns >90° on
|
||||||
|
smoothed output.
|
||||||
|
3. **Acceptance** — re-run the diagnostic script (same 4 cases × 6
|
||||||
|
seeds) and compare: `turns>45°` count drops sharply, no vertical
|
||||||
|
wall in the last 10 points. Final gate: user visually approves the
|
||||||
|
Web UI verify page (restart server; post-processing is Python-side,
|
||||||
|
no ONNX re-export needed).
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
- Balabit re-segmentation (velocity-threshold gesture splitting) and
|
||||||
|
retraining — revisit after this lands if curvature is still
|
||||||
|
unsatisfactory.
|
||||||
|
- Scroll subsystem — no reported issues.
|
||||||
Reference in New Issue
Block a user