docs: spec for mouse post-processing rework + Heun sampling

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
dog
2026-07-09 17:11:07 +08:00
parent 7890b07a01
commit 3e7a194356

View File

@@ -0,0 +1,95 @@
# Mouse trajectory quality: post-processing rework + Heun sampling
**Date:** 2026-07-09
**Status:** approved
**Scope:** inference-side only (`src/ai_mouse/`). No retraining, no ONNX re-export.
## Problem
Generated mouse trajectories look unnatural in two ways (confirmed by
diagnostic plots, 4 cases × 6 seeds, bundled weights):
1. **Endpoint artifacts** — trajectories hit the target at near-right
angles ("vertical wall" of points stacked at the target x), or hook
back after overshooting. Start segments show abrupt kinks.
2. **Exaggerated curvature** — large dome arcs on straight moves, loops
on short moves. Up to 8 direction changes >45° per trace (max 135°).
## Diagnosis
Symptom 1 is manufactured by post-processing in `_postprocess.py`:
- `enforce_forward_monotonic` hard-clips forward to [0, 1]. Natural
overshoot past the target becomes a stack of points at forward=1 with
varying lateral → the vertical wall.
- `snap_endpoints` drags the last 6 points toward (1, 0) with quadratic
easing. When the raw sample ends off-target, the drag direction fights
the trajectory's own direction → hooks.
- `smooth_start` multiplies `lateral[1]` by 1/5 and releases abruptly
after point n → start kinks.
Symptom 2 is mostly learned from data (Balabit fixed-window click-anchored
segmentation includes mid-gesture starts and composite move+hover
gestures) and is **out of scope** here — deferred to a possible follow-up
(gesture re-segmentation + retrain). Coarse 10-step Euler sampling
contributes secondary jitter and IS in scope.
## Design
### 1. Post-processing pipeline rework (`_postprocess.py`, `mouse.py`)
Current order: `snap_endpoints → smooth_start → enforce_forward_monotonic
→ gaussian_smooth(lateral)`.
New order (steps run in this sequence):
1. **Soft monotonic** (replaces `enforce_forward_monotonic`):
- No `clip(0, 1)`.
- Tolerate small backtracking: enforce `forward[i] >= forward[i-1] - 0.02`.
- Allow overshoot past 1.0; soft-compress extremes beyond ~1.08 with
tanh so the path never flies far past the target.
2. **Continuous start damping** (replaces `smooth_start`):
- Smoothstep-ramped lateral damping over the first n points; no
abrupt release, no local `max()` monotonic fix (step 1 owns that).
3. **Smoothing**`gaussian_smooth` applied to both forward and lateral
(currently lateral only).
4. **Global residual correction** (replaces `snap_endpoints`, runs last
so endpoints stay exact after smoothing):
- Compute residuals of first/last points vs (0,0)/(1,0).
- Distribute the correction over the whole curve with smoothstep
weights (weight → 1 at the corrected end, → 0 at the opposite end).
- Endpoints land exactly; approach direction stays natural.
Function signatures, the `generate()` API, and the exact-endpoint
guarantee are preserved.
### 2. Sampling: Euler → Heun (`mouse.py`)
Replace the 10-step first-order Euler loop with 10-step Heun
(predictor-corrector): per step, evaluate v at x and at the Euler
prediction, advance with the average. NFE 10 → 20; each call is a
d_model=128 transformer (~1-2 ms CPU), total latency stays ~40 ms.
Seed reproducibility unaffected (randomness is only in the init noise
and duration sampling, both unchanged).
### 3. Tests and acceptance
1. **Golden regression re-capture**`tests/unit/data/golden_mouse.npz`
is re-captured with the new pipeline (expected, intentional behavior
change; scroll golden untouched). CHANGELOG entry.
2. **Unit tests** (`tests/unit/test_postprocess.py`) — backtrack
tolerance, overshoot compression, exact endpoint hit after global
correction, correction weights 0/1 at the ends, no turns >90° on
smoothed output.
3. **Acceptance** — re-run the diagnostic script (same 4 cases × 6
seeds) and compare: `turns>45°` count drops sharply, no vertical
wall in the last 10 points. Final gate: user visually approves the
Web UI verify page (restart server; post-processing is Python-side,
no ONNX re-export needed).
## Out of scope
- Balabit re-segmentation (velocity-threshold gesture splitting) and
retraining — revisit after this lands if curvature is still
unsatisfactory.
- Scroll subsystem — no reported issues.