121 lines
3.0 KiB
Markdown
121 lines
3.0 KiB
Markdown
# ai_mouse
|
|
|
|
Human-like mouse trajectory and scroll wheel event generator. Inference runs on
|
|
ONNX Runtime; the only runtime dependencies are `numpy` and `onnxruntime`.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
pip install git+https://github.com/<owner>/ai_mouse.git
|
|
```
|
|
|
|
For GPU inference (optional), replace `onnxruntime` with the GPU variant:
|
|
|
|
```bash
|
|
pip install onnxruntime-gpu # CUDA / TensorRT
|
|
# or
|
|
pip install onnxruntime-directml # Windows DirectML
|
|
```
|
|
|
|
## Quick start
|
|
|
|
### Mouse trajectory
|
|
|
|
```python
|
|
from ai_mouse import generate
|
|
|
|
points = generate(start=(100, 200), end=(900, 400))
|
|
# [(x, y, t_ms), ..., (cx, cy, t_down), (cx, cy, t_up)]
|
|
```
|
|
|
|
### Scroll wheel
|
|
|
|
```python
|
|
from ai_mouse import generate_scroll
|
|
|
|
events = generate_scroll(start_scroll_y=0, target_scroll_y=2000)
|
|
# [{"deltaY": 120, "deltaMode": 0, "t": 32}, ...]
|
|
```
|
|
|
|
### Class API (recommended for repeated calls)
|
|
|
|
```python
|
|
from ai_mouse import MouseModel
|
|
|
|
m = MouseModel() # session created once
|
|
for target in target_list:
|
|
pts = m.generate((cx, cy), target)
|
|
```
|
|
|
|
### Custom providers / GPU
|
|
|
|
```python
|
|
from ai_mouse import MouseModel
|
|
|
|
m = MouseModel(providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
|
|
# or
|
|
m = MouseModel(providers=["DmlExecutionProvider"])
|
|
```
|
|
|
|
### Reproducibility
|
|
|
|
```python
|
|
m.generate(start, end, seed=42)
|
|
```
|
|
|
|
## API summary
|
|
|
|
| Name | Purpose |
|
|
|---|---|
|
|
| `generate(start, end, *, n_points=64, speed=None, click=True, seed=None)` | One-shot call; internal lru_cache singleton |
|
|
| `MouseModel(model_path=None, providers=None, seed=None)` | Persistent session |
|
|
| `generate_scroll(...)` / `ScrollModel(...)` | Same shape for scroll |
|
|
| `ai_mouse.errors.{ModelLoadError, GenerationError}` | Exception hierarchy |
|
|
|
|
## Thread safety
|
|
|
|
`MouseModel.generate` and `ScrollModel.generate` are safe to call concurrently
|
|
from multiple threads — ORT `InferenceSession` is itself thread-safe.
|
|
|
|
## Development
|
|
|
|
The repo contains optional dev-only tooling under `tools/` for training your
|
|
own models, running the FastAPI web UI, and evaluating output quality. Install
|
|
with the `dev` group:
|
|
|
|
```bash
|
|
uv sync --group dev
|
|
```
|
|
|
|
Common commands:
|
|
|
|
```bash
|
|
# Web UI (collect + train + verify in browser)
|
|
uv run python tools/serve.py
|
|
|
|
# Training (after collecting your own data)
|
|
uv run python -m tools train --data data/traces.jsonl --output data/models_v2
|
|
|
|
# Convert Balabit corpus to trace format
|
|
uv run python -m tools balabit-adapter --input data/balabit_raw \
|
|
--output data/pretrain_traces.jsonl
|
|
|
|
# Eval report
|
|
uv run python -m tools eval --model-dir data/models_v2 \
|
|
--reference data/pretrain_traces.jsonl --output data/eval_reports/report.md
|
|
|
|
# Re-export ONNX after retraining
|
|
uv run python -m tools.export_onnx --flow-ckpt data/models_v2 \
|
|
--scroll-ckpt data/scroll_models --output src/ai_mouse/assets/
|
|
```
|
|
|
|
Tests:
|
|
|
|
```bash
|
|
uv run pytest tests/unit # library-only (no torch)
|
|
uv run pytest tests/tools # full dev suite
|
|
```
|
|
|
|
After retraining you need to re-export and rebuild the wheel for the new
|
|
weights to ship; the in-app Verify endpoint always uses bundled weights.
|