gesture-scoring-engine

Gesture Scoring Engine

Gesture Scoring Engine is a local research prototype for geometry-based hand-gesture scoring. It converts hand landmarks into a compact 26-dimensional feature vector, compares attempts with population gesture descriptors, adapts scoring strictness over repeated attempts, and visualizes how score variables can be interpreted over time.

The public repository contains the core Python code, tests, selected experiment figures, and a static browser demo. It does not include the raw HaGRID data, processed descriptor artifacts, private project notes, LaTeX report sources, or local build caches.

Open the hosted demo:

https://rt64m.github.io/gesture-scoring-engine/web_demo/

What Is Included

• Core scoring and analysis modules in src/
• Reproducibility and export scripts in scripts/
• Virtual-subject strategy experiments in experiments/
• Regression tests in tests/
• A local static showcase demo in web_demo/
• Selected paper/demo figures in report/figures/ and supplemental figures in artifacts/figures/

Gesture Scope

The maintained target set contains ten static hand gestures:

fist, four, ok, one, palm, peace, three, stop, call, rock

All scoring, demo data, and published figures are built around this ten-gesture scope.

Research Boundary

This project does not use real patient data. Patient or recovery wording in the demo refers to simulated/virtual-subject histories. Camera input in the browser demo is processed locally in the browser; the demo does not upload frames, landmarks, scores, or user state to a project backend.

Repository Layout

gesture-scoring-engine/
+-- src/                  # core feature, descriptor, scoring, threshold, progress modules
+-- scripts/              # CLI demos, descriptor building, data export, report figure scripts
+-- experiments/          # virtual-subject comparison experiments
+-- tests/                # pytest regression tests
+-- web_demo/             # static browser demo
+-- report/figures/       # selected generated figures
+-- artifacts/figures/    # supplemental generated figures
+-- pyproject.toml
+-- uv.lock
+-- README.md
`-- .gitignore

Requirements

• Python 3.12 or compatible
• uv for dependency management
• A modern browser for web_demo/
• Browser camera permission and network access to the configured MediaPipe assets if using the live classifier page

Install dependencies:

uv sync

Run tests:

uv run python -m pytest tests -q

Web Demo

The easiest way to view the demo is through GitHub Pages:

https://rt64m.github.io/gesture-scoring-engine/web_demo/

No Python environment is required for the hosted demo.

To run the same demo locally, refresh exported demo data if needed:

Refresh exported demo data:

uv run python scripts/export_demo_data.py

Start a static server from the repository root:

uv run python -m http.server 8000

Open:

http://localhost:8000/web_demo/

The demo currently includes:

• ten-gesture camera classification from one sampled screenshot per second, with the previous-second snapshot shown and analyzed below the live preview
• Palm cold-start scoring scenarios where the first adaptive score matches direct scoring, then later scores change through the learned user model and threshold state
• simulated Palm skeleton playback from a curled-finger patient attempt toward a five-finger upright target
• traces for tau, baseline, momentum, blend weight, distance, and challenge-zone state
• an inverse-scoring view where adaptive score can stay in a narrow band while tau tightens and inferred progress rises

The classifier page can be paused at any time. Pausing keeps the camera preview visible but stops the one-second snapshot sampling and analysis refresh.

Core CLI Examples

Run a cold-start scoring demo:

uv run python scripts/demo_cold_start.py --gesture fist --profile improving_sequence

Run a dynamic-threshold simulation:

uv run python scripts/simulate_progression.py --scenario fatigue_dip --rounds 12 --noise 2

Generate a simulated progress report:

uv run python scripts/demo_progress.py --scenario linear_recovery --days 30

Run the virtual-subject strategy comparison:

uv run python experiments/run_comparison.py --patients-per-curve 3 --sessions 8 --attempts-per-session 10

The patients argument denotes virtual subjects generated by the simulator, not real patients.

Data Notes

The descriptor-building pipeline expects HaGRID-style landmark annotation JSON files under data/raw/, but raw and processed data are intentionally not committed. To rebuild descriptors locally, place or download the required annotations and run:

uv run python scripts/download_hagrid_annotations.py
uv run python -m scripts.build_descriptors

Generated descriptor and feature files are local artifacts and remain ignored by Git.

Validation Snapshot

Recent local validation before publication:

• uv run python -m pytest tests -q: 47 tests passed
• LaTeX citation/reference audit for the private report: no undefined citations, missing figures, or BibTeX errors
• Browser smoke test for the static demo: navigation, scenario switching, playback controls, inverse scoring view, desktop layout, and mobile-sized layout passed

License

No license has been declared yet. Until a license is added, all rights are reserved by default.