Technology

Ice Coach™ uses computer vision to analyze figure skating videos entirely within the browser. All processing is client-side — no data is sent to any server.

Processing Flow

Skeleton Detection

MediaPipe PoseLandmarker detects 33 body keypoints from each video frame. Accelerated by GPU (WebGL). When multiple people are visible, IoU-based tracking follows the target skater.

MediaPipe PoseLandmarker, IoU Tracking

Jump Segment Detection

Calculates the vertical position of the center of gravity from skeleton data and detects jump segments (takeoff to landing) from its changes.

Center of Gravity (CoG) Velocity

Trajectory Rendering

Tracks blade contact points frame by frame, removes noise with median filter and moving average, then renders smooth Catmull-Rom bezier curves.

Median Filter, Moving Average, Catmull-Rom Spline

Scale Calibration

MobileSAM detects the skater's region, and combined with the input height, calculates the pixel-to-real-world conversion factor.

MobileSAM (ONNX)

3D Position Recovery

Recovers 3D rink positions from monocular video using a pinhole camera model. Depth is computed from the pixel size of the skeleton (spine): Z = S_real × f / S_px (S_real: real spine length [m], f: focal length [px], S_px: observed spine length [px]). X, Y are recovered from the hip screen coordinates: X = (hip_x - cx) × Z / f. Post-processing: median filter for spine shortening during rotation, Gaussian smoothing, and velocity clamping.

Pinhole Camera, Median Filter, Gaussian Smoothing, Velocity Clamp

Under-Rotation Detection

Computes body facing direction from the MoCap shoulder line perpendicular on the XZ plane. Quantifies under-rotation as the angle difference between landing and settled orientation (averaged over 0.1–0.35s post-landing). Subtracts 35° checkout offset for normal post-landing rotation, then classifies by ISU criteria (OK/q/</<<).

MoCap Shoulder Facing, Settled Averaging, Checkout Offset, ISU Thresholds

Playback & Rendering

Pre-renders analysis results (skeleton + trajectory) as ImageBitmaps, then overlays them in sync with video frames via requestVideoFrameCallback.

Canvas 2D, requestVideoFrameCallback, ImageBitmap

Architecture

Video File (local)
    |
    v
MediaPipe PoseLandmarker (WebGL GPU)
    |  33 keypoints x N frames
    v
Jump Detection (CoG peak)
    |  Jump segments
    v
Smoothing (Median + Moving Average)
    |  Noise-free trajectory
    v
MobileSAM + Height Calibration
    |  Pixel-to-meter scale
    v
3D Recovery (depth ∝ ref/obs size)
    |  Rink coordinates
    v
Pre-render Overlays (ImageBitmap[])
    |
    v
Canvas + requestVideoFrameCallback
    |  Frame-synced display
    v
Browser (no server)

Privacy by Design

All processing runs entirely in the browser. Video, skeleton data, and analysis results are never sent to any server. Works offline after the initial download.

Open Source

The technical approach and algorithm design philosophy behind Ice Coach are published as technical reports.

me-keh-dev/ice-coach-core

Technical reports & algorithm documentation

Software used in Ice Coach

Library	License
MediaPipe Tasks Vision	Apache 2.0
ONNX Runtime Web	MIT
ONNX Runtime Common	MIT
MobileSAM	Apache 2.0
Next.js	MIT
React	MIT
React DOM	MIT
Three.js	MIT
React Three Fiber	MIT
React Three Drei	MIT
Zustand	MIT
Suspend React	MIT
its-fine	MIT
React Use Measure	MIT
Camera Controls	MIT
Troika Three Text	MIT
Scheduler	MIT
use-sync-external-store	MIT
styled-jsx	MIT
@swc/helpers	Apache 2.0
@dimforge/rapier3d-compat	Apache 2.0
Protobuf.js	BSD 3-Clause
FlatBuffers	Apache 2.0
fflate	MIT
Draco3D	Apache 2.0
hls.js	Apache 2.0
Potpack	ISC
Maath	MIT
Meshline	MIT
@monogrid/gainmap-js	MIT
detect-gpu	MIT

Contact

For technical questions or feedback, please reach out via GitHub Issues.