Sports video annotation: player tracking, pose & ball data
Sports video is the most demanding tracking environment we label. A player serves at full extension, sprints, and changes direction in a fraction of a second; the ball travels faster than almost any object in consumer video; broadcast cuts change the angle without warning. Sports-analytics models need annotation that survives all of it.
What we annotate in sports footage
Tennis is the reference sport we have shipped, but the same schema transfers directly to badminton, cricket, football, basketball, hockey, and combat sports-anywhere the questions are 'who is where, doing what, when.'
| Label type | Sports use | Examples |
|---|---|---|
| Player tracks (per-frame masks) | Player tracking, movement & speed metrics | each player tracked with a persistent identity for the whole rally |
| Bounding boxes | Detection models, broadcast graphics | players, referees, ball kids, equipment |
| Keypoints / skeletons | Pose estimation, biomechanics, technique | full-body joint skeletons per player per frame |
| Ball annotation | Ball tracking, bounce detection, trajectory | small-object boxes with occlusion-aware tracks |
| Event / frame tags | Action recognition, highlight generation | serve, forehand, backhand, volley, point start/end |
| Court segmentation | Homography, coordinate mapping | court surface, lines, net |
How athlete tracking works
An annotator clicks the player once; the segmentation engine returns a body-accurate mask-including through motion blur-and one more click sends it to the tracker, which follows the athlete through the full clip. Players cross behind the net post, behind each other, behind graphics: the track marks those frames outside and re-acquires the same identity, so identities never swap mid-rally.
Player-tracking metrics-distance covered, sprint counts, court-coverage heatmaps-are integrals over per-frame positions. Jittery or interpolated annotation adds noise to every one of those integrals, which is why we produce a mask on every frame with motion-adaptive smoothing rather than the keyframe shortcuts most tools take.
Pose data & event grammar
Keypoint skeletons turn tracking into biomechanics: a serve annotated at joint level lets a model measure racket-drop depth, knee flexion, or landing asymmetry. Skeleton definitions are configurable per project. Frame-level event tags segment a broadcast into its grammar-point start, serve, stroke type, bounce, point end-and live in the same project as the tracks, so the export aligns them on a shared frame clock.
Formats & delivery
Sports datasets export as COCO JSON (boxes, masks, and keypoints in the standard keypoints schema), YOLO for real-time player detectors, and per-frame annotated images with tracking applied for sequence models. Event tags export alongside, so an action-recognition model and a detection model train from the same versioned delivery.