Slicing and dicing soccer: automatic detection of complex events from spatio-temporal data

The automatic detection of events in sport videos has important applications for data analytics, as well as for broadcasting and media companies. This paper presents a comprehensive approach for detecting a wide range of complex events in soccer videos starting from positional data. The event detector is designed as a two-tier system that detects atomic and complex events. Atomic events are detected based on temporal and logical combinations of the detected objects, their relative distances, as well as spatio-temporal features such as velocity and acceleration. Complex events are defined as temporal and logical combinations of atomic and complex events, and are expressed by means of a declarative Interval Temporal Logic (ITL). The effectiveness of the proposed approach is demonstrated over 16 different events, including complex situations such as tackles and filtering passes. By formalizing events based on principled ITL, it is possible to easily perform reasoning tasks, such as understanding which passes or crosses result in a goal being scored. To counterbalance the lack of suitable, annotated public datasets, we built on an open source soccer simulation engine to release the synthetic SoccER (Soccer Event Recognition) dataset, which includes complete positional data and annotations for more than 1.6 million atomic events and 9,000 complex events.