Yes, AI has fundamentally changed medicine. Yes, it’s likely to change industries like transportation and defense forever.
It also has brought a lot to sports.
If you talk to anybody in baseball, for example, they’ll tell you how players now wear harnesses to deliver granular data to data centers, in order to crunch the numbers on every swing of the bat, or other movement by a player in a complex field. Any time a leaf rustles in the outfield, the AI picks it up.
But let’s take another sport: figure skating. That’s a pretty detailed operation, too, in its own way. Here, the use of AI data systems is not predominantly aimed at in-depth year-over-year career statistics, but in judging performers fairly.
Scoring the Moves
In a TED talk about the applications of AI to synchronized skating, Amelie Chan notes things like a “total element score” used as a metric in assessing figure skating, and how a tech panel and judging panel work together to provide results.
“Sounds pretty straightforward, right?” Chan asks rhetorically, before pointing out some of the complexities that people might not think about when they’re watching at home.
For one thing, judges have to factor in those little bumps and hiccups that occur in a figure skating presentation.
“Ice tends to be a little slippery,” Chan said.
Then, also, there’s human nature to consider.
“This sport can sometimes get political,” she said. “Human biases always exist, and as skaters, we put out the best skate that we can, and the rest is up to the panel. … (but) the top post of all time on (a particular figure skating forum) is a very heated complaint against the judging system.”
The complexity of measuring every skater requires a high degree of synchronicity, looking at things like body line and leg angle.
Chan theorizes the development of a “Synchrobot” that could do this kind of analysis consistently and fairly for every dancer.
The Individual and the Team
In discussing the challenges of getting a good baseline for such an AI analysis system, Chan points out that there are a limited number of videos online. So there’s a potential data scarcity problem. Also, data may have to be hand labeled.
To do this, she says, you’ll also need a convolutional neural network, one of the fundamental network types that evolved during the early work on deep computer vision.
Using low, medium and high layers, and things like filters and padding, the program defines features and edges to provide what Chan calls a “hierarchical representation of an image.”
“It learns on its own what filters are most effective for detecting these features without being explicitly told what to look for,” she says of CNN capabilities.
Chan goes over a sequence of verification and other tools that can conceivably handle all of these challenges.
“We can use a CNN to detect skaters in the frame, and create bounding boxes,” she says. “Then, with use of pose estimation and calculations, we can help view the technical criteria, for example, measuring the vectors and calculating the angle between the skater’s legs, or detecting whether the skater’s foot is higher than their head.”
There’s also other group analysis that can go into the mix.
“This can also be used to point out unison,” she explains. “It can suggest moments of unison to the judges, and if the skater’s limbs angles are all parallel vectors, then they must be matching. However, when skaters are in uncommon positions, a CNN that is trained on normal pose estimation data sets will ultimately fail, which is why we can resolve this by pre-training our CNN using the shuffle and learn method developed by researchers at CMU and Facebook AI. It is a sequence verification task that makes it possible to train a model to learn, unsupervised, with meaningful results. So to train it’ll extract three frames from video, either shuffling it or not, and guess whether or not the sequence is temporally correct.”
Can these tools mitigate human bias? And what else can they do?
“Through this process of sequence verification, the model can gain a sensitivity and an intuition specifically for pose estimation for figure skating. In using deep learning to observe both the individual and the team, we can take steps forward on this path towards automating scoring aspects and synchronized skating, mitigating human biases and contributing to a fair world of figure skating, but it’s clear that the possibilities are endless. If fairness to us means to use accessible resources, this could easily be tweaked for coaching and self-feedback.”
All of this, Chan suggests, will improve the scoring of ice-skating moves and performances.
“It’s up to us to question what we personally want to do with it, and what changes we personally want to see with the world,” she says in conclusion.
A Real Impact
Imagine a dozen (or maybe 16) skaters, male and female, in sophisticated and elegant shapes and combinations, moving in unison to a common rhythm.
Now imagine the power of machines to identify precise leg angles and body positioning, and bring data-driven scoring to the arena.
That’s the kind of amazing impact that new technologies will have in the sports world, not just in skating, but elsewhere, too. So it’s one thing to think about as we move through 2025, where we’re just beginning to see all of those interesting impacts on our world.