( 3 February 2023)  Sitting rinkside for days on end at a competition one has ample time for the mind to wander. One subject on which the mind often wanders is the quality of the judging.  Sitting six days rinkside at US Nationals one question that came more and more to the front for me is whether skating has become an unjudgeable sport.  It certainly seemed like it was going in that direction several years when we wrote about the accuracy of calls due to the limitations of the replay system technology in use, and after this week in San Jose the answer seems decidedly in the affirmative.

While it is certainly true that the best skaters in a competition end up towards the top of the standings and the worst at the bottom, a strong case can also be made that the specific placements of the skaters are often only an approximation of the true placements the skaters deserve.  When IJS was first introduced, the calculated uncertainty in skater's marks (based on statistics of the marks and possible errors in calls and GoEs given) was about three points; that is, if skaters scored within three points of each other the result was not statistically significant and a reverse order of placement for the skaters being the truth was a significant possibility.  I recall back then (in 2004) ISU judge Steve Winkler lecturing me that over time the statistical agreement of the judges would get better and we would see considerable improvement.  Over the intervening nearly 20 years, however, the situation has only gotten worse.  Statistical agreement of the judges has not improved, and the accuracy of calls has gotten increasingly suspect as the difficulty of the elements performed has increased over the last two decades.

In 2004 quads were less frequent in programs for the small number of men who attempted them, and women rarely included triple Axel or above in their programs.  Today, triple Axels and above are common for both men and women (at least the Russian women), and some are even attempted by men even as low as the Novice level.  At the same time the human capabilities of officials to correctly evaluate these difficult elements, and the capability of replay systems to aid officials, have not kept increased.  For quads it appears we are at the point that errors are too fleeting for any human to perceive unassisted, and for the available technology to reliably come to the rescue.

There are reasons other than element calls for why skating is now unjudgeable; a significant one, in particular, being the fool's errand of trying to meaningfully combine the evaluation of the technical merit of a program with the artistic success of the program in the same performance.  This is something we hope to address elsewhere, but for this discussion we continue by looking at some of the calls from the 2023 U.S. Nationals in San Jose.  The photos included in this article were taken from rinkside sitting a few feet from the replay video camera so they are nearly identical to the view seen by the Technical Panel and judges viewing the replays, with the same perspective.

We first have three examples from Ilia Malinin's free skate program: his quad Axel, a quad Salchow, a triple Lutz and a triple toe loop.

Quad Axel

The following two pictures show the takeoff and landing of the quad Axel (element 1) which was called on the quarter.

The skater takes off with about one-eighth pre-rotation at the foot and hips, typical for an Axel jump takeoff.

When trying to determine the skater's rotation angle at the takeoff of a jump it is not uncommon for the foot, hips and shoulders to be slightly different, as it is in this case, though all the body parts quickly synchronize in the ascent of the jump as the skater pulls in.

At the landing is the missing rotation one-quarter as called, more and deserving of an under-rotation call, or less?

Basically, the answer comes down to where would you visualize the perpendicular to the line of flight on the ice against which one must compare the orientation of the blade to make the call (and doing it without having the line of flight shown above).  Without a perspective calculation and a reference grid - something that the current replay technology does not provide but which is critically needed for accurate calls - one's perception is only an estimate.

[Also notice in the above image, that as on some jump takeoffs, for some jump landings the rotation angle of the blade, shoulders, and hips can be distinctly different.  Which body part should be used to measure the skater's rotations in the air?  Is using the blade always the right choice?]

Taking advantage of the post-competition luxury of being to do the perspective calculation at leisure, the following image shows our calculated perspective correction for the one-quarter-missing-rotation line.

The snow kicked up by the landing makes it difficult to say if the missing rotation is exactly one-quarter, or a few degrees less, but our impression is that the landing is a bit before the quarter. Our measurement uncertainty of plus/minus a few degrees, however, prevents a definitive conclusion.

[For those curious why this attempt was not successful, analysis shows the attempt had as much time in the air as successful attempts at prior competitions, and as much takeoff angular momentum.  The main difference was this attempt was a bit slower getting over the right side and pulled in, resulting in a slightly lower average rotation rate in the air, by about 3%.]

Quad Salchow

Next we have the takeoff and landing of the quad Salchow (element 4) called fully rotated.

Midway through throwing free leg, there is already one-quarter pre-rotation.  In the frame immediately preceding this one the skater is facing to the right entering the jump.  Here they are facing towards the boards.	Instant of takeoff, the jump is fully forward with a half-rotation missing, and the skater facing to the left.  The toe picks are still on the ice.

[Note: Except for toe loops, pre-rotation is not called by technical panels, and skaters are given credit for all the rotation on the ice they execute prior to takeoff, earning the same points as a skater that does not pre-rotate.]

Here the landing is facing forward 13 msec prior to the instant of landing.  This is close to a downgrade, however the skater rotates about 10 degrees in those 13 msec (based on the rotation rate of the skater at this point in his flight), making the missing rotation about 160-170 degrees.  With the pre-rotation and missing rotation at the landing,  this jump had only a bit more than 3 rotations in the air.  4S< with GoEs of -3 reduce the point value of this element by 7.6 points, compared to the actual points scored.

The replay system, however, appears to not have caught this.  In our opinion it is because the replay system has neither the time resolution nor spatial resolution needed for the most difficult jumps; and unlike our ability to do rigorous mathematical analysis of individual images in post-competition leisure, the replay system has none of that capability, though it easily could. Due to the limitations in human perception and the technical limitations of the replay system, the Technical Panel cannot determine the exact instant of landing nor the exact position of the blade at landing, making elements impossible to measured accurately, and if they cannot be measured accurately they become unjudgeable.

To identify the instant of landing with the current replay system, one is looking for one fuzzy needle-in-a-haystack frame that the replay system may not have even captured, for a skater who will have rotated a considerable amount in the time between frames.  On the contrary, what we can do here that the replay system cannot, is interpolate higher resolution images both spatially and temporally to alleviate this limitation.  In addition, the images we analyze have about eight times the spatial resolution of the replay system.

3Lz+3A+3T+SEQ

This is the killer element of the program, worth a base value of 19.91 points when executed in the second half of the program.  This combination was called fully rotated for all three jumps.  We zero in here on the triple Lutz and the triple Toe loop.  The triple Axel took off with the typical Axel pre-rotation and was full rotated at the landing.

3Lz

The triple Lutz takeoff is pre-rotated about one-eighth rotation at the blade, and a bit more at the shoulders, an amount that is normal for triples and quads.

At the landing, the jump is missing 100 degrees of rotation, give or take 10-15 degrees, by our measurement.  The blade position on this landing is very close to that for the quad Axel.

This landing has the same issues as the landing of the quad Axel with respect to deciding exactly at what position angle the blade made landing contact with the ice.  Where would you visualize the one-quarter reference line to determine if this is landed at the quarter, or slightly more or less?  [The one-quarter reference line added to the quad Axel image does not apply here, as the viewing geometry is different.]

3T

In the three jump sequence the direction of the skater changed during the execution of the jump sequence.  The line of flight was to the upper left for the triple Lutz, right to left perpendicular to the direction view for the triple Axel and toward the camera and to the lower right for the ending triple toe loop, but mostly towards the camera.

The following shows the triple toe loop takeoff.   The red line shows the projected line of fight from takeoff to landing in the ice.

Entering the jump, just prior to the tap.  The direction of entry is towards the camera and to the lower right in the image.	During the tap the skater starts turning forward, throwing the free leg, and arms as in an Axel takeoff, as the skater takes on an arms overhead air position.
Instant of takeoff with toe picks still on the ice  The skater continues the Axel style takeoff, turning on the ice fully forward. At this point the skater has missing rotation of 1/2 of a rotation on the ice.	The skater has landed with rotation fully forward.

With the pre-rotation, this "triple" jump is measured to have 2 rotations in the air.  This is a textbook example of a "toe Axel" jump takeoff.  A downgrade call would have cost the skater at least 4.1 points.

[Dynamically, the elbows of the skater are too far from the rotation axis of the skater and slow the rotation compared to other possible air positions.  The skater also has only 0.44 seconds in the air, which is the root cause for the missing rotation.  The arms-overhead air position is only dynamically advantageous if the arms are completely straight and the hands on the rotation axis.]

Lucas Broussard 4Lz

Similar measurement of triples and quads for several other skaters show examples of significant pre-rotation or uncalled (or ambiguous) missing rotation on the landings. Singles and doubles rarely have these issues.  Some notable examples, without adding more images to this already too long article, are under-rotations for Amber Glenn and Isabeau Levito, and forward takeoffs for Levito.

Instead, we close with Lucas Broussard's quad Lutz from the Junior Men's free skate.

For this jump there is nearly a full half-rotation of pre-rotation.  The skater was facing to the right on the tap.

Despite the poor air position on the takeoff, the skater salvaged this jump expertly and landed fully backwards, giving him a bit more than 3.5 rotations in the air.

We also examined Broussard's triple Axels and measure that they takeoff with the typical small pre-rotation for Axel takeoffs, but land nearly fully backwards.  These examples would offer some interesting dynamical studies.

Other Unjudgeable Elements

Within our definition of an unjudgeable element being one that cannot be accurately measured to definitively determine what was actually executed, elements other than jumps fall into this definition.  These include spins (in all disciplines) throws and twist lifts.

Spins (including pair and dance spins):  The replay system has neither the temporal nor spatial resolution to accurately measure the exact positions of the skaters (so whether they meet rules and levels requirements) nor the the exact number of rotations in position.  In addition, the replay system lacks a reference grid to assist determining if body positions meet requirements.

Throws:  These have the same issues as jumps in accurately determining takeoff and landing missing rotation.

Twist Lifts:  Twists can be called downgraded, but never seem to be, even though we have photographed a few over the years.  Twists are not called for under-rotations (the codes do not exist), even though under-rotations are not uncommon.  Even if these errors were attempted to be called, the replay system is incapable of making a precise measurement of the woman's exact rotations in the air, since in a triple twist lift the woman rotates nearly as fast as in a triple jump.

In addition to the limitations facing Technical Panels in using current reply equipment of limited capability, it is even worse for the judges.  At major competitions judges have replay screens of their own, but these are smaller and lower resolution than the Technical Panel screens.  So judges are even more limited in their ability to truly discern in detail what is happening for many elements.

In our opinion, the inability to accurately and consistently measure what takes place in the highest point value elements makes skating an unjudgeable sport with the current technology, and makes the details of the result nothing more than the educated guesswork of the officials.  In a baseball analogy, the technical panels are currently like an umpire with really bad cataracts trying to call balls and strikes.  Another analogy might be trying to resolve a photo finish in speed skating where a blurry camera captures images only 30 times a second and the finish line is not marked on the ice.

Accurate consistent calls for all competitors that makes skating quantitatively judgeable will not be achieved until a next generation of reply system is created.  Why the delay?