Controversy at the European Championship flares over the results, and the use of the presentation marks.
The recent European Championships have generated a great deal of controversy over the correctness of the results. Much of this controversy has centered around the use of program component marks, with some questioning whether the judges are using these marks correctly, and some even suggesting the judges may be using these marks to manipulate the results.
Even a cursory look at the three program components for presentation (Performance, Choreography and Interpretation) makes it clear something is fishy. In the ladies and men's events, for example, with a total of about 120 performances (nearly 1500 sets of marks) every judge gave every skater nearly equal marks for Performance, Choreography and Interpretation (within 0.25 of their average). This is not a problem unique to Europeans. It was true throughout the Grand Prix and Junior Grand Prix this season. In fact it has been true from the very beginning of the use of the new system two years ago.
You are a good skater, all three marks are the same. You are a bad skater, all three marks are the same. You are an in the middle skater, all three marks are the same. Something clearly is going on here. What is it?
Some in the ISU say all that is going on here is the need for additional education, but that seems simplistic. The judges have been educated for getting on three years now and none of them (based on the marks they give) seem to be able to distinguish between the presentation program components. Yes, more education is always a good thing, but that does not appear to be the source of the problem, for if it where at least some of the judges would get it right by now and spread out their marks by more than an insignificant +/- 0.25 points -- but none did at Europeans.
There are many criteria involved in the presentation program components (up to 22), but according to the marks given, the skaters do all these many different things equally well, even the skaters in the middle who otherwise can't seem to do any of the many other things marked equally well. That strains credibility.
It is suggested here that much of the problem is that the presentation program components are just too darn complicated for the average person to mark independently, and too arcane for the average judge (skater and coach for that matter) to relate to.
With the new judging system it is as though we all woke up one day and it was decided everybody had to buy a new car, and the engineers in Detroit designed it so each foot and hand had 6 tasks to continually accomplish while driving. Then, when there are a lot of accidents and people aren't driving so well, the engineers say, it is all the driver's fault, they simply aren't well enough educated, when the real problem is the design of the car, which needs to be simplified.
During Worlds in Nagano (2002) I had a long discussion with Frank Carroll over the future of the second mark in the proposed system. He said that in his view having seven criteria in the 6.0 presentation mark were more than needed. He liked the idea of breaking it down into just three easy to understand marks. However, instead of taking the seven criteria and consolidating them into three simple marks what we have now are three marks with about seven criteria each. Not what people bargained for at all. Further, this approach seems to requires an advanced degree in music theory and dance appreciation to understand it, something most people in this sport don't have and never will.
The complexity of the presentation marks is completely at odds with the original purposes of the new system as it has been continually described in ISU materials from day-one -- to make the scoring more accessible, more understandable to everyone, and more judgeable as a sport. Instead the presentation program components are less accessible and more arcane. This is not in the best interest of rebuilding fan interest and participation in the sport.
It is also driving many of the judges nuts, though few will publicly admit it. You cannot expect the average person to mark the elements in real time, and keep track of 22 performance criteria, and keep track of short, medium and long transitions, and their quality, and whether all the skating skills are demonstrated, and get it all correct.
The uniformity in the presentation marks suggests that they are currently too difficult for the average person to judge, too difficult for the average spectator to understand, and too complex for the average skater (who is about 12 years old in the US) to understand. The ISU seems to have lost sight that not only must the judge be able to understand and use the presentation marks correctly, the average coach has to be able to understand it to put everything into a program that is asked for, and the average skater (those 12-year-olds) has to be able to understand it if they are going to learn it and present it on the ice.
The ISU needs to embrace the original goals for the new scoring system (which are admirable) and simplify the three presentation program components so they capture in the marks what is essential to be judged in this part of competition. Really, just how many aspects of performance does one need to decide who is the best skater. Not 22, not by far. And we are after all judging a sport here and not the Opera.
In parallel with simplification, education has its important role. But education is a two sided coin. The student must be committed to learning the material, and the instructor must be effective in communication the concepts.
Based on the marks, the ISU education efforts so far seems to have been completely ineffectual in communicating the concepts. The descriptions of the presentation program components in the ISU materials are so full of weasel words, confusing definitions and incorrect terminology it is a wonder if anyone other than those who wrote them have but the vaguest idea of what they are about. A much better job is needed here.
A few examples from the ISU documentation to illustrate this point:
The judge is to assess the "physical, emotional and intellectual involvement" of the skater. How one divines the emotions and intellectual state of a 10-year-old Juvenile girl or a 14-year-old Novice man from their skating is a mystery me, as is the reason why one would even want to try. Whether the skater is "physically committed" and "sincere in emotion" can only be guesswork on the part of the judge.
"Carriage is a trained inner strength of the body that makes possible ease of movement from the center of the body." Wrong. According to the dictionary and past convention in skating, carriage is "the manner of holding and moving one's head and body." It takes strength and training to hold and move one's body in a controlled way. But carriage is not strength. Strength is "the state, property or quality of being strong." It is "the power to resist strain or stress." But strength is not the position in which one holds one's body.
The judge should assess the extent to which "the skater radiates energy resulting in an invisible connection with the audience." How does one measure an invisible connection with the audience? By an applause meter? All the judge can know for sure is if the skater makes a connection with the judge; and if the skater doesn't make an invisible connection with the judge, maybe the problem is with the judge and not the skater.
The judge should assess whether "movement phrases are distributed in such a way they communicate from every angle in a 360 degree skater-viewer relationship." The judge only views the program from one direction. Whether the program "communicates" in the same way, or to the same extent, in every possible direction is pure guesswork on the part of the judge. As a photographer who frequently shoots the same program from significantly different direction during a season, I know that different programs generally communicate equally well in every direction to different extents. But I only know that because I have seen them from several different directions to compare them. The judges don't.
In regard to education, the descriptions need to be rewritten in terms that the average judge, skater, coach, parent and fan can understand in a clear and precise fashion. This needs to be done in the language of skating and not the language of music and modern dance.
Further, some concrete guidelines along the lines of "if a skater does this (easily identifiable thing) then that is 2.0 or 5.0 or 7.0 use of the ice" or whatever. Currently the judges must not only struggle with the complexity and the arcane descriptions but also a percentile approach to awarding the marks. The latter is not a particularly precise way to assign the marks
It should be possible to get out of the presentation marks what is needed for competition, but not as they are currently structured and described. The system is a done deal at the ISU. It will be a done deal in the U.S. in May. It's time for the ISU to get out of the defensive mode and into the evolutionary mode of making improvements where they are needed. One area where they are desperately needed is in the presentation marks.
Copyright 2005 by George S. Rossano
7 Feb 2005