One of the great promises of CoP was the introduction of greater objectivity into the evaluation of skating. Advocates of CoP love to represent the current 6.0 system as completely lacking in objectivity. It's 100% subjective they say. CoP, they claim, is much more objective -- 35% percent objective ISU representative Ted Barton boasted during the Grand Prix, as though 35% was a huge accomplishment.
Reality, however, is a little different.
First, consider the current 6.0 system. Throughout its use, the 6.0 system has included, and still includes, a numerical value assigned to every jump through triple Axel. In pairs, lifts are also specified on a difficulty scale organized into six categories. Spins too, have a well established relative difficulty ranking. In addition, each level of skating has well established mid-points and ranges appropriate for the content typically performed at each level of competition.
While not specified in as great detail as CoP (and a better job could/should be done), most of the elements of skating have well established degrees of difficulty that provide some objectivity in the 6.0 system. Just how much objectivity, is subject to debate, but even if one excludes the presentation mark as completely subjective, a conservative estimate is that about 20% of the score in the current 6.0 system is objective in singles and pairs.
The ISU is currently claiming that CoP is 35% objective. But is it? The answer is, not even close.
The worst case is compulsory dance. Each compulsory dance has been divided into 6 to 8 segments, each of which has a base value assigned to it. This would be the objective part of the dance, but each couple skates the same steps, so each couple gets the same objective points, and thus the objective points have no impact on the results. The results in the compulsory dances are determined entirely by the subjective quality factors and the subjective program component scores. The results of the compulsory dance events, then, are 100% subjective, just as they have always been, and CoP offers no improvement over the past. In addition, the base values for the dance segments are normalized to 10 points for each dance, regardless of their relative difficulties. Thus, not only are the CD results completely subjective for each dance. They are completely subjective from one dance to another, making comparison of the results in different dances meaningless.
In the other dances, and in singles and pairs, the base values can have some impact on the results, since each competitor has some latitude in the choice of elements. However, just how much objectivity does this introduce into the scoring process?
We define here objective points to mean the points the skater receives for a specific action (element) not subject to interpretation by the judge, for which the skater receives a specific number of points independent of the judge's opinion about the quality of the element. The objective points, then, are the points for each element attempted with a quality rating of -3. No matter what the skater does, the skater is guaranteed at least that minimum number of points for the attempted element. Starting at a floor of -3 quality, all points for quality factor -2 and above are subjective; i.e., are subject to the interpretation and opinion of the judge.
Examining each type of program for the various disciplines of skating one finds that for the original dance and the short programs only 15-20% of the points are objective. In the free dance and free skates, only 20-25% of the points are objective. The motivated reader can use the software available on this website to run their own test cases to see for themself how the different program types compare in objectivity.
The bottom line is, the claim of 35% objectivity for CoP is a vast misrepresentation of reality. Objectivity in CoP varies from a minimum of 0% in compulsory dance, to a maximum of somewhat more than 25% in free skating. This is virtually no improvement over the current 6.0 system, and a far cry from the system that 18 months ago was promised would be primarily objective in nature. The failure to achieve greater objectivity in CoP compared to the current 6.0 system is another example of CoP's failed potential.
The fact that the scores in CoP are primarily subjective -- anywhere from 75% to 100% -- goes a long way towards explaining the lack of startling results in the Grand Prix competitions. The best skaters placed in the top third; the worst skaters ended up in the bottom third; and the mediocre skaters ended up in the middle. Just as in seasons past. Just as they do under just about any scoring system. Highly ranked skaters from past seasons did not go down in flames and poor skaters did not go roaring to the top. It was pretty much same-old same-old. Why? Because the vast majority of the scores are as subjective as they ever were, and the same judges were judging according to more or less the same criteria that they always have.
Overall, CoP has produced no real improvement this season, and in some important respects has made things worse. Both CoP and the current 6.0 system put the best skaters in the top group and the worst skaters in the bottom group, and the 6.0 system does it with a lot less complexity, confusion and cost. But while the 6.0 system uses relative placements to insure the individual placements are statistically meaningful, CoP does not, and the mathematics of CoP insures that individual placements in CoP have little statistical significance at all. Analysis of the marks in the Grand Prix competitions shows that many placements were determined by the flip of a coin in the random selection of the judges, and that any point difference between two competitors of less than 1-1.5 points is meaningless and corresponds to a virtual tie -- problems that do not exist in the current 6.0 system, or the Modern Era 6.0 system proposed by the Australian federation.
Copyright 2003 by George S. Rossano