The ordinal scoring system used in competitive ice skating is admittedly a bit obscure. Nevertheless, given the computerless era in which it was developed it is a marvel of ingenuity in the way it makes use of the greater ability of humans to make comparative judgments as opposed to absolute judgments, and in the way it minimizes the effects of systematic bias in determining the results of a competition. From a mathematical standpoint it can be shown to be superior to all other scoring systems currently in use in other sports that make use of human judges in determining results. But boy, is it hard to explain to a newcomer how it works.
For several years the scoring system has been under continuous attack from one quarter or another. The ISU, and more recently the IOC, have come to believe that the public does not understand how the system works, and thus is suspicious of the results it produces.
Given the large numbers of spectators at actual events who can be found recording scores and calculating results as event unfold it would appear that most enthusiasts who closely follow the sport actually do understand how the system works; but among more casual observers, however, whose contact with competitions is via TV or the news media understanding of the system is less common. Over the years TV, the media, the ISU, and the USFSA have made little or no attempt to educate the general public about the scoring system. In discussing this with one prominent skating commentator we were told that they tried once, it was a disaster, and they would never do it again. About the only thing TV does better in this respect than competition organizers, is TV's practice of showing the ordinals for each skater, so the TV audience gets a better idea of how consistent the judges are - and whose marks may be out of line - than does the audience in the arena.
In 1995 the skating community began tinkering with the idea of changing to a more user friendly scoring system. The so-called "clipped mean" method of scoring was considered. This is the method used in gymnastics where the high and low marks from the panel are dropped and the remaining scores are averaged (or totaled in a "clipped total" method). Changing to this method was close to being adopted by both the USFSA and the ISU, but after it was shown through mathematical analysis that the clipped mean method is less immune to judges bias than the ordinal method the idea was dropped. Nevertheless the method, which is used in professional competitions, is now used in the USFSA ProAm competitions.
A second area of criticism that has arisen recently is the fact that in the ordinal method a lower placing skater performing after higher placing skaters can change the order of finish of the higher placing skaters. For example, in a few major competitions in the past few years a skater performing last who ended up placing third or fourth caused the order of the top two skaters who had already performed to swap places. On several occasions this season ISU President, Ottavio Cinquanta, refered to this as a great injustice to the skaters swapping place and characterized this as a flaw in the system which demonstrated that it needed to be replaced. More recently at the World Championships in Lausanne, the IOC President, Juan Antonia Samaranch piled on also criticising the system for being too difficult to understand and unfair in allowing the placements of skaters to be altered after they had completed their performances.
The characteristic of the ordinal system which allows lower placing skaters to affect the order of higher placing skaters, in reality, is not so much a flaw of the method itself, but of the practice of releasing results piecemeal.
Because humans are better at making comparative judgment versus absolute judgments, the system is designed, through the use of the ordinals, to use comparative placements in determining the results. However, in order to determine the final results using this approach the scores for all the skaters must first be known; i.e., the event must be over.
Suppose for example, with one skater to go the top two leaders have a 5-4 split for first place marks. If the final skater is placed above the two prior leaders by a few judges that skater may end up third, but in the process may take away some of the first place marks from the former leader and cause the top two skaters to swap places. If the results are released piecemeal one gets to see the way the judges marks affect the skaters placements, but the process looks odd. If the results were only released at the end of the event ,all one would see is that the skater who ultimately is placed first won an a closely split panel (frequently on a majority of seconds), and nothing would seem odd about it.
So if the system is so great, why change it?
Because results are released skater by skater now, because image is everything, and because the integrity of the sport rests in part on the public believing that the results are fair.
Although Mr. Cinquanta has said on several occasions that the system will not be changed merely to indulge the public, and that it was the primary responsibility of the ISU to protect the sport; it is clear that public perception is largely motivating the search for a better system.
In the context of the above discussion, we infer that a better system from the ISU's perspective would be one that is easy to understand, is at least as immune to judges bias as the current system, and does not allow the results of lower placing skater performing later to change the order of finish of higher placing skaters who skater earlier; i.e., the piecemeal results and the final results must be the same.
At this point it is not at all clear what form such a system might take or how it would be determined whether a new system is superior to the current one. Certainly the use of modern computers to process results affords opportunities that did not exist when the ordinal system was developed. One change seems obvious however. To prevent skaters from swapping places as events unfold, judges will have to begin marking on an absolute scale. The question then becomes how should the judges absolute marks be combined to give the best answer and still maintain a high level of immunity to the effects of random errors and judges bias - and still be easily understandable. In a future article we will provide a mathematical comparison of several possible scoring systems and how they compare in accuracy and immunity to errors and bias.