As has been described previously, the USFSA has become nearly exclusively results oriented in recent years. In May, the rules of the International Committee were completely rewritten to facilitate this focus, to consolidate USFSA efforts to bring home as many medals as possible, and to meet quotas set by the USOC. To assist in selecting a World Team most likely to win the greatest number of medals, the International Committee has implemented a ranking system to aid in its deliberations.
The use of a ranking system has been under consideration by the Committee for several years. One system studied three years ago consisted of combining the judges' marks from international competitions in a given season to predict how skaters would place at the subsequent World Championships. This approach appeared to have some promise, but the study was canceled and the method was never adopted.
Subsequently a second method was developed, and in May put into use. This method is a point based system which has some similarities to the ISIO rankings, but also a number of differences that lead to some odd results. The May rankings, which we obtained in June include the following results, all of which are patently absurd in light of results this past season:
There are some who would argue that a point based ranking system will always lead to bogus results and should never be used. Making use of a point based system ourselves, however, we would not go that far; but having devoted considerable effort to developing the ISIO point model and having extensively compared it to several other ranking methods, we are painfully aware that point based methods must be carefully structured and their point models precisely tuned to give reasonable results. It is clear from the above results that the International Committee rankings tell more about the failings of the method than they do about the skills of the skaters. While at one level this is an interesting academic exercise, the idea that such an obviously flawed method would even be considered for use in selecting who the US will allow to compete on an international level is troublesome.
The Committee's rankings make use of international competitions, and the National, Sectional, and Regional Championships at the Novice through Senior levels. Five competition categories are defined, with international and domestic competition results intermixed. Each competition category has a weighting factor assigned, subdivided by competition level (i.e., junior vs. senior, etc.). Point totals are assigned based on placement. The method does not take into account differences in activity (number of competitions entered), the size of the competition entered, or the skill levels of the actual competitors in a given competition. The point model used is primarily a linear model which appears somewhat arbitrary in its details.
There are, of course, several factors that play into the decision making process of selecting the World Team members, the most important of which is the National Championships. Nevertheless it is troubling that such a flawed ranking system should be made a part of the process at all due to the skewed impact it may have on competitive careers, and team choices. As examples, we note the following characteristics of the ranking system:
While the idea of using a ranking system to assist in the selection of international team members has some merit, the current system leaves so much to be desired one can only hope that it is either replaced, extensively modified, or ignored before too much damage is done - with the first of the three choices the most desirable.