Saturday, July 12, 2008

"Judge Not, Lest Ye Be Judged"

So says the Good Book. And it is ever so apparent when a car show is judged by good folks with good intentions, but a flawed system. The participants end up judging the judges, as is the case with the Belgrade show. Were they "experts" in judging on a points system? Perhaps, but only a handful of people in New England can lay claim to that, so it is not likely. Were they impartial? If they had friends in the show, human nature would dictate that there was probably at least some bias, intentional or not.

Judging is one of the most critical parts of any car show. As discussed in previous posts, there are generally two forms of judging:

1) a JUDGED show, judged by impartial, expert judges on a points basis, and
2) a PEER JUDGED show, judged by the participants.

In a judged show, the judges absolutely must be impartial. Preferably, they would be brought in from outside, so they have no "horse in the race." They would be experts in using the points system. And they would really know cars, inside and out. From what I can gather, most judges in a judged show in these parts are not even acquainted with the points system - they make up their own. Here is a typical sheet from the Rules & Standards for a judged, points system show.

If a show organizer cannot find qualified, expert and impartial judges for their show, then they should really consider going to a peer judged show. But even a peer judged show has the potential for unfairness. So, to reduce the potential for participants to use the "buddy system" of voting (I'll vote for yours if you vote for mine), the "limited ballot" method seems to work fairly well.

Here's how the limited ballot works:

Let's say you have 18 classes. You would print up ballots, each with only (6) classes, chosen at random. When a participant registers, he is given a ballot that does not include his own class. Since there is much less chance that his ballot will include the class(es) of his best buds, it is less likely that the "buddy system" of voting would carry much effect. Why? Because even if his "best bud's" class is on his ballot, it is unlikely that he would also be on his buddy's ballot.

A number of good shows use this system effectively, and the participants generally have no gripe with it, because it is more fair than "normal" peer judging, and more fair than a judged show whose judges know the participants, and where bias or friendship could affect voting.

Of course, there is also the balloting system where only the spectators vote. This is the least fair of all, since most spectators know nothing about cars. But it does have its place - it is the best method for choosing "People's Choice".

No comments: