[NSRCA-discussion] Judging by committee?
Dave Burton
burtona at atmc.net
Thu Jan 31 14:52:38 AKST 2008
There is a way to solve the peer judging and several other problems with
changing maneuver schedules for Master's class.
Let Masters class fly the most current FAI P schedule as a separate class.
This provides a way that FAI class can judge Masters and be completely
familiar with the maneuvers and Masters class can judge FAI and be
completely familiar with the schedule. Then the rules committee does not
have to come up with a new schedule periodically as it changes every other
year just like FAI. The schedules (P & Masters) are so close in difficulty
that flying the P schedule should not be any problem for masters class
flyers.
OK, Flame suit on!
From: nsrca-discussion-bounces at lists.nsrca.org
[mailto:nsrca-discussion-bounces at lists.nsrca.org] On Behalf Of Mark Atwood
Sent: Thursday, January 31, 2008 3:56 PM
To: NSRCA Mailing List
Subject: Re: [NSRCA-discussion] Judgeing by commitee?
For our "matrix" version, the A& B masters groups, we effectively ran 2
contests. The scorer set up a second masters only contest for the B panel
to enter their scores. It worked quite well with only a little confusion.
It did a great job of picking the top 5 guys and getting them into the top
8. I'm pretty sure you could argue that 7-12th place might have had some
variance...but I think that's true regardless of the format.
-Mark
On 1/31/08 3:49 PM, "Anthony Romano" <anthonyr105 at hotmail.com> wrote:
I suspected this would require super- human objectivity as well as be a
logistical nightmare. However, everyone reall knows the sequence. Really
like the matrix system but not sure how much work that makes for the scorer.
Anyone have any thoughts on how to score that
One idea that was kicked around in D1 was fly an extra round in Masters to
generate an extra throw away. Each round two masters pilots judge and don't
fly rotating through the entire class. It seems like the time required would
work out the same because the group had two less pilots but again lot of
objectivity ( conscious and unconscious ) required especially as the contest
end grew near.
Anthony
_____
Date: Thu, 31 Jan 2008 15:14:15 -0500
From: atwoodm at paragon-inc.com
To: nsrca-discussion at lists.nsrca.org; nsrca-discussion at lists.f3a.us
Subject: Re: [NSRCA-discussion] Judgeing by commitee?
Anthony,
I have to agree with Jim, but for different reasons. We did this about 4
years back at our district championships with the masters class. We had 17
pilots in masters, and only one (me) in FAI, and another 6 or 7 in advanced.
So getting any judging at all would have required heavily using the
Intermediate and Sportsman classes to judge, OR, heavily burdening the few
Advanced guys...and sitting through 17 masters flights is a looooong
sentence.
So we did the peer judging scenario. Given the options, it worked very
well. But it requires some serious juggling to even try and make it work
well. We used peer judging for 4 of the 6 rounds. Two flight lines, with a
rolling panel of judges. 5 judges on each line, tossed high and low by
maneuver leaving 7 pilots not judging at any given time. This allowed the
person before and after each flight some time to prep and decompress before
having to jump in the chair for 5 flights and then start over on the second
line.
It's a VERY VERY VERY busy process, not to mention that unless you
completely randomly resort the flight line each round, the pilot will be
judged but the same group...or maybe more importantly NOT judged by the same
group each round.
It worked...but it was messy. I would only do it again if we were presented
with the same grossly offset numbers of entries.
On a related note... A better solution was tried a few years later when we
had similar numbers (16 masters pilots)
We created 2 classes of masters...A and B. we still used FAI and Advanced
judges, but we were also able to sprinkle in B judges for A and vice versa.
We did 4 rounds for each group. Took the top 4 from each group and combined
them and they flew the last 2 rounds as a "Finalists" group (with the other
8 judging and flying in their own group for the bottom 8 spots.)
This was MUCH more workable, and I think netted a fairer event in the long
run.
-Mark
On 1/31/08 2:46 PM, "Woodward, Jim" <jim.woodward at baesystems.com> wrote:
Hey Anthony,
**** Attempting a 50 words or less approach without too much regard for
political correctness *****
I don't think peer judging works. I don't think it sends the right message
about problem solving or achieving a more accurate score per maneuver for
each pilot. Psychology 101 would predict that it does not foster the right
mindset or circumstances for a competitive environment (Reality TV shows
like Survivor are based on one form or another of peer judging).
The #1 component that must be correct for it to work is that all
pilot/judges see and subtract about the exact same number of points per
maneuver see the same downgrades. The situation doesn't compute if one
judge is off from the others or uses impression judging. A bunch of stuff
should probably be in place for this to work like: large number of judges,
drop high score, drop low score, etc. The highest caliber of honor,
integrity, and judge-education is required by all competitors to make this
work.
I witnessed this as a Masters pilot watching the FAI contest. I watched the
flying and this scenario VERY close. My opinion is that I would chose not to
compete in FAI in a peer judging scenario.
Thanks,
Jim W.
CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and proprietary information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient(s),
please contact the sender by reply e-mail and destroy all copies of the
original message.
_____
From: nsrca-discussion-bounces at lists.f3a.us
[mailto:nsrca-discussion-bounces at lists.f3a.us] On Behalf Of Anthony Romano
Sent: Thursday, January 31, 2008 1:44 PM
To: nsrca-discussion at lists.f3a.us
Subject: [NSRCA-discussion] Judgeing by commitee?
Finally got a chance to read the current K-factor and saw a note on the
Tangerine contest. The article mentioned FAI was judged by a commity of the
FAI pilots. Could someone please provide details. Do you think you could
keep your objectivity? For those that were there how did it work out? Sound
interesting because you would finally be judged by pilots who know the FAI
rules and the sequence.
Could this be a solution for the oversized Masters class? Obvious drawbacks
too, but trying to inspire some thought.
Anthony
_____
Helping your favorite cause is as easy as instant messaging. You IM, we
give. Learn more.
<http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
<http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
_____
_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
_____
Helping your favorite cause is as easy as instant messaging. You IM, we
give. Learn more.
<http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
<http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
_____
_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20080131/b41091d0/attachment-0001.html
More information about the NSRCA-discussion
mailing list