[NSRCA-discussion] Judgeing by commitee?

Thu Jan 31 11:56:02 AKST 2008

For our ³matrix² version, the A& B masters groups, we effectively ran 2
contests.  The scorer set up a second masters only contest for the B panel
to enter their scores.  It worked quite well with only a little confusion.

It did a great job of picking the top 5 guys and getting them into the top
8.  I¹m pretty sure you could argue that 7-12th place might have had some
variance...but I think that¹s true regardless of the format.

-Mark

On 1/31/08 3:49 PM, "Anthony Romano" <anthonyr105 at hotmail.com> wrote:

> 
>   I suspected this would require super- human objectivity as well as be a
> logistical nightmare. However, everyone reall knows the sequence. Really like
> the matrix system but not sure how much work that makes for the scorer. Anyone
> have any thoughts on how to score that
>   One idea that was kicked around in D1 was fly an extra round in Masters to
> generate an extra throw away. Each round two masters pilots judge and don't
> fly rotating through the entire class. It seems like the time required would
> work out the same because the group had two less pilots but again lot of
> objectivity ( conscious and unconscious ) required especially as the contest
> end grew near. 
>  
> Anthony
> 
>> 
>> Date: Thu, 31 Jan 2008 15:14:15 -0500
>> From: atwoodm at paragon-inc.com
>> To: nsrca-discussion at lists.nsrca.org; nsrca-discussion at lists.f3a.us
>> Subject: Re: [NSRCA-discussion] Judgeing by commitee?
>> 
>> Anthony,
>> 
>> I have to agree with Jim, but for different reasons.  We did this about 4
>> years back at our district championships with the masters class.  We had 17
>> pilots in masters, and only one (me) in FAI, and another 6 or 7 in advanced.
>> So getting any judging at all would have required heavily using the
>> Intermediate and Sportsman classes to judge, OR, heavily burdening the few
>> Advanced guys...and sitting through 17 masters flights is a looooong
>> sentence.
>> 
>> So we did the peer judging scenario.  Given the options, it worked very well.
>> But it requires some serious juggling to even try and make it work well.  We
>> used peer judging for 4 of the 6 rounds.  Two flight lines, with a rolling
>> panel of judges.  5 judges on each line, tossed high and low by maneuver
>> leaving 7 pilots not judging at any given time.  This allowed the person
>> before and after each flight some time to prep and decompress before having
>> to jump in the chair for 5 flights and then start over on the second line.
>> 
>> It¹s a VERY VERY VERY busy process, not to mention that unless you completely
>> randomly resort the flight line each round, the pilot will be judged but the
>> same group...or maybe more importantly NOT judged by the same group each
>> round.
>> 
>> It worked...but it was messy.  I would only do it again if we were presented
>> with the same grossly offset numbers of entries.
>> 
>> On a related note... A better solution was tried a few years later when we
>> had similar numbers (16 masters pilots)
>> 
>> We created 2 classes of masters...A and B.   we still used FAI and Advanced
>> judges, but we were also able to sprinkle in B judges for A and vice versa.
>> We did 4 rounds for each group.  Took the top 4 from each group and combined
>> them and they flew the last 2 rounds as a ³Finalists² group (with the other 8
>> judging and flying in their own group for the bottom 8 spots.)
>> 
>>  This was MUCH more workable, and I think netted a fairer event in the long
>> run.
>> 
>> -Mark
>> 
>>   
>> 
>> 
>> On 1/31/08 2:46 PM, "Woodward, Jim" <jim.woodward at baesystems.com> wrote:
>> 
>>> Hey Anthony,
>>>  
>>> **** Attempting a 50 words or less approach without too much regard for
>>> political correctness *****
>>>  
>>> I don¹t think peer judging works.  I don¹t think it sends the right message
>>> about problem solving or achieving a more accurate score per maneuver for
>>> each pilot.   Psychology 101 would predict that it does not foster the right
>>> mindset or circumstances for a competitive environment (Reality TV shows
>>> like Survivor are based on one form or another of peer judging).
>>>  
>>> The #1 component that must be correct for it to work is that all
>>> pilot/judges see and subtract about the exact same number of points per
>>> maneuver see the same downgrades.  The situation doesn¹t compute if one
>>> judge is off from the others or uses impression judging.  A bunch of stuff
>>> should probably be in place for this to work like:  large number of judges,
>>> drop high score, drop low score, etc. The highest caliber of honor,
>>> integrity, and judge-education is required by all competitors to make this
>>> work.  
>>>  
>>> I witnessed this as a Masters pilot watching the FAI contest.  I watched the
>>> flying and this scenario VERY close. My opinion is that I would chose not to
>>> compete in FAI in a peer judging scenario.
>>>  
>>> Thanks,
>>> 
>>> Jim W.
>>>  
>>> 
>>>  
>>> CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is
>>> for the sole use of the intended recipient(s) and may contain confidential
>>> and proprietary information.  Any unauthorized review, use, disclosure or
>>> distribution is prohibited.  If you are not the intended recipient(s),
>>> please contact the sender by reply e-mail and destroy all copies of the
>>> original message.
>>> 
>>> 
>>> From: nsrca-discussion-bounces at lists.f3a.us
>>> [mailto:nsrca-discussion-bounces at lists.f3a.us] On Behalf Of Anthony Romano
>>> Sent: Thursday, January 31, 2008 1:44 PM
>>> To: nsrca-discussion at lists.f3a.us
>>> Subject: [NSRCA-discussion] Judgeing by commitee?
>>> 
>>> Finally got a chance to read the current K-factor and saw a note on the
>>> Tangerine contest. The article mentioned FAI was judged by a commity of the
>>> FAI pilots. Could someone please provide details. Do you think you could
>>> keep your objectivity? For those that were there how did it work out? Sound
>>> interesting because you would finally be judged by pilots who know the FAI
>>> rules and the sequence.
>>>  Could this be a solution for the oversized Masters class? Obvious drawbacks
>>> too, but trying to inspire some thought.
>>>  
>>> Anthony
>>>  
>>>  
>>>   
>>> 
>>> 
>>> 
>>> 
>>> Helping your favorite cause is as easy as instant messaging. You IM, we
>>> give. Learn more.
>>> <http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
>>> 
>>> 
>>> _______________________________________________
>>> NSRCA-discussion mailing list
>>> NSRCA-discussion at lists.nsrca.org
>>> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
>> 
> 
> 
> Helping your favorite cause is as easy as instant messaging. You IM, we give.
> Learn more. <http://im.live.com/Messenger/IM/Home/?source=text_hotmail_join>
> 
> _______________________________________________
> NSRCA-discussion mailing list
> NSRCA-discussion at lists.nsrca.org
> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20080131/84db2670/attachment-0001.html 