[NSRCA-discussion] Judging by committee?
seefo at san.rr.com
seefo at san.rr.com
Fri Feb 1 09:27:00 AKST 2008
I wasn't suggesting throwing out the high and low score per maneuver actually, but the high and low judge entirely. This of course requires you to have more judges to do so. I do understand your reasoning though of potentially eliminating the judge who actually got it right. One of things we used to say out here in pilot's meetings about judging is "Don't always assume the lower score is wrong."
I think the idea here is not to remove the stray 0 given, but remove any obvious bias against a specific pilot. Of course, I say that having said I'd trust anyone I fly against to judge me fairly, so perhaps that's a moot point.
One thing I hadn't thought about was the matter that each pilot would be flying in front of a different set of judges. I'm not sure I like that notion either.
The one thing Jason brought up just now too is the fact that we've all been the benefactor of the judges missing some big mistake from time to time. But strangely enough, when I got away with something big, nearly all of the pilots I was competing with caught it. I DO think the peer judging has merit. It just may be the negatives outweigh the positives.
-Doug
---- Mark Atwood <atwoodm at paragon-inc.com> wrote:
> I have to strongly agree with Del on this one. Dropping scores in this
> manner just isn¹t statistically valuable. You almost always just throw out
> an entire judge.
>
> I think those that favor this approach do so because of the stray ³0² that
> they¹ve received sometime in their pattern career, and see dropping highs
> (no one REALLY wants to throw out their high scores) and lows as a way to
> get rid of the Zero awarded by the local Snap Nazi. It¹s just the wrong
> approach.
>
> Maybe with 30 judges it would have some relevance, but not with 5 or even
> 10. AND...more often than not, I¹ve been MUCH more annoyed at the judge who
> MISSED a zero, than one that maybe awarded one unjustifiably.
>
> I¹d rather go the route of having a ³Zero² judge, who¹s sole job is to
> determine that the correct maneuver was flown and that a Snap was a snap and
> a spin a spin. The other judges simply judge what was flown... If a
> maneuver was missed or out of sequence, the zero judge overrides those
> scores. Ok...there¹s issues with that too (missing a point on a point roll
> for example would be an issue)...just thinking as I type.
>
> Anyhow...dropping highs and lows isn¹t the solution. (IMHO)
>
> -Mark
>
>
> On 2/1/08 11:40 AM, "Del Rykert" <drykert2 at rochester.rr.com> wrote:
>
> > I have always felt that throwing away a high and a low score not only insults
> > the judges but encourages judges to be number generators in the middle of the
> > road as they question giving a zero when they feel it was earned but knowing
> > it will be tossed if others didn't see the error.
> >
> > To punish a judge that does a better job is always going to be the wrong
> > approach. I have repeatedly seen where the judge giving the lower score as
> > being the problem. I can't ever recall personally seeing Santa clausing while
> > at a contest. If judging is a bad as some seem to think they why not have
> > judges certified yearly for all contests. At least then you will be using
> > people that have proven they knew the rules when they took their
> > certification.
> >
> > Del
> >
> > ----- Original Message -----
> > From: <seefo at san.rr.com <mailto:seefo at san.rr.com> >
> > To: "NSRCA Mailing List" <nsrca-discussion at lists.nsrca.org
> > <mailto:nsrca-discussion at lists.nsrca.org> >
> > Sent: Thursday, January 31, 2008 9:01 PM
> > Subject: Re: [NSRCA-discussion] Judging by committee?
> >
> >> > For as long as I've been competing there has always been the problem of
> >> judging being uninformed, inaccurate, etc. Not pointing fingers, it just is
> >> what it is whenever you have volunteer judging.
> >> >
> >> > Personally I think the efforts made by Jason and the Tangerine people
> >> should be applauded, REGARDLESS of whether or not the outcome is deemed
> >> valid. The fact is they tried something new to solve a problem. If it works,
> >> cool. If not, that's ok too. It just eliminates another incorrect choice at
> >> solving the problem. This sort of outside-the-box thinking should be
> >> encouraged.
> >> >
> >> > Having sat through MANY competitions where every one of my peers sees
> >> errors that few, if any judges on the line catch (a certain TOC pilot's wrong
> >> direction snap that only Peter Wessels caught comes to mind), I think the
> >> idea of peer judging has a great deal of merit, even if only as a case study
> >> comparison to standard judging methods. Who better to know what to look for
> >> in a flight than the people flying it? So long as there is a certain amount
> >> of normalization (such as dropping high & low scores) to reduce bias, I
> >> honestly can't think of a reason not to try it.
> >> >
> >> > To a person, I trust everyone I know at this level of competition to judge
> >> accurately and fairly, regardless of them being my competitor or not.
> >> >
> >> > -Doug Cronkhite
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > ---- Ken Thompson <kthompson at stx.rr.com <mailto:kthompson at stx.rr.com> >
> >> wrote:
> >>> >> I'm sorry Jason,
> >>> >>
> >>> >> But my comments have little to do with my respect for you, which I
> >>> obviously have, it's more that I have a much different opinion with respect
> >>> with the comments by Cameron Smith.
> >>> >> Cameron...your reputation precedes you...if you would like to take this
> >>> off line, I'd be happy to do so. You are opinionated, that's no problem, I
> >>> am also, but to call someone out in our forum, that's not right...So to
> >>> quote you...I'm here to say things honestly!!!
> >>> >> Let's both say what's on our mind...to quote you..."nice to see someone
> >>> with a set" I have mine, how 'bout you?
> >>> >>
> >>> >> Ken Thompson
> >>> >> ----- Original Message -----
> >>> >> From: JShulman
> >>> >> To: NSRCA Mailing List
> >>> >> Sent: Thursday, January 31, 2008 7:12 PM
> >>> >> Subject: Re: [NSRCA-discussion] Judging by committee?
> >>> >>
> >>> >>
> >>> >> Hey guys, I don't mind Cameron's comments. I haven't exactly had much
> >>> sleep today, or I should say this morning, so I might have come off a bit
> >>> different than usual. I accept his opinion as being his, and welcome his
> >>> comments on how to help to improve judging. Not just for FAI, but for all of
> >>> pattern.
> >>> >>
> >>> >> Regards,
> >>> >> Jason
> >>> >> www.jasonshulman.com <http://www.jasonshulman.com>
> >>> >> www.shulmanaviation.com <http://www.shulmanaviation.com>
> >>> >> www.composite-arf.com <http://www.composite-arf.com>
> >>> >>
> >>> >> -----Original Message-----
> >>> >> From: nsrca-discussion-bounces at lists.nsrca.org
> >>> <mailto:nsrca-discussion-bounces at lists.nsrca.org>
> >>> [mailto:nsrca-discussion-bounces at lists.nsrca.org]On Behalf Of Ken Thompson
> >>> >> Sent: Thursday, January 31, 2008 8:06 PM
> >>> >> To: NSRCA Mailing List
> >>> >> Subject: Re: [NSRCA-discussion] Judgeing by commitee?
> >>> >>
> >>> >>
> >>> >> This has got to me a freakin' joke...I haven't been posting much, but
> >>> things like this will make me come back to life!!!
> >>> >>
> >>> >> I don't believe the majority would want me to get back on the band
> >>> wagon with my opinions...this kind of post is pushing me in that
> >>> direction...
> >>> >>
> >>> >> Ken Thompson
> >>> >> NSRCA 3646
> >>> >> AMA 685343
> >>> >> ----- Original Message -----
> >>> >> From: C. Smith
> >>> >> To: 'NSRCA Mailing List'
> >>> >> Sent: Thursday, January 31, 2008 6:48 PM
> >>> >> Subject: Re: [NSRCA-discussion] Judgeing by commitee?
> >>> >>
> >>> >>
> >>> >> Jim,
> >>> >>
> >>> >> I just got home, Sat down to eat my dinner & read the emails...
> >>> >>
> >>> >> When you read this to me this afternoon it did not make the
> >>> impression it did when I read it.....
> >>> >>
> >>> >> I am not trying to blow smoke up your skirt please believe me...
> >>> >>
> >>> >> It is refreshing to see someone say things honestly & in such a
> >>> way.... Too few say what needs to be said.....
> >>> >>
> >>> >>
> >>> >>
> >>> >> Jason's response came across as self serving damage control.. I
> >>> do not think he comes across as a sincere person. As a matter of fact he
> >>> seems very insecure......
> >>> >>
> >>> >> He came back at you by saying 3 of the 4 believe it WILL work..
> >>> But I could have sworn you showed me emails from Ryan disagreeing with the
> >>> practice..... I would bet you Joe & Ryan in public would agree with Jason
> >>> just to associate them selves with him.. I call this by many names....Weak,
> >>> Intellectually Dishonest & self serving..
> >>> >>
> >>> >> Oh well I just want you to know Good job-- Nice to see someone
> >>> with a set.
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> --------------------------------------------------------------------------
> >>> >> From: nsrca-discussion-bounces at lists.nsrca.org
> >>> <mailto:nsrca-discussion-bounces at lists.nsrca.org>
> >>> [mailto:nsrca-discussion-bounces at lists.nsrca.org] On Behalf Of Woodward, Jim
> >>> >> Sent: Thursday, January 31, 2008 2:47 PM
> >>> >> To: NSRCA Mailing List
> >>> >> Subject: Re: [NSRCA-discussion] Judgeing by commitee?
> >>> >>
> >>> >>
> >>> >> Hey Anthony,
> >>> >>
> >>> >>
> >>> >> **** Attempting a 50 words or less approach without too much regard
> >>> for political correctness *****
> >>> >>
> >>> >>
> >>> >> I don't think peer judging works. I don't think it sends the right
> >>> message about problem solving or achieving a more accurate score per
> >>> maneuver for each pilot. Psychology 101 would predict that it does not
> >>> foster the right mindset or circumstances for a competitive environment
> >>> (Reality TV shows like Survivor are based on one form or another of peer
> >>> judging).
> >>> >>
> >>> >>
> >>> >> The #1 component that must be correct for it to work is that all
> >>> pilot/judges see and subtract about the exact same number of points per
> >>> maneuver see the same downgrades. The situation doesn't compute if one
> >>> judge is off from the others or uses impression judging. A bunch of stuff
> >>> should probably be in place for this to work like: large number of judges,
> >>> drop high score, drop low score, etc. The highest caliber of honor,
> >>> integrity, and judge-education is required by all competitors to make this
> >>> work.
> >>> >>
> >>> >>
> >>> >> I witnessed this as a Masters pilot watching the FAI contest. I
> >>> watched the flying and this scenario VERY close. My opinion is that I would
> >>> chose not to compete in FAI in a peer judging scenario.
> >>> >>
> >>> >>
> >>> >> Thanks,
> >>> >>
> >>> >> Jim W.
> >>> >>
> >>> >>
> >>> >>
> >>> >> CONFIDENTIALITY NOTICE: This e-mail message, including any
> >>> attachments, is for the sole use of the intended recipient(s) and may
> >>> contain confidential and proprietary information. Any unauthorized review,
> >>> use, disclosure or distribution is prohibited. If you are not the intended
> >>> recipient(s), please contact the sender by reply e-mail and destroy all
> >>> copies of the original message.
> >>> >>
> >>> >>
> >>> >>
> >>> --------------------------------------------------------------------------
> >>> >>
> >>> >> From: nsrca-discussion-bounces at lists.f3a.us
> >>> <mailto:nsrca-discussion-bounces at lists.f3a.us>
> >>> [mailto:nsrca-discussion-bounces at lists.f3a.us] On Behalf Of Anthony Romano
> >>> >> Sent: Thursday, January 31, 2008 1:44 PM
> >>> >> To: nsrca-discussion at lists.f3a.us
> >>> <mailto:nsrca-discussion at lists.f3a.us>
> >>> >> Subject: [NSRCA-discussion] Judgeing by commitee?
> >>> >>
> >>> >>
> >>> >> Finally got a chance to read the current K-factor and saw a note on
> >>> the Tangerine contest. The article mentioned FAI was judged by a commity of
> >>> the FAI pilots. Could someone please provide details. Do you think you could
> >>> keep your objectivity? For those that were there how did it work out? Sound
> >>> interesting because you would finally be judged by pilots who know the FAI
> >>> rules and the sequence.
> >>> >> Could this be a solution for the oversized Masters class? Obvious
> >>> drawbacks too, but trying to inspire some thought.
> >>> >>
> >>> >> Anthony
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> --------------------------------------------------------------------------
> >>> >>
> >>> >> Helping your favorite cause is as easy as instant messaging. You
> >>> IM, we give. Learn more.
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> --------------------------------------------------------------------------
> >>> >>
> >>> >>
> >>> >> _______________________________________________
> >>> >> NSRCA-discussion mailing list
> >>> >> NSRCA-discussion at lists.nsrca.org
> >>> <mailto:NSRCA-discussion at lists.nsrca.org>
> >>> >> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
> >>> <http://lists.nsrca.org/mailman/listinfo/nsrca-discussion>
> >>> >>
> >>> >>
> >>> >>
> >>>
> ------------------------------------------------------------------------------
> >>> >>
> >>> >>
> >>> >> _______________________________________________
> >>> >> NSRCA-discussion mailing list
> >>> >> NSRCA-discussion at lists.nsrca.org
> >>> <mailto:NSRCA-discussion at lists.nsrca.org>
> >>> >> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
> >>> <http://lists.nsrca.org/mailman/listinfo/nsrca-discussion>
> >> >
> >> > _______________________________________________
> >> > NSRCA-discussion mailing list
> >> > NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
> >> > http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
> >> <http://lists.nsrca.org/mailman/listinfo/nsrca-discussion>
> >
> > _______________________________________________
> > NSRCA-discussion mailing list
> > NSRCA-discussion at lists.nsrca.org
> > http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
>
>
More information about the NSRCA-discussion
mailing list