[NSRCA-discussion] Fw: Judging . . . . . .

[Vicente "Vince" Bortone]

The only way is having zero judges.  We cannot afford that.  In AMA the zero counts so we are good.  I am not sure how it works now for FAI using the TBL method.    

Vicente "Vince" Bortone
----- Original Message -----
From: John Gayer <jgghome at comcast.net>
To: General pattern discussion <nsrca-discussion at lists.nsrca.org>
Sent: Sat, 10 Dec 2011 04:54:13 -0000 (UTC)
Subject: Re: [NSRCA-discussion] Fw: Judging . . . . . .
As I understand it, TBL is applied to the raw score for each judge for 
the whole flight, not the maneuver, in an effort to correct for biases 
and bad judging. Of course if the valid zero still puts the "lone wolf 
zero judge" too far from the norm, then his scores /will/ be discarded.
However no one is going to come up with a system that corrects for 
judges that miss the extra spin or give a score when the wrong maneuver 
is flown, especially when they are in the majority. What if they all 
miss it and the pilot fesses up afterwards? How does a scoring system 
cope? That is up to us as individual judges to fix, not the scoring system.
John
On 12/9/2011 4:35 PM, Del wrote:
> *Bob.. It is not so much that people don't have a solid understanding 
> of STATISTICAL ANALYSIS as much as the problem of it happens more 
> often than some are willing to admit or realize that a judge that 
> award a zero can be the only one that was the accurate judge. The 
> assumption being that if only one judge gave a zero then the rest must 
> be correct and the lone wolf is the wrong one. History has shown that 
> repeatedly is not the case in actuality. Speaking from my own personal 
> experience when judging both in Canada and the U.S. ~ yes `~ even at 
> the Nats.. Shudder..!!! *
> **
> * Del*
>
> *Subject:* [NSRCA-discussion] Fw: Judging . . . . . .
>
> Actually, no.
> The only statistically valid comparison is when a large enough
> number of judges are judging the same flight. Most local contests
> have only two judges per flight. There is no way to know if one
> judge is doing a better job than the other because you don't have
> a large enough group to establish an average score.
> The only thing you may see are "high" and "low" judges, but that
> is not important unless there is bias, which falls into my first
> statement.
> The NATS comes close to having large enough judging groups. But,
> at least up until now, no one has been willing to propose using
> any statisical analysis on the judges performance.
> The Worlds uses TBL to analyze judging performance and exclude
> "bad" scores. It is contriversial, since most people don't have a
> solid understanding of the statistical analysis used. The raw
> scores are posted during the preliminaries, and everyone uses
> those to determine placement. But when TBL is applied, it can
> alter the pilot positions by tossing out scores that are ruled
> statistically invalid.
> Bob Kane
> getterflash at yahoo.com
> *From:* Peter Vogel <vogel.peter at gmail.com>
> *To:* Bob Kane <getterflash at yahoo.com>; General pattern discussion
> <nsrca-discussion at lists.nsrca.org>
> *Cc:* nsrca <nsrca-discussion at lists.nsrca.org>
> *Sent:* Friday, December 9, 2011 10:40 AM
> *Subject:* Re: [NSRCA-discussion] Judging . . . . . .
>
> Doesn't having a scoring system that allows for aggregated data
> analysis help with judging the judges over time?
>
> Peter+
>
> Sent from my iPhone4S
>
> On Dec 9, 2011, at 7:04 AM, Bob Kane <getterflash at yahoo.com
> <mailto:getterflash at yahoo.com>> wrote:
>
>> Personally, I think we need to continue to focus of the quality
>> of judging before we get too distracted on automated score entry.
>>
>> 2 - Know the rules
>> 3 - Apply the rules
>> 4 - Be consistent
>> Bob Kane
>> getterflash at yahoo.com <mailto:getterflash at yahoo.com>
>> _______________________________________________
>> NSRCA-discussion mailing list
>> NSRCA-discussion at lists.nsrca.org
>> <mailto:NSRCA-discussion at lists.nsrca.org>
>> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
>
>
>
>
> ------------------------------------------------------------------------
> _______________________________________________
> NSRCA-discussion mailing list
> NSRCA-discussion at lists.nsrca.org
> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
>
>
>
> _______________________________________________
> NSRCA-discussion mailing list
> NSRCA-discussion at lists.nsrca.org
> http://lists.nsrca.org/mailman/listinfo/nsrca-discussion