[NSRCA-discussion] Fw: Judging . . . . . .

[Del]

Bob.. It is not so much that people don't have a solid understanding of STATISTICAL ANALYSIS as much as the problem of it happens more often than some are willing to admit or realize that a judge that award a zero can be the only one that was the accurate judge. The assumption being that if only one judge gave a zero then the rest must be correct and the lone wolf is the wrong one.  History has shown that repeatedly is not the case in actuality. Speaking from my own personal experience when judging both in Canada and the U.S. ~ yes `~ even at the Nats..   Shudder..!!! 
 
    Del
  Subject: [NSRCA-discussion] Fw: Judging . . . . . .


  Actually, no.

  The only statistically valid comparison is when a large enough number of judges are judging the same flight. Most local contests have only two judges per flight.  There is no way to know if one judge is doing a better job than the other because you don't have a large enough group to establish an average score.

  The only thing you may see are "high" and "low" judges, but that is not important unless there is bias, which falls into my first statement.  

  The NATS comes close to having large enough judging groups.  But, at least up until now, no one has been willing to propose using any statisical analysis on the judges performance. 

  The Worlds uses TBL to analyze judging performance and exclude "bad" scores. It is contriversial, since most people don't have a solid understanding of the statistical analysis used.  The raw scores are posted during the preliminaries, and everyone uses those to determine placement.  But when TBL is applied, it can alter the pilot positions by tossing out scores that are ruled statistically invalid.    

  Bob Kane
  getterflash at yahoo.com

  From: Peter Vogel <vogel.peter at gmail.com>
  To: Bob Kane <getterflash at yahoo.com>; General pattern discussion <nsrca-discussion at lists.nsrca.org> 
  Cc: nsrca <nsrca-discussion at lists.nsrca.org> 
  Sent: Friday, December 9, 2011 10:40 AM
  Subject: Re: [NSRCA-discussion] Judging . . . . . .


  Doesn't having a scoring system that allows for aggregated data analysis help with judging the judges over time?


  Peter+

  Sent from my iPhone4S

  On Dec 9, 2011, at 7:04 AM, Bob Kane <getterflash at yahoo.com> wrote:


    Personally, I think we need to continue to focus of the quality of judging before we get too distracted on automated score entry.



    2 - Know the rules
    3 - Apply the rules
    4 - Be consistent


    Bob Kane
    getterflash at yahoo.com
    _______________________________________________
    NSRCA-discussion mailing list
    NSRCA-discussion at lists.nsrca.org
    http://lists.nsrca.org/mailman/listinfo/nsrca-discussion








------------------------------------------------------------------------------


  _______________________________________________
  NSRCA-discussion mailing list
  NSRCA-discussion at lists.nsrca.org
  http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20111209/0a118838/attachment.html>