[NSRCA-discussion] Fw: Judging . . . . . .

[Del]

Then it shouldn't be inferred or suggested that competitors don't understand "STATISTICAL ANALYSIS" . Statistically one could argue that more judges will make errors from 20 years ago because of the increased workload and difficulty catching ever downgrade when people are judging as many as 4 classes in one event. Granted that is rare but that also happens..  Some of the most capable judges used to avoid the chair just because of the demand could affect their flights coming up right after serving. 
 
    Del 
  ----- Original Message ----- 
  From: John Gayer 
  To: General pattern discussion 
  Sent: Friday, December 09, 2011 11:54 PM
  Subject: Re: [NSRCA-discussion] Fw: Judging . . . . . .


  As I understand it, TBL is applied to the raw score for each judge for the whole flight, not the maneuver, in an effort to correct for biases and bad judging. Of course if the valid zero still puts the "lone wolf zero judge" too far from the norm, then his scores will be discarded.
  However no one is going to come up with a system that corrects for judges that miss the extra spin or give a score when the wrong maneuver is flown, especially when they are in the majority. What if they all miss it and the pilot fesses up afterwards? How does a scoring system cope?  That is up to us as individual judges to fix, not the scoring system.

  John

  On 12/9/2011 4:35 PM, Del wrote: 
    Bob.. It is not so much that people don't have a solid understanding of STATISTICAL ANALYSIS as much as the problem of it happens more often than some are willing to admit or realize that a judge that award a zero can be the only one that was the accurate judge. The assumption being that if only one judge gave a zero then the rest must be correct and the lone wolf is the wrong one.  History has shown that repeatedly is not the case in actuality. Speaking from my own personal experience when judging both in Canada and the U.S. ~ yes `~ even at the Nats..   Shudder..!!! 

        Del
      Subject: [NSRCA-discussion] Fw: Judging . . . . . .


      Actually, no.

      The only statistically valid comparison is when a large enough number of judges are judging the same flight. Most local contests have only two judges per flight.  There is no way to know if one judge is doing a better job than the other because you don't have a large enough group to establish an average score.

      The only thing you may see are "high" and "low" judges, but that is not important unless there is bias, which falls into my first statement.  

      The NATS comes close to having large enough judging groups.  But, at least up until now, no one has been willing to propose using any statisical analysis on the judges performance. 

      The Worlds uses TBL to analyze judging performance and exclude "bad" scores. It is contriversial, since most people don't have a solid understanding of the statistical analysis used.  The raw scores are posted during the preliminaries, and everyone uses those to determine placement.  But when TBL is applied, it can alter the pilot positions by tossing out scores that are ruled statistically invalid.    

      Bob Kane
      getterflash at yahoo.com

      From: Peter Vogel <vogel.peter at gmail.com>
      To: Bob Kane <getterflash at yahoo.com>; General pattern discussion <nsrca-discussion at lists.nsrca.org> 
      Cc: nsrca <nsrca-discussion at lists.nsrca.org> 
      Sent: Friday, December 9, 2011 10:40 AM
      Subject: Re: [NSRCA-discussion] Judging . . . . . .


      Doesn't having a scoring system that allows for aggregated data analysis help with judging the judges over time?


      Peter+

      Sent from my iPhone4S

      On Dec 9, 2011, at 7:04 AM, Bob Kane <getterflash at yahoo.com> wrote:


        Personally, I think we need to continue to focus of the quality of judging before we get too distracted on automated score entry.



        2 - Know the rules
        3 - Apply the rules
        4 - Be consistent


        Bob Kane
        getterflash at yahoo.com
        _______________________________________________
        NSRCA-discussion mailing list
        NSRCA-discussion at lists.nsrca.org
        http://lists.nsrca.org/mailman/listinfo/nsrca-discussion








--------------------------------------------------------------------------
      _______________________________________________
      NSRCA-discussion mailing list
      NSRCA-discussion at lists.nsrca.org
      http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

     

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

------------------------------------------------------------------------------


  _______________________________________________
  NSRCA-discussion mailing list
  NSRCA-discussion at lists.nsrca.org
  http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20111210/944d4bd2/attachment.html>