NSRCA Judge Ranking System

Tue Dec 14 09:43:32 AKST 2004

D5 votes yes on the move of Kansas from D6 to D5

Dan Curtis

----- Original Message ----- 
From: "Ron Van Putte" <vanputte at cox.net>
To: <discussion at nsrca.org>
Sent: December 14, 2004 12:19 PM
Subject: NSRCA Judge Ranking System


> 
> On Dec 14, 2004, at 10:25 AM, <rcaerobob at cox.net> wrote:
> 
> > BUT - if you correlated the placings of contestants against the 
> > judges' score to come up with a "validity coefficient", you could use 
> > THAT to rank a judge, as the coefficient would be a constant relative 
> > to that judges' performance with respect to the outcome of an event.
> 
> The NSRCA judge ranking system does this.  It is assumed that the 
> consensus rankings are correct and compares the judge's ranking 
> relative to the consensus.  There are those who will argue that using 
> the judging consensus  as a standard is not accurate.  However, that's 
> exactly how we award trophies at a contest.  BTW, it should be obvious 
> that the fewer the number of contestants, the less accurate the 
> consensus ranking will be.  That's why we required that at least five 
> judges must be on the line.  This is why Nats Finals and F3A Team 
> Selections are the only data we used.
> 
> The NSRCA judge ranking system also evaluates how a judge's scores for 
> individual contestants compares to the consensus scores.  A particular 
> judge's scoring ' errors' for all pilots are combined for a total 
> scoring 'error'.
> 
> The two factors are combined for a ranking score.
> 
> I hope this brief explanation helps clarify the situation for those who 
> did not read my earlier explanations in the K-Factor over the past two 
> years.
> 
> Ron Van Putte
> 
> >> From: Bob Kane <getterflash at yahoo.com>
> >> Date: 2004/12/14 Tue AM 11:18:53 EST
> >> To: discussion at nsrca.org
> >> Subject: Re: FW: Motion to adopt.
> >>
> >> You are correct, the opportunities to be ranked as a
> >> judge are few and far between, but they are the only
> >> statistically valid comparisons. I am hoping someone
> >> who is a statistician will jump in and offer the
> >> mathmatical proof, I know enough to be dangerous. I
> >> wish it were possible to enter judge's scores from
> >> every contest into some kind of database to come up
> >> with a performance ranking. Unfortunately, it is
> >> statistically invalid since they are judging different
> >> events at different times with no correlation between
> >> them. The only thing you could compute would be an
> >> average awarded score and a range for each judge which
> >> while interesting, means nothing about absolute
> >> performance.
> >>
> >> --- vicenterc at comcast.net wrote:
> >>
> >>> Hi Tony:
> >>>
> >>> The problem is that the judges that have the
> >>> opportunity to participate in 5 judges event is very
> >>> low.  Are you sure that the data even with two
> >>> judges is not significant?  I will agree that with
> >>> the 5 judges the data will be more significant using
> >>> using two contest only.  With two judges doing all
> >>> country with several contest per judge will become
> >>> significant data also.  At least will be more
> >>> significant than the method is used now.  The
> >>> results from the previous years Nats. should be also
> >>> used since this is three judge event.  In this way,
> >>> NSRCA will give more opportunity to all certified
> >>> judges and the process will be transparent and clear
> >>> to all.  There are other benefits, yes we love
> >>> competition and this will be a competition to become
> >>> better judges.  The end results is that NSRCA will
> >>> help improve not only the pilots skills but also the
> >>> judging skills which is very important part of our
> >>> sport.
> >>>
> >>> Best regards,
> >>>
> >>> Vince Bortone
> >>>
> >>> -------------- Original message --------------
> >>>
> >>> Chris/Vince:
> >>>
> >>> The judging program that the Team created does just
> >>> that!  You can see the standings on the NSRCA
> >>> website.  The standings are created from the data of
> >>> contests where we use a minimum of 5 judges per
> >>> flight.  This is only done (to my knowledge) at the
> >>> US Nats FINALS, and the Team Selection Contest.  The
> >>> requirement of 5 judges in essential to get valid
> >>> statistics.
> >>>
> >>> Tony Stillman
> >>> Radio South
> >>> 3702 N. Pace Blvd.
> >>> Pensacola, FL 32505
> >>> 1-800-962-7802
> >>> www.radiosouthrc.com
> >>> ----- Original Message -----
> >>> From: Chris Larson
> >>> To: discussion at nsrca.org
> >>> Sent: Monday, December 13, 2004 10:32 PM
> >>> Subject: RE: FW: Motion to adopt.
> >>>
> >>>
> >>> I agree Vince.
> >>>
> >>> HUGE undertaking, but the results would be
> >>> fantastic.  Imagine the ability to compare yourself
> >>> to all judges across the country.  Now that's
> >>> motivation to better yourself at your craft.
> >>>
> >>> Hmmm..... At the same time, how about a national
> >>> PILOT ranking system, similar to the one used in the
> >>> USTA  ( Tennis ).  I believe they know their rank in
> >>> their given class on a national basis.  Might be
> >>> kind of fun, and good for us guys who can't travel
> >>> to the Nats!  We could still be "competing" with
> >>> guys from the other districts!
> >>>
> >>> Publish the top judges and the top competitors in
> >>> each class in the K-Factor.
> >>>
> >>> Chris Larson
> >>> L & D Sales
> >>> 209-274-2176 Office / Fax
> >>> 209-304-0865 Cellular
> >>> -----Original Message-----
> >>> From: discussion-request at nsrca.org
> >>> [mailto:discussion-request at nsrca.org]On Behalf Of
> >>> vicenterc at comcast.net
> >>> Sent: Monday, December 13, 2004 7:49 PM
> >>> To: discussion at nsrca.org; NSRCA List
> >>> Cc: Derek Koopowitz
> >>> Subject: Re: FW: Motion to adopt.
> >>>
> >>>
> >>> I am firm believer of statistics. I have not chance
> >>> to read the program proposed but I am sure that
> >>> should be based on statistical formulation.  In
> >>> statistics, more data is compiled more valid will be
> >>> the results and conclusions obtained from that
> >>> analysis.  I am not am expert but I would like to
> >>> see in the future a process of judge selection that
> >>> is based on all information compiled in ALL contests
> >>> around the nation.  Yes, I am proposing that all
> >>> data be compiled and used to determine the best
> >>> judges.  I don't think that this is very difficult
> >>> task since the data is already available in
> >>> electronic format.  We will need to make sure that
> >>> the CD sends the information to the NSRCA.  I know
> >>> that there will be many happy faces if some of the
> >>> new candidates support this effort.
> >>>  Mortal guys like me will say that this is
> >>> impossible.  However, this is very easy task to
> >>> someone expert in databases and computer
> >>> programming.  We just need to make sure that all
> >>> scoring programs produce the data in the format
> >>> required.  That program will use the judge NSRCA
> >>> number to send the information to be processed at
> >>> the end of the year or season.
> >>> The results of a process like this will be of
> >>> immense value.  First, we will know how we are doing
> >>> compared with others as judges in the nation.
> >>> Second, we will have the opportunity to try to
> >>> improve each year base on the feedback.  In the
> >>> nats., we will have more flexibility in selecting
> >>> judges for different classes and team selection.
> >>> Sincerely,
> >>> Vince Bortone
> >>>
> >>>
> >>>
> >>>
> >>> -------------- Original message --------------
> >>>
> >>>> Just goes to show how some people have selected
> >>> memory loss with regard to
> >>>> the judge evaluation program... Please read the
> >>> quoted lines that RVP quoted
> >>>> below - Eric H. is VERY familiar with the judge
> >>> selection process and was
> >>>> instrumental in getting it through the board of
> >>> directors on 12/10/2002.
> >>>>
> >>>> Why is he questioning it now when he proposed its
> >>> vote? Oh I get it... It's
> >>>> election time!
> >>>>
> >>>> -----Original Message-----
> >>>> From: Ron Van Putte [mailto:vanputte at nuc.net]
> >>>> Sent: Tuesday, December 10, 2002 9:41 AM
> >>>> To: officers at nsrca.org
> >>>> Cc: Tony Stillman
> >>>> Subject: Re: Motion to adopt.
> >>>>
> >>>> [NSRCA Officers]
> >>>> I second the motion.
> >>>>
> >>>> As I explained at the Nats board meeting, Tony
> >>> Stillman, Matt Kebabjian, Don
> >>>> Ramsey and I have hammered out the skeleton of the
> >>> judge evaluation prog
> >>
> >>
> >> =====
> >> Bob Kane
> >> getterflash at yahoo.com
> >>
> >>
> >> 
> >> __________________________________
> >> Do you Yahoo!?
> >> Meet the all-new My Yahoo! - Try it today!
> >> http://my.yahoo.com
> >>
> >>
> >> =================================================
> >> To access the email archives for this list, go to
> >> http://lists.f3a.us/pipermail/nsrca-discussion/
> >> To be removed from this list, go to 
> >> http://www.nsrca.org/discussionA.htm
> >> and follow the instructions.
> >>
> >>
> >
> > Bob Pastorello, El Reno, OK, USA
> > rcaerobob at cox.net
> > www.rcaerobats.net
> >
> > =================================================
> > To access the email archives for this list, go to
> > http://lists.f3a.us/pipermail/nsrca-discussion/
> > To be removed from this list, go to 
> > http://www.nsrca.org/discussionA.htm
> > and follow the instructions.
> >
> 
> =================================================
> To access the email archives for this list, go to
> http://lists.f3a.us/pipermail/nsrca-discussion/
> To be removed from this list, go to http://www.nsrca.org/discussionA.htm
> and follow the instructions.
> 
=================================================
To access the email archives for this list, go to
http://lists.f3a.us/pipermail/nsrca-discussion/
To be removed from this list, go to http://www.nsrca.org/discussionA.htm
and follow the instructions.