NSRCA Judge Ranking System

Tue Dec 14 09:20:05 AKST 2004

On Dec 14, 2004, at 10:25 AM, <rcaerobob at cox.net> wrote:

> BUT - if you correlated the placings of contestants against the 
> judges' score to come up with a "validity coefficient", you could use 
> THAT to rank a judge, as the coefficient would be a constant relative 
> to that judges' performance with respect to the outcome of an event.

The NSRCA judge ranking system does this.  It is assumed that the 
consensus rankings are correct and compares the judge's ranking 
relative to the consensus.  There are those who will argue that using 
the judging consensus  as a standard is not accurate.  However, that's 
exactly how we award trophies at a contest.  BTW, it should be obvious 
that the fewer the number of contestants, the less accurate the 
consensus ranking will be.  That's why we required that at least five 
judges must be on the line.  This is why Nats Finals and F3A Team 
Selections are the only data we used.

The NSRCA judge ranking system also evaluates how a judge's scores for 
individual contestants compares to the consensus scores.  A particular 
judge's scoring ' errors' for all pilots are combined for a total 
scoring 'error'.

The two factors are combined for a ranking score.

I hope this brief explanation helps clarify the situation for those who 
did not read my earlier explanations in the K-Factor over the past two 
years.

Ron Van Putte

>> From: Bob Kane <getterflash at yahoo.com>
>> Date: 2004/12/14 Tue AM 11:18:53 EST
>> To: discussion at nsrca.org
>> Subject: Re: FW: Motion to adopt.
>>
>> You are correct, the opportunities to be ranked as a
>> judge are few and far between, but they are the only
>> statistically valid comparisons. I am hoping someone
>> who is a statistician will jump in and offer the
>> mathmatical proof, I know enough to be dangerous. I
>> wish it were possible to enter judge's scores from
>> every contest into some kind of database to come up
>> with a performance ranking. Unfortunately, it is
>> statistically invalid since they are judging different
>> events at different times with no correlation between
>> them. The only thing you could compute would be an
>> average awarded score and a range for each judge which
>> while interesting, means nothing about absolute
>> performance.
>>
>> --- vicenterc at comcast.net wrote:
>>
>>> Hi Tony:
>>>
>>> The problem is that the judges that have the
>>> opportunity to participate in 5 judges event is very
>>> low.  Are you sure that the data even with two
>>> judges is not significant?  I will agree that with
>>> the 5 judges the data will be more significant using
>>> using two contest only.  With two judges doing all
>>> country with several contest per judge will become
>>> significant data also.  At least will be more
>>> significant than the method is used now.  The
>>> results from the previous years Nats. should be also
>>> used since this is three judge event.  In this way,
>>> NSRCA will give more opportunity to all certified
>>> judges and the process will be transparent and clear
>>> to all.  There are other benefits, yes we love
>>> competition and this will be a competition to become
>>> better judges.  The end results is that NSRCA will
>>> help improve not only the pilots skills but also the
>>> judging skills which is very important part of our
>>> sport.
>>>
>>> Best regards,
>>>
>>> Vince Bortone
>>>
>>> -------------- Original message --------------
>>>
>>> Chris/Vince:
>>>
>>> The judging program that the Team created does just
>>> that!  You can see the standings on the NSRCA
>>> website.  The standings are created from the data of
>>> contests where we use a minimum of 5 judges per
>>> flight.  This is only done (to my knowledge) at the
>>> US Nats FINALS, and the Team Selection Contest.  The
>>> requirement of 5 judges in essential to get valid
>>> statistics.
>>>
>>> Tony Stillman
>>> Radio South
>>> 3702 N. Pace Blvd.
>>> Pensacola, FL 32505
>>> 1-800-962-7802
>>> www.radiosouthrc.com
>>> ----- Original Message -----
>>> From: Chris Larson
>>> To: discussion at nsrca.org
>>> Sent: Monday, December 13, 2004 10:32 PM
>>> Subject: RE: FW: Motion to adopt.
>>>
>>>
>>> I agree Vince.
>>>
>>> HUGE undertaking, but the results would be
>>> fantastic.  Imagine the ability to compare yourself
>>> to all judges across the country.  Now that's
>>> motivation to better yourself at your craft.
>>>
>>> Hmmm..... At the same time, how about a national
>>> PILOT ranking system, similar to the one used in the
>>> USTA  ( Tennis ).  I believe they know their rank in
>>> their given class on a national basis.  Might be
>>> kind of fun, and good for us guys who can't travel
>>> to the Nats!  We could still be "competing" with
>>> guys from the other districts!
>>>
>>> Publish the top judges and the top competitors in
>>> each class in the K-Factor.
>>>
>>> Chris Larson
>>> L & D Sales
>>> 209-274-2176 Office / Fax
>>> 209-304-0865 Cellular
>>> -----Original Message-----
>>> From: discussion-request at nsrca.org
>>> [mailto:discussion-request at nsrca.org]On Behalf Of
>>> vicenterc at comcast.net
>>> Sent: Monday, December 13, 2004 7:49 PM
>>> To: discussion at nsrca.org; NSRCA List
>>> Cc: Derek Koopowitz
>>> Subject: Re: FW: Motion to adopt.
>>>
>>>
>>> I am firm believer of statistics. I have not chance
>>> to read the program proposed but I am sure that
>>> should be based on statistical formulation.  In
>>> statistics, more data is compiled more valid will be
>>> the results and conclusions obtained from that
>>> analysis.  I am not am expert but I would like to
>>> see in the future a process of judge selection that
>>> is based on all information compiled in ALL contests
>>> around the nation.  Yes, I am proposing that all
>>> data be compiled and used to determine the best
>>> judges.  I don't think that this is very difficult
>>> task since the data is already available in
>>> electronic format.  We will need to make sure that
>>> the CD sends the information to the NSRCA.  I know
>>> that there will be many happy faces if some of the
>>> new candidates support this effort.
>>>  Mortal guys like me will say that this is
>>> impossible.  However, this is very easy task to
>>> someone expert in databases and computer
>>> programming.  We just need to make sure that all
>>> scoring programs produce the data in the format
>>> required.  That program will use the judge NSRCA
>>> number to send the information to be processed at
>>> the end of the year or season.
>>> The results of a process like this will be of
>>> immense value.  First, we will know how we are doing
>>> compared with others as judges in the nation.
>>> Second, we will have the opportunity to try to
>>> improve each year base on the feedback.  In the
>>> nats., we will have more flexibility in selecting
>>> judges for different classes and team selection.
>>> Sincerely,
>>> Vince Bortone
>>>
>>>
>>>
>>>
>>> -------------- Original message --------------
>>>
>>>> Just goes to show how some people have selected
>>> memory loss with regard to
>>>> the judge evaluation program... Please read the
>>> quoted lines that RVP quoted
>>>> below - Eric H. is VERY familiar with the judge
>>> selection process and was
>>>> instrumental in getting it through the board of
>>> directors on 12/10/2002.
>>>>
>>>> Why is he questioning it now when he proposed its
>>> vote? Oh I get it... It's
>>>> election time!
>>>>
>>>> -----Original Message-----
>>>> From: Ron Van Putte [mailto:vanputte at nuc.net]
>>>> Sent: Tuesday, December 10, 2002 9:41 AM
>>>> To: officers at nsrca.org
>>>> Cc: Tony Stillman
>>>> Subject: Re: Motion to adopt.
>>>>
>>>> [NSRCA Officers]
>>>> I second the motion.
>>>>
>>>> As I explained at the Nats board meeting, Tony
>>> Stillman, Matt Kebabjian, Don
>>>> Ramsey and I have hammered out the skeleton of the
>>> judge evaluation prog
>>
>>
>> =====
>> Bob Kane
>> getterflash at yahoo.com
>>
>>
>> 		
>> __________________________________
>> Do you Yahoo!?
>> Meet the all-new My Yahoo! - Try it today!
>> http://my.yahoo.com
>>
>>
>> =================================================
>> To access the email archives for this list, go to
>> http://lists.f3a.us/pipermail/nsrca-discussion/
>> To be removed from this list, go to 
>> http://www.nsrca.org/discussionA.htm
>> and follow the instructions.
>>
>>
>
> Bob Pastorello, El Reno, OK, USA
> rcaerobob at cox.net
> www.rcaerobats.net
>
> =================================================
> To access the email archives for this list, go to
> http://lists.f3a.us/pipermail/nsrca-discussion/
> To be removed from this list, go to 
> http://www.nsrca.org/discussionA.htm
> and follow the instructions.
>

=================================================
To access the email archives for this list, go to
http://lists.f3a.us/pipermail/nsrca-discussion/
To be removed from this list, go to http://www.nsrca.org/discussionA.htm
and follow the instructions.