<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.16544" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2>TBL - scores go back to the pilots as is... raw. TBL
is only factored once a round is finished and the results calculated. With
TBL scores are adjusted to the whole score and result in whole numbers.
Remember, the entire score isn't thrown out - just a maneuver score. When
a score is dropped then TBL is run on the entire set of scores again to
recalculate the mean and std. deviation - it is an iterative process until all
scores fall within the range that are calculated for the
pilots/judges.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2>TBL was used at the TOC for all years starting in 1999
onwards. It has been used at the WC since 2001 (I
think).</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2>Bottom line - the system works very well and in the end the
correct order of finish is picked - which is what we want. In all the
years of scoring with TBL at the TOC I saw very few judges scores discarded
which probably says more about the quality of judging than anything else.
I'd be very curious what the results have been at the F3A WC's since it was
implemented.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=156491504-19102007><FONT face=Arial
color=#0000ff size=2>I would love to see TBL in use at the Nats for the Finals
in both classes - we certainly have enough judges to support its
use.</FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> nsrca-discussion-bounces@lists.nsrca.org
[mailto:nsrca-discussion-bounces@lists.nsrca.org] <B>On Behalf Of
</B>rcmaster199@aol.com<BR><B>Sent:</B> Thursday, October 18, 2007 8:51
PM<BR><B>To:</B> nsrca-discussion@lists.nsrca.org<BR><B>Subject:</B> Re:
[NSRCA-discussion] D3 Championship - Scoring System
Overhaul-LONG<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV><SPAN contentEditable=false
style="DISPLAY: inline-block"></SPAN>Earl,</DIV>
<DIV> </DIV>
<DIV>If I remember correctly, some of the results from the Judge Evaluation have
been presented either in the KF or the website in past years. But I
haven't seen any results posted lately. Perhaps you are right...posting might
serve a purpose.</DIV>
<DIV> </DIV>
<DIV>Curious...in the TBL method, do the adjustments to a judge's scores come in
whole or fractional numbers? How are the scoresheets that are given back to the
pilots handled.....I mean, do they show adjusted scores or as judged? Or
both?</DIV>
<DIV> </DIV>
<DIV>Koop, was TBL used in the TOC? </DIV>
<DIV> </DIV>
<DIV>If we have access to the algorithm, I wouldn't mind taking a peek of how
the 2007 F3A Nats Final could turn out. I would bet that it wouldn't make much
difference on where the pilots placed.</DIV>
<DIV> </DIV>
<DIV>Matt</DIV>
<DIV><BR></DIV>
<DIV><BR></DIV>-----Original Message-----<BR>From: Earl Haury
<ejhaury@comcast.net><BR>To: NSRCA Mailing List
<nsrca-discussion@lists.nsrca.org><BR>Sent: Thu, Oct 18 10:24
PM<BR>Subject: Re: [NSRCA-discussion] D3 Championship - Scoring System Overhaul
-LONG<BR><BR>
<DIV id=AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253>
<DIV><FONT face=Arial>We miss an opportunity for judges to evaluate
their performance by not distributing the results of the analysis of that
performance (at major meets). I, for one, would like the judges performance data
to be published in the K-Factor (after all - the pilots scores are published).
However, realizing that some are squeamish about this, I think that we should
still provide each judge with the analysis of his / her specific performance. At
least, each judge would then have an indication of their current skills and any
variation over the years.</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>Earl</FONT></DIV>
<BLOCKQUOTE
style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV style="FONT: 10pt arial">----- Original Message ----- </DIV>
<DIV
style="BACKGROUND: #e4e4e4; FONT: 10pt arial; font-color: black"><B>From:</B>
<A title=derekkoopowitz@gmail.com href="mailto:derekkoopowitz@gmail.com">Derek
Koopowitz</A> </DIV>
<DIV style="FONT: 10pt arial"><B>To:</B> <A
title=nsrca-discussion@lists.nsrca.org
href="mailto:nsrca-discussion@lists.nsrca.org">'NSRCA Mailing List'</A> </DIV>
<DIV style="FONT: 10pt arial"><B>Sent:</B> Thursday, October 18, 2007 7:53
PM</DIV>
<DIV style="FONT: 10pt arial"><B>Subject:</B> Re: [NSRCA-discussion] D3
Championship - Scoring System Overhaul -LONG</DIV>
<DIV><BR></DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007><FONT face=Arial
color=#0000ff size=2>I'll post my dissertation on TBL again since this issue
seems to crop up time and time again...</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007>
<DIV
style="mso-element: dropcap-dropped; mso-element-frame-hspace: 3.0pt; mso-element-wrap: auto; mso-element-anchor-vertical: paragraph; mso-element-anchor-horizontal: column; mso-height-rule: exactly; mso-element-linespan: 3"><FONT
face=Garamond><SPAN class=296224900-19102007>T</SPAN>he Tarasov-Bauer-Long
(TBL) Scoring method has been around since the 1970’s.<SPAN
style="mso-spacerun: yes"> </SPAN>It has been used in the full size
arena since 1978 and has been used at every full size IAC World Championship
since 1980.<SPAN style="mso-spacerun: yes"> </SPAN>The TBL method
applies proven statistical probability theory to the judge’s scores to resolve
style differences and bias, and to avoid the inclusion of potential faulty
judgements in contest results.<SPAN style="mso-spacerun: yes"> </SPAN>To
understand just why we need TBL, and how it works, is if considerable
importance to us all.<SPAN style="mso-spacerun: yes"> </SPAN>It is
important to the pilots because it is there to reduce the prospect of
unsatisfactory judgements affecting their results, and it is important for
judges because it will introduce a completely new dimension of scrutiny into
the sequence totals, and it will also discreetly engage the attention of the
Chief Judge, or Contest Director, if the judges conclusions differ
sufficiently from all those other judges on the same panel.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>When people get together to judge how well a
pre-defined competitive task is being tackled, the range of opinions is often
diverse.<SPAN style="mso-spacerun: yes"> </SPAN>This is entirely natural
among humans where the critique of any display of skill relies on the
interpretation of rapidly changing visual cues.<SPAN
style="mso-spacerun: yes"> </SPAN>In order to minimize the prospect of
any “way out opinions” having too much effect on the result, it is usual to
average the accumulated scores to arrive at a final assessment, which takes
everybody’s opinion into account.<SPAN style="mso-spacerun: yes">
</SPAN>Unfortunately this averaging approach can achieve the opposite of what
we really want, which is to identify, and where needed, remove those “way out
opinions” because they are the ones most likely to be ill-judged and therefore
should be discarded, leaving the rest to determine the more appropriate
result.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>In aerobatics the process of judging according
to the rulebook normally leads to a series of generally similar personal
views.<SPAN style="mso-spacerun: yes"> </SPAN>However, one judge’s
downgrading may be harsher or more lenient than the next, their personal
feelings toward each competitor or aircraft type may predispose toward favor
or dislike (bias), and they will almost certainly miss or see things that
other judges do not.<SPAN style="mso-spacerun: yes"> </SPAN>How then can
we “judge” the judges and so reach a conclusion, which has good probability of
acceptance by all the concerned parties?</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>The key word is <B
style="mso-bidi-font-weight: normal"><I
style="mso-bidi-font-style: normal">probability</I></B>, the concept of a
perceived level of confidence in collectively viewed judgements has entered
the frame.<SPAN style="mso-spacerun: yes"> </SPAN>What we really mean is
that we must be confident that opinions pitched outside some pre-defined level
of reasonable acceptability will be identified as such and will not be
used.<SPAN style="mso-spacerun: yes"> </SPAN>This sort of situation is
the daily bread and butter of well established probability theory which, when
suitably applied, can produce a very clear cut analysis of numerically
expressed opinions provided that the appropriate criteria have been carefully
established beforehand.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>What has been developed through several
previous editions is some arithmetic which addresses the judge’s raw scores in
such a way that any which are probably unfair are discarded with an
established level of confidence.<SPAN style="mso-spacerun: yes">
</SPAN>To understand the process you need only accept some quite simple
arithmetic procedures, which are central to what is called “statistical
probability”.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>The TBL scoring system in effect does the
following:</FONT></DIV>
<DIV class=MsoListBullet2
style="MARGIN: 0in 0.25in 12pt 0.75in; TEXT-INDENT: -0.25in; TEXT-ALIGN: left; mso-list: l0 level1 lfo1"
align=left><SPAN
style="FONT-SIZE: 14pt; FONT-FAMILY: Symbol; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: Symbol; mso-bidi-font-family: Symbol"><SPAN
style="mso-list: Ignore">·<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><FONT face=Garamond>Commonizes the judging
styles.</FONT></DIV>
<DIV class=MsoListBullet2
style="MARGIN: 0in 0.25in 12pt 0.75in; TEXT-INDENT: -0.25in; TEXT-ALIGN: left; mso-list: l0 level1 lfo1"
align=left><SPAN
style="FONT-SIZE: 14pt; FONT-FAMILY: Symbol; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: Symbol; mso-bidi-font-family: Symbol"><SPAN
style="mso-list: Ignore">·<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><FONT face=Garamond>Computes TBL scores</FONT></DIV>
<DIV class=MsoListBullet2
style="MARGIN: 0in 0.25in 12pt 0.75in; TEXT-INDENT: -0.25in; TEXT-ALIGN: left; mso-list: l0 level1 lfo1"
align=left><SPAN
style="FONT-SIZE: 14pt; FONT-FAMILY: Symbol; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: Symbol; mso-bidi-font-family: Symbol"><SPAN
style="mso-list: Ignore">·<SPAN
style="FONT: 7pt 'Times New Roman'">
</SPAN></SPAN></SPAN><FONT face=Garamond>Publishes results</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond></FONT> </DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>Commonizing the judging styles involves
remodeling the scores to bring all the judging styles to a common format and
removing any natural bias between panel members.<SPAN
style="mso-spacerun: yes"> </SPAN>Following some calculations, each
judge’s set of scores are squeezed or stretched and moved en-bloc up or down
so that the sets all show the same overall spread and have identical averages
(bias).<SPAN style="mso-spacerun: yes"> </SPAN>Within each set the pilot
order and score progression must remain unaltered, but now valid score
comparisons are possible between all the panel judges on behalf of each
pilot.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>Computing the TBL score involves looking at the
high and low scores in each pilot’s set and throws out any that are too “far
out” to be fair.<SPAN style="mso-spacerun: yes"> </SPAN>This is done by
subtracting the average for the set from each one and dividing the result by
the “sample standard deviation” - if the result of this sum is greater than
1.645 then according to statistical probability theory we can be at least 90%
confident that it is unfair, so the score is discarded.<SPAN
style="mso-spacerun: yes"> </SPAN>This calculation and the
mathematically derived 1.645 criteria is the key to the correctness of the TBL
process, and is based on many years of experience by the full size aerobatics
organization with contest scores at all levels.<SPAN
style="mso-spacerun: yes"> </SPAN>The discarding of any scores of course
changes for a pilot the average and standard deviation of their remaining
results, and so the whole process is repeated.<SPAN
style="mso-spacerun: yes"> </SPAN>After several cycles any “unfair”
scores will have gone, and those that remain will all satisfy the essential
90% confidence criteria.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>Publishing the results is derived by averaging
each pilot’s scores.<SPAN style="mso-spacerun: yes"> </SPAN>The final
TBL iteration therefore has any appropriate penalty/bonus values applied and
the results are then sorted in order of descent of the total scores to rank
the pilots first to last.<SPAN style="mso-spacerun: yes"> </SPAN>These
final scores may, or may not, be normalized to 1000 points, depending on the
setting for the selected class.</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>Educating and improving the judges is a useful
by-product of this process in that it provides all the bells and whistles how
each judge has performed by comparison with the overall judging panel average
and when seen against the 90% level of confidence criteria.<SPAN
style="mso-spacerun: yes"> </SPAN>The TBL system will produce an
analysis showing each judge the percentage of scores accepted as “OK”, and a
comparison with the panel style (spread of score) and bias
(average).</FONT></DIV>
<DIV class=MsoBodyText style="MARGIN: 0in 0in 12pt; TEXT-ALIGN: left"
align=left><FONT face=Garamond>Unfortunately TBL, by definition, brings with
it a 10% possibility of upsetting an honest judge’s day.<SPAN
style="mso-spacerun: yes"> </SPAN>The trade-off is that we expect not
only to achieve a set of results with at least 90% confidence that are “fair”
every time, but that the system also provides us with a wonderful tool to
address our judging standards.<SPAN style="mso-spacerun: yes">
</SPAN>TBL will ensure that every judge’s opinion has equal weight, and that
each sequence score by each judge is accepted only if it lies within an
acceptable margin from the panel average.<SPAN
style="mso-spacerun: yes"> </SPAN>TBL, however, by necessity takes the
dominant judging panel view as the “correct” one and it can’t make right
scores out of wrong ones.<SPAN style="mso-spacerun: yes"> </SPAN>If 6
out of 8 judges are distracted and make a mess out of one pilots efforts, then
for TBL this becomes the controlling assessment of that pilots performance,
and the other 2 diligent judges who got it right will see their scores
unceremoniously zapped.<SPAN style="mso-spacerun: yes"> </SPAN>In
practice this would be extremely unusual - from the judging line it is almost
impossible to deliberately upset the final results without collusion between a
majority of the judges, and if that starts to happen then someone is
definitely on the wrong planet.</FONT></DIV></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=296224900-19102007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left>
<HR>
</DIV>
<DIV dir=ltr align=left><FONT face=Tahoma size=2><B>From:</B> <A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A>
[<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>]
<B>On Behalf Of </B><A
href="mailto:vicenterc@comcast.net">vicenterc@comcast.net</A><BR><B>Sent:</B>
Thursday, October 18, 2007 8:11 AM<BR><B>To:</B> NSRCA Mailing
List<BR><B>Subject:</B> Re: [NSRCA-discussion] D3 Championship - Scoring
System Overhaul<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV>Tony,</DIV>
<DIV> </DIV>
<DIV>Do you know if the TBL system eliminates the high and low
scores? I think that is a good solution but we can not do it in local
contests. Probably we could in some contests since we have many Masters
vs. F3A. </DIV>
<DIV> </DIV>
<DIV>Do you know "link" where we can read about TBL system?</DIV>
<DIV> </DIV>
<DIV class=signature id=signature>--<BR>Vicente "Vince" Bortone</DIV>
<DIV> </DIV>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">--------------
Original message -------------- <BR>From: "Tony" <<A
href="mailto:tony@radiosouthrc.com">tony@radiosouthrc.com</A>> <BR>
<STYLE>#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 A:link {
COLOR: blue; TEXT-DECORATION: underline
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 SPAN.EmailStyle17 {
COLOR: navy; FONT-FAMILY: Arial; mso-style-type: personal-reply
}
#AOLMsgPart_3_850a6352-9570-4af6-a752-5bdc9529c253 DIV.Section1 {
page: Section1
}
</STYLE>
<DIV class=Section1>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">This TBL will find
these problems and is in use at World Champs. The problem is that you
need at least 5 judges on a line.</SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"></SPAN></FONT> </DIV>
<DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Tony Stillman,
President</SPAN></FONT><FONT color=navy><SPAN
style="COLOR: navy"></SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Radio South,
Inc.</SPAN></FONT><FONT color=navy><SPAN
style="COLOR: navy"></SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">139 Altama
Connector, Box 322</SPAN></FONT><FONT color=navy><SPAN
style="COLOR: navy"></SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">Brunswick</SPAN></FONT><FONT
face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">, GA
31525</SPAN></FONT><FONT color=navy><SPAN
style="COLOR: navy"></SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial">1-800-962-7802</SPAN></FONT><FONT
color=navy><SPAN style="COLOR: navy"></SPAN></FONT></DIV>
<DIV class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><A
href="mailto:tony@radiosouthrc.com">tony@radiosouthrc.com</A></SPAN></FONT></DIV></DIV>
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<DIV class=MsoNormal><B><FONT face=Tahoma size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma"> <A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A>
[<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>]
<B><SPAN style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B><A
href="mailto:vicenterc@comcast.net">vicenterc@comcast.net</A><BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> Thursday, October 18, 2007 10:44
AM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B> NSRCA Mailing List;
NSRCA Mailing List<BR><B><SPAN style="FONT-WEIGHT: bold">Subject:</SPAN></B>
Re: [NSRCA-discussion] D3 Championship - Scoring System
Overhaul</SPAN></FONT></DIV></DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"></SPAN></FONT> </DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt">I agree with George. If I remember right from
statistics courses 20 years ago, this type of problem follows the normal
distribution or bell shape curve. In order to have any significant
precision in the scoring system we will need to have at least 33 judges per
round. We all know that it is impossible for us to have 33
judges per round. I also agree with George that at the end of the
contest the winner is usually the best pilot and if you ask around very high
percentage will agree with the results.</SPAN></FONT></DIV></DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"></SPAN></FONT> </DIV></DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt">Well, we need a PHD in statistics to help us
out.</SPAN></FONT></DIV></DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"></SPAN></FONT> </DIV></DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt">--<BR>Vicente "Vince"
Bortone</SPAN></FONT></DIV></DIV>
<DIV>
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"></SPAN></FONT> </DIV></DIV>
<BLOCKQUOTE
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; MARGIN-TOP: 5pt; PADDING-LEFT: 4pt; MARGIN-BOTTOM: 5pt; PADDING-BOTTOM: 0in; MARGIN-LEFT: 3.75pt; BORDER-LEFT: #1010ff 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<DIV class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt">-------------- Original message --------------
<BR>From: <<A
href="mailto:glmiller3@suddenlink.net">glmiller3@suddenlink.net</A>>
<BR><BR>> I've said this before and most people don't "get it", but we
are asking more <BR>> precision from our scoring system than is
mathematically possible. <BR>> <BR>> In the case of FAI scores, the
raw scores are rounded to one significant digit <BR>> (whole numbers
only from 1-10). This means that after any mathematical <BR>>
manipulation, the result is only "Accurate" to one digit and the result
should <BR>> be rounded to that digit using some standardized "rounding
algorithm".... We are <BR>> manipulating a single digit of significance
and basing outcomes on up to EIGHT <BR>> digits (1234.5678 ).
Mathematically speaking any "normalized score" between <BR>> 950.0000
and 1000.0000 is the same score.....900 to 949; 850-900; etc. All <BR>>
other problems of judgin! g! incon sistency, bias, averaging, etc pale in
<BR>> comparison. <BR>> <BR>> As I've said before, I'm astounded
that a system that is so mathematically <BR>> flawed can provide
results that are as good as they are....for my part, I <BR>> usually
feel like I have been ranked pretty fairly compared to the other pilots
<BR>> in my class even though the ranking is based on statistically
meaningless <BR>> numbers. <BR>> <BR>> George <BR>> <BR>>
<BR>> ---- "Woodward wrote: <BR>> > Guys - <BR>> > <BR>>
> <BR>> > <BR>> > 1. Why do we average judge's scores
together? <BR>> > <BR>> > a. The whole system is predicated on
judges being <BR>> > competent/consistent/correct/un-biased. If this
first rule is violated <BR>> > ranging from small to 100 point raw
score difference, the idea of now <BR>> > "averaging" the score also
has no validity. <BR>> > b. Why not just let each judge's score
stand as! ! is, < B R><SPAN style="FONT-WEIGHT: bold">> >
unaltered, and produce two normalized scores per round? <BR>> > c.
Averaging the scores together may be doing a disservice <BR>> > to
everyone. <BR>> > d. You would basically have two sets of scores per
round. <BR>> > IE you may end up with 1000 points on one card, and
an 800 on the other. <BR>> > <BR>> > e. This would identify
immediately any cause for concern. <BR>> > f. Would it provide an
immediate training tool back to our <BR>> > system of pilots,
judges, and CD? <BR>> > g. Attempting to "un-average" the scores to
determine what <BR>> > happened takes place anyway on the flight
line, after the scores are <BR>> > printed. <BR>> > <BR>>
> 2. Are we asking too much from judges? <BR>> > <BR>> > a.
Is applying downgrades, then counting backwards from 10, <BR>> > in
the context of "turnaround" pattern where maneuvers can happen <BR>>! ;
> back-to-back quickly, too d! ifficul t across the full spectrum of
<BR>> > competitor/judges? <BR>> > <BR>> > 3. Dropping
Rounds: <BR>> > <BR>> > a. Is this still a good idea? <BR>>
> b. I wouldn't mind dropping one round, but it was explained <BR>>
> to me last night that this is an artifact from the days of when
people <BR>> > would break a prop on touch-n-goes, and in general
lower equipment <BR>> > reliability. <BR>> > c. In the age of
higher equipment reliability, is the <BR>> > 'round-drop' scenarios
still good, left as is? <BR>> > <BR>> > 4. Dropping Rounds -
Take 2: <BR>> > <BR>> > a. In the context of point #1, maybe
we should be allowed <BR>> > to drop the lowest scored judge from
each round, versus the entire <BR>> > round. <BR>> > b. Why
should the pilot drop the entire round, when one <BR>> > judge may
have scored him 1000 points, and the other 800? ! <BR>> ; > c. If
you end up with a! tie at the end, you just keep <BR>> > counting
"1000's" until the other pilot runs out - tie is now "untied." <BR>>
> <BR>> > <BR>> > <BR>> > I hope some smart guys can
chime in on potential over-hauling idea. <BR>> > <BR>> >
<BR>> > <BR>> > <BR>> > <BR>> >
________________________________ <BR>> > <BR>> > From: <A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A>
<BR>> > [<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>]
On Behalf Of Lisa & <BR>> > Larry <BR>> > Sent: Wednesday,
October 17, 2007 10:27 PM <BR>> > To: 'NSRCA Mailing List' <BR>>
> Subject: Re: [NSRCA-discussion] D3 Championship <BR>> >
<BR>> > <BR>> > <BR>> > OK...I probably shouldn't start
this, but I will... <BR>> > <BR>> > <BR>> > <BR>>
> I haven't read all the threads, but I have read the ones below
<BR>> > in this series. <B>> > <BR>> > <BR>> >! ;
<BR>& gt; > The NSRCA has already set the standard and a method to
determine <BR>> > judging bias and has held a NSRCA member
accountable to this standard <BR>> > this year and the AMA
sanctioned the individual. This is fact....Agree <BR>> > with the
method to determine bias (or not) it was used to impose an AMA <BR>>
> sanction on a member. <BR>> > <BR>> > <BR>> >
<BR>> > IMHO this discussion suggests that bias has occurred in the
D3 <BR>> > championship or possibly another at the same level FAI.
If this is the <BR>> > case the NSRCA must review this and apply the
same discipline using the <BR>> > same measurables to provide for
the same sanctions. <BR>> > <BR>> > <BR>> > <BR>>
> If the NSRCA is unwilling to investigate or isn't willing to use
<BR>> > the same method to determine bias, then clearly we (the
NSRCA and AMA) <BR>> > ha! ve dise nfranchised a NSRCA member and
should r! ethink his sanction. <BR>> > <BR>> > <BR>> >
<BR>> > Our rules and penalties must check and balance. Then they
must <BR>> > be applied to all members equally regardless of status
in membership. <BR>> > This is the only way to reduce / eliminate
bias. I'm also unwilling to <BR>> > entertain the thought the
District Championship is any less important to <BR>> > the NATS.
They are both sanctioned contests ran by a CD accountable to <BR>> >
the AMA. <BR>> > <BR>> > <BR>> > <BR>> > Flame
suit on... <BR>> > <BR>> > <BR>> > <BR>> > Larry
Diamond <BR>> > <BR>> > <BR>> > <BR>> > -----
Original Message ----- <BR>> > <BR>> > From: Mike Hester
<BR>> > <BR>> > To: NSRCA Mailing List <BR>! > &g t;
<BR>> > <BR>> > Sent: Wednesday, October 17, 2007 2:24 PM
<BR>> > ! <BR>> ; > Subject: Re: [NSRCA-discussion] D3
Championship <BR>> > <BR>> > <BR>> > <BR>> > He's
not alone. Although he probably should work on the delivery <BR>> >
;) <BR>> > <BR>> > <BR>> > <BR>> > I would support
any of the 5 proposals that Ryan listed. Judging <BR>> > FAI can be
frustrating enough, but to be told you're not getting it <BR>> >
right when you're already doing everything you know how to do, that's a
<BR>> > hard pill to swallow regardless of the statement's accuracy.
<BR>> > <BR>> > <BR>> > <BR>> > You guys out there
do need to realize these guys can fly...and <BR>> > are very
good...problem is they're flying against this Jason dude, <BR>> >
travels a lot, flys all the time, might even have a national title or 2
<BR>> > along the way, not sure. I'm sure ! you kno w the type.
*ahem* <BR>> > <BR>> > <BR>> > <BR>> > Because my
wife ! general ly keeps scores in D3, we have some <BR>> > pretty
good access to each and every score entered. I can tell you guys <BR>>
> without a doubt at times there are some SERIOUS differences in scores
<BR>> > between judges on the same round. I don't mean a little, I
mean like 100 <BR>> > points on the RAW score. Even if this Jason
character was flying <BR>> > straight 10s, the differences if you
work them out mean the others are <BR>> > barely flying a straight
line....and that's not the case. I have no <BR>> > doubt these guys
don't think they should be beating Jason in a 6 round <BR>> >
contest where 2 of the rounds are "F" rounds, but I am sure most people
<BR>> > would agree the scoring could use some improvements.
<BR>> > <BR>> > <BR>> > <BR>> > Being one of these
evil incompetent D3 masters judges *! ahem* I <BR>> > would
certainly support more of a cooperative effort than some kind of <BR>>!
> p rotest. I have been very supportive of all the FAI guys and
especially <BR>> > the scoring, and am usually the guys everybody
throws something at <BR>> > during a judging seminar because I'm
trying to clarify something that <BR>> > effects mainly FAI.
<BR>> > <BR>> > <BR>> > <BR>> > I think to
identify the "problem" will take a willingness to <BR>> > recognize
that the situation is caused by a LOT of factors, not any one <BR>>
> or two. If anyone's interested, I'll outline the ones I see clearly.
<BR>> > <BR>> > <BR>> > <BR>> > I'm not sure if
this will all have the intended effect that jim <BR>> > was looking
for in the end, but if nothing else it does draw some <BR>> >
attention to a situation and we should have a closer look. <BR>> >
<BR>> &! gt; <BR>> > <BR>> > As for me, soon I'll be
practicing, bracing for the onslought of <BR>> > FAI pilots come to
master! s to pu nish me =) <BR>> > <BR>> > <BR>> >
<BR>> > -Mike <BR>> > <BR>> > ----- Original Message
----- <BR>> > <BR>> > From: McLaughlin, Ryan (FRS.JAX)
<BR>> > <BR>> > <BR>> > To: NSRCA Mailing List <BR>>
> <BR>> > <BR>> > Sent: Wednesday, October 17, 2007 1:50 PM
<BR>> > <BR>> > Subject: Re: [NSRCA-discussion] D3
Championship <BR>> > <BR>> > <BR>> > <BR>> > I
didn't want you to stand alone in this...it's too <BR>> > important.
<BR>> > <BR>> > -----Original Message----- <BR>> > From:
<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A>
<BR>> > [<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>]
On Behalf Of Woodward, <BR>> > ! Jim <BR>> > Sent: Wednesday,
October 17, 2007 1:31 PM <BR>> > To: NSRCA Mailing List <BR>>
> Subject: Re: [NSRCA-discussion] D3 Champ! ionship <BR>> >
<BR>> > Ryan M., <BR>> > <BR>> > I think this takes the
cake as a first time <BR>> > nsrca-list email. Thank you for the
support. <BR>> > <BR>> > Jim W. <BR>> > <BR>> >
<BR>> > <BR>> > CONFIDENTIALITY NOTICE: This e-mail message,
<BR>> > including any attachments, is for the sole use of the
intended <BR>> > recipient(s) and may contain confidential and
proprietary information. <BR>> > Any unauthorized review, use,
disclosure or distribution is prohibited. <BR>> > If you are not the
intended recipient(s), please contact the sender by <BR>> > reply
e-mail and destroy all copies of the original message. <BR>> >
<BR>> > <BR>> > <BR>> > <BR>> >
________________________________ <BR>> > <BR>&g! t; >
<BR>> > From: <A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A>
<BR>> > [<A
href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>]
On Behalf Of <BR>>! ; > McLaughlin, Ryan (FRS.JAX) <BR>> >
Sent: Wednesday, October 17, 2007 1:19 PM <BR>> > To: <A
href="mailto:nsrca-discussion@lists.nsrca.org">nsrca-discussion@lists.nsrca.org</A>
<BR>> > Subject: [NSRCA-discussion] D3 Championship <BR>> >
<BR>> > This is my first post to the NSRCA list as I am <BR>>
> a bit 'internet shy', but I thought I might be able to add some value
to <BR>> > the FAI judging discussion Jim W started. Although I tend
to err on the <BR>> > side of diplomacy : ), I believe the feelings
Jim expressed are <BR>> > legitimate and shared by many FAI
competitors throughout the country. <BR>> > As a long time
participant, I realize that bias is not a new problem but <BR>> > I
do not think we should accept this is as a "fact of life" and move on.
<BR>> > I think we have an excellent opportunity h! ere and we
should make the <BR>> > most of it. <BR>> > <BR>> > The
primary issue to address in my opinion is <BR>> > not disparity i! n
judgi ng standards between judges, though as Earl points <BR>> >
out, this is important. Rather, it is the different standard applied to
<BR>> > pilots within one score set--i.e.. scoring a pilot lower or
higher based <BR>> > on who he is. Our penchant for creating
"superstars" is the most <BR>> > discouraging aspect of FAI
competition. To remedy this, we must all <BR>> > make a conscious
decision to change a long established tradition in our <BR>> >
sport. Are we ready to take this on? <BR>> > <BR>> >
Complaining isn't the answer and neither is <BR>> > staying quiet, a
mistake that has made the FAI competitors as <BR>> > responsible as
anyone else for the situation. To this end, I submit for <BR>> >
your review the following ideas to specifically target the F! AI bias
<BR>> > issue: <BR>> > <BR>> > 1. Sacrifice one FAI
round per contest to serve <BR>> > as an "open" round for all
contestants e! xpected to judge FAI during the <BR>> > event. Allow
everyone to compare notes and use this as a coaching <BR>> >
opportunity. <BR>> > <BR>> > 2. Drop one FAI pilot to Masters
at each <BR>> > contest to serve as a judge for all rounds and use
volunteers from other <BR>> > classes to serve as the others. This
would have to be an agreement made <BR>> > among FAI pilots.
<BR>> > <BR>> > 3. Extend the pilots meeting to go over
<BR>> > specific issues, maybe a new one or two every meet rather
than just <BR>> > pointing out the landing zone, etc. Make a "mini"
judging seminar <BR>> > mandatory each contest. <BR>> >
<BR>> > 4. Certify judges for FAI on a volunteer basis <BR>> >
and only use "certified" judges in the contest. <BR>> > <BR>>
& gt; 5. Utilize peer judging, in other words, have <BR>> > FAI
pilots judge themselves. If a pilot is not flying, he is judging <BR>>
> his fel! low com petitors. <BR>> > <BR>> > Some of this
may seem radical, but I believe <BR>> > there is room for a bit of
this. Pattern belongs to us right? I <BR>> > welcome any ideas or
critique anyone can offer. I will clarify any of <BR>> > the above
upon request. <BR>> > <BR>> > Thank you for your
consideration. <BR>> > <BR>> > Ryan McLaughlin <BR>> >
Eustis, Florida <BR>> > <BR>> > <BR>> > <BR>> >
<BR>> > ________________________________ <BR>> > <BR>> >
<BR>> > This message w/attachments (message) may be <BR>> >
privileged, confidential or proprietary, and if you are not an intended
<BR>> > recipient, please notify the sender, do not use or ! share i
t and delete <BR>> > it. Unless specifically indicated, this message
is not an offer to sell <BR>> > or a solicitation of any investment
products or other financial product <BR>> > or service, an official
! confirm ation of any transaction, or an official <BR>> > statement
of Merrill Lynch. Subject to applicable law, Merrill Lynch may <BR>>
> monitor, review and retain e-communications (EC) traveling through
its <BR>> > networks/systems. The laws of the country of each
sender/recipient may <BR>> > impact the handling of EC, and EC may
be archived, supervised and <BR>> > produced in countries other than
the country in which you are located. <BR>> > This message cannot be
guaranteed to be secure or error-free. This <BR>> > message is
subject to terms available at the following link: <BR>> > <A
href="http://www.ml.com/e-communications_terms/"
target=_blank>http://www.ml.com/e-communications_terms/</A>. By messaging
with Merrill <BR>> > Lynch you consent to the foregoing. <BR>! >
&g t; <BR>> > <BR>> > ________________________________
<BR>> > <BR>> > <BR>> > <BR>> > <BR>> >
</SPAN></B></SPAN></FONT></DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><B><FONT
face="Times New Roman" size=3><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 12pt">
<HR align=center width="100%" SIZE=2>
</SPAN></FONT></B></DIV>
<DIV class=MsoNormal><B><FONT face="Times New Roman" size=3><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 12pt"><BR>> > <BR>> >
_______________________________________________ <BR>> >
NSRCA-discussion mailing list <BR>> > <A
href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A>
<BR>> > <A
href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion"
target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A>
<BR>> > <BR>> > <BR>> > ________________________________
<BR>> > <BR>> > <BR>> >
_______________________________________________ <BR>> >
NSRCA-discussion mailing list <BR>> > <A
href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A>
<BR>> > <A
href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion"
target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A>
<BR>> > <BR>> <BR>>
_______________________________________________ <BR>> NSRCA-discussion
mailing list <BR>> <A
href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A>
<BR>> <A
href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion"
target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A>
</SPAN></FONT></B></DIV></BLOCKQUOTE></DIV></BLOCKQUOTE>
<DIV>
<HR>
<DIV></DIV>_______________________________________________<BR>NSRCA-discussion
mailing list<BR><A
href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A><BR><A
href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion"
target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A></DIV></BLOCKQUOTE></DIV>
<DIV id=AOLMsgPart_4_850a6352-9570-4af6-a752-5bdc9529c253
style="FONT-SIZE: 12px; MARGIN: 0px; COLOR: #000; FONT-FAMILY: Tahoma, Verdana, Arial, Sans-Serif; BACKGROUND-COLOR: #fff"><PRE style="FONT-SIZE: 9pt"><TT>_______________________________________________ NSRCA-discussion mailing list <A href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A> <A href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion" target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A> </TT></PRE></DIV>
<DIV class=AOLPromoFooter>
<HR style="MARGIN-TOP: 10px">
Email and AIM finally together. You've gotta check out free <A
href="http://o.aolcdn.com/cdn.webmail.aol.com/mailtour/aol/en-us/index.htm?ncid=AOLAOF00020000000970"
target=_blank>AOL Mail</A>!<BR></DIV></BODY></HTML>