<font color='black' size='2' face='arial'>
<div><FONT style="BACKGROUND-COLOR: transparent">Scott Smith recently reached out to Don Ramsey and me regarding TBL and it's first born, FPS, (Fair Play System) which is used in IAC competitions. TBL essentially changes the scores of whole flights in toto for any given judge, while FPS takes it down a step to the individual maneuver, and changes a judge's given scores, maneuver for maneuver, as assessed by the statistical package. </FONT></div>
<div><FONT style="BACKGROUND-COLOR: transparent"></FONT> </div>
<div><FONT style="BACKGROUND-COLOR: transparent">To me, it is fundamentally wrong to artificially change a judge's score based on statistics, just to "normalize" the whole panel of judges and reduce the scoring variation, judge to judge and judge panel to judge panel. In the examples we discussed, pilots all placed the same with straight up scores and with TBL applied. But I am not convinced that that will always be true. In one example in fact, not only did the average scores each judge gave became constant, so did each judge's standard deviation. TBL is that powerful and that augmenting.....</FONT></div>
<div><FONT style="BACKGROUND-COLOR: transparent"></FONT> </div>
<div><FONT style="BACKGROUND-COLOR: transparent">I want to keep an open mind about statistical augmentation of the scores a judge gives. I am leaning towards FPS more but still, it will be judging by statistics so to speak. BTW---For this to work, Perceived Zeroes will require input from the Chief Judge in every case before the zeroes become Hard Zeroes, for all judges. Part of the existing Judge's Rules will require revision....National Comps are one thing, and local comps something else??.....</FONT></div>
<div> </div>
<div>I think there should be lots more conversation before any changes are implemented. The main question to answer is "Is the present system broken?"</div>
<div> </div>
<div>Regards</div>
<div> </div>
<div>MattK<br>
<br>
<br>
</div>
<div style="FONT-SIZE: 10pt; COLOR: black; FONT-FAMILY: arial,helvetica">-----Original Message-----<br>
From: Ryan Smith <smaragdz@comcast.net><br>
To: 'General pattern discussion' <nsrca-discussion@lists.nsrca.org><br>
Sent: Sun, Aug 18, 2013 7:51 pm<br>
Subject: Re: [NSRCA-discussion] World F3A contest<br>
<br>
<div id=AOLMsgPart_2_ce239982-3c3f-4c27-9d4f-3ccd13829666>
<div class=aolReplacedBody lang=EN-US vlink="purple" link="blue">
<div class=WordSection1>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">TBL are intials, standing for Tarasov, Bauer, Long; the people that came up with it. Below is an explanation scalped from a post/email that Derek Koopowitz wrote a while back.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">The Tarasov-Bauer-Long (TBL) Scoring method has been around since the 1970's.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">It has been used in the full size arena since 1978 and has been used at every full size IAC World Championship since 1980. The TBL method applies proven statistical probability theory to the judge's scores to resolve style differences and bias, and to avoid the inclusion of potential faulty judgements in contest results.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">Why we need TBL</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">To understand just why we need TBL, and how it works, is of considerable importance to us all. It is important to the pilots because it is there to reduce the prospect of unsatisfactory judgements affecting their results, and it is important for judges because it will introduce a completely new dimension of scrutiny into the sequence totals, and it will also discreetly engage the attention of the Chief Judge, or Contest Director, if the judges conclusions differ sufficiently from all those other judges on the same panel.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">When people get together to judge how well a pre-defined competitive task is being tackled, the range of opinions is often diverse. This is entirely natural among humans where the critique of any display of skill relies on the interpretation of rapidly changing visual cues. In order to minimize the prospect of any "way out opinions" having too much effect on the result, it is usual to average the accumulated scores to arrive at a final assessment, which takes everybody's opinion into account.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">Unfortunately this averaging approach can achieve the opposite of what we really want, which is to identify, and where needed, remove those "way out opinions" because they are the ones most likely to be ill-judged and therefore should be discarded, leaving the rest to determine the more appropriate result. In aerobatics the process of judging according to the rulebook normally leads to a series of generally similar personal views. However, one judge's downgrading may be harsher or more lenient than the next, their personal feelings toward each competitor or aircraft type may predispose toward favor or dislike (bias), and they will almost certainly miss or see things that other judges do not.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">How then can we "judge" the judges and so reach a conclusion, which has good probability of acceptance by all the concerned parties? The key word is probability, the concept of a perceived level of confidence in collectively viewed judgements has entered the frame. What we really mean is that we must be confident that opinions pitched outside some pre-defined level of reasonable acceptability will be identified as such and will not be used. This sort of situation is the daily bread and butter of well established probability theory which, when suitably applied, can produce a very clear cut analysis of numerically expressed opinions provided that the appropriate criteria have been carefully established beforehand.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">What has been developed through several previous editions is some arithmetic which addresses the judge's raw scores in such a way that any which are probably unfair are discarded with an established level of confidence. To understand the process you need only accept some quite simple arithmetic procedures, which are central to what is called "statistical probability". The TBL scoring system in effect does the following:</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">* Communizes the judging styles.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">* Computes TBL scores</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">* Publishes results</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">Communizing the judging styles involves remodelling the scores to bring all the judging styles to a common format and removing any natural bias between panel members. Following some calculations, each judge's set of scores are squeezed or stretched and moved en-bloc up or down so that the sets all show the same overall spread and have identical averages (bias). Within each set the pilot order and score progression must remain unaltered, but now valid score comparisons are possible between all the panel judges on behalf of each pilot.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">Computing the TBL score involves looking at the high and low scores in each pilot's set and throws out any that are too "far out" to be fair. This is done by subtracting the average for the set from each one and dividing the result by the "sample standard deviation" - if the result of this sum is greater than 1.645 then according to statistical probability theory we can be at least 90% confident that it is unfair, so the score is discarded.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">This calculation and the mathematically derived 1.645 criteria is the key to the correctness of the TBL process, and is based on many years of experience by the full size aerobatics organization with contest scores at all levels.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">The discarding of any scores of course changes for a pilot the average and standard deviation of their remaining results, and so the whole process is repeated. After several cycles any "unfair" scores will have gone, and those that remain will all satisfy the essential 90% confidence criteria.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">Publishing the results is derived by averaging each pilot's scores. The final TBL iteration therefore has any appropriate penalty/bonus values applied and the results are then sorted in order of descent of the total scores to rank the pilots first to last.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">These final scores may, or may not, be normalized to 1000 points, depending on the setting for the selected class. Educating and improving the judges is a useful by-product of this process in that it provides all the bells and whistles how each judge has performed by comparison with the overall judging panel average and when seen against the 90% level of confidence criteria.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">The TBL system will produce an analysis showing each judge the percentage of scores accepted as "OK", and a comparison with the panel style (spread of score) and bias (average). Unfortunately TBL, by definition, brings with it a 10% possibility of upsetting an honest judge's day. The trade-off is that we expect not only to achieve a set of results with at least 90% confidence that are "fair" every time, but that the system also provides us with a wonderful tool to address our judging standards. TBL will ensure that every judge's opinion has equal weight, and that each sequence score by each judge is accepted only if it lies within an acceptable margin from the panel average.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">TBL, however, by necessity takes the dominant judging panel view as the "correct" one and it can't make right scores out of wrong ones. If 6 out of 8 judges are distracted and make a mess out of one pilots efforts, then for TBL this becomes the controlling assessment of that pilots performance, and the other 2 diligent judges who got it right will see their scores unceremoniously zapped. In practice this would be extremely unusual - from the judging line it is almost impossible to deliberately upset the final results without collusion between a majority of the judges, and if that starts to happen then someone is definitely on the wrong planet.</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN></div>
<div>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<div class=MsoNormal><B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">From:</SPAN></B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> <A href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A> [<A href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>] <B>On Behalf Of </B>Jeff and Claire<br>
<B>Sent:</B> Sunday, August 18, 2013 6:47 PM<br>
<B>To:</B> 'General pattern discussion'<br>
<B>Subject:</B> Re: [NSRCA-discussion] World F3A contest</SPAN></div>
</div>
</div>
<div class=MsoNormal> </div>
<div class=MsoNormal><SPAN style="FONT-FAMILY: ">What does TBL stand for?</SPAN></div>
<div class=MsoNormal><SPAN style="FONT-FAMILY: "> </SPAN></div>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<div class=MsoNormal><B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">From:</SPAN></B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> <A href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A> [<A href="mailto:nsrca-discussion-bounces@lists.nsrca.org">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>] <B>On Behalf Of </B>Jon Lowe<br>
<B>Sent:</B> Sunday, August 18, 2013 3:54 PM<br>
<B>To:</B> <A href="mailto:nsrca-discussion@lists.nsrca.org">nsrca-discussion@lists.nsrca.org</A><br>
<B>Subject:</B> Re: [NSRCA-discussion] World F3A contest</SPAN></div>
</div>
<div class=MsoNormal> </div>
<div>
<div class=MsoNormal><SPAN style="FONT-FAMILY: ">After all pilots fly in front of each set of judges on day 4 of prelims, I think. That would be part of the normalization process. At least that is how I remember it from previous WC's.</SPAN></div>
</div>
<div>
<div class=MsoNormal><SPAN class=apple-style-span><SPAN style="FONT-SIZE: 13.5pt; FONT-FAMILY: ">Jon</SPAN></SPAN><SPAN style="FONT-FAMILY: "></SPAN></div>
</div>
<div>
<div class=MsoNormal style="MARGIN-BOTTOM: 12pt"><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">-----Original Message-----<br>
From: John Fuqua <<A href="mailto:johnfuqua@embarqmail.com">johnfuqua@embarqmail.com</A>><br>
To: 'General pattern discussion' <<A href="mailto:nsrca-discussion@lists.nsrca.org">nsrca-discussion@lists.nsrca.org</A>><br>
Sent: Sun, Aug 18, 2013 4:40 pm<br>
Subject: Re: [NSRCA-discussion] World F3A contest</SPAN></div>
<div id=AOLMsgPart_2_a2a2496b-ce38-4c11-adc2-eeb06634c7ba>
<div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: ">When do they do the TBL?</SPAN><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "></SPAN></div>
</div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: "> </SPAN><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "></SPAN></div>
</div>
<div style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: windowtext 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<div>
<div class=MsoNormal><B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">From:</SPAN></B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> <A href="mailto:nsrca-discussion-bounces@lists.nsrca.org">nsrca-discussion-bounces@lists.nsrca.org</A> [<A href="mailto:nsrca-discussion-bounces@lists.nsrca.org?">mailto:nsrca-discussion-bounces@lists.nsrca.org</A>] <B>On Behalf Of </B>Jon Lowe<br>
<B>Sent:</B> Sunday, August 18, 2013 3:33 PM<br>
<B>To:</B> <A href="mailto:nsrca-discussion@lists.nsrca.org">nsrca-discussion@lists.nsrca.org</A><br>
<B>Subject:</B> Re: [NSRCA-discussion] World F3A contest</SPAN></div>
</div>
</div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> </SPAN></div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">A click on the Team USA logo on the NSRCA home page takes you to the Team USA website. That has a link to Cindy Wickhizer's page:</SPAN></div>
</div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> </SPAN></div>
</div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "><A href="https://2013worldsteamusa.shutterfly.com/" target=_blank>https://2013worldsteamusa.shutterfly.com/</A>, </SPAN></div>
</div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "> </SPAN></div>
</div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">which has been where Mark is sending info. He and I both previously announced that on this list.</SPAN></div>
</div>
</div>
<div>
<div>
<div class=MsoNormal><SPAN class=apple-style-span><SPAN style="FONT-SIZE: 13.5pt; FONT-FAMILY: ">Jon</SPAN></SPAN><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: "></SPAN></div>
</div>
</div>
<div>
<div style="MARGIN-BOTTOM: 12pt">
<div class=MsoNormal><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: ">-----Original Message-----<br>
From: Gordon Seeling <<A href="mailto:gseeling@q.com">gseeling@q.com</A>><br>
To: nsrca-discussion <<A href="mailto:nsrca-discussion@lists.nsrca.org">nsrca-discussion@lists.nsrca.org</A>><br>
Sent: Sun, Aug 18, 2013 2:35 pm<br>
Subject: [NSRCA-discussion] World F3A contest</SPAN></div>
</div>
<div id=AOLMsgPart_0_b1ff61d3-f217-4b9a-9feb-d7bc8e7cd8f1><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">is there a computer man in the NSRCA ??if there is ,please post the </SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">results on the nsrca web site, so rank & file will know what is going on.</SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">_______________________________________________</SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">NSRCA-discussion mailing list</SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black"><A href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A></SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black"><A href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion" target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A></SPAN></TT><SPAN style="COLOR: black"></SPAN></PRE></div>
</div>
</div>
</div>
</div>
<div id=AOLMsgPart_3_a2a2496b-ce38-4c11-adc2-eeb06634c7ba><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">_______________________________________________</SPAN></TT></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black">NSRCA-discussion mailing list</SPAN></TT></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black"><A href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A></SPAN></TT></PRE><PRE style="BACKGROUND: white"><TT><SPAN style="COLOR: black"><A href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion" target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A></SPAN></TT></PRE></div>
</div>
</div>
</div>
</div>
<div id=AOLMsgPart_3_ce239982-3c3f-4c27-9d4f-3ccd13829666 style="FONT-SIZE: 12px; MARGIN: 0px; COLOR: #000; FONT-FAMILY: Tahoma, Verdana, Arial, Sans-Serif; BACKGROUND-COLOR: #fff"><PRE style="FONT-SIZE: 9pt"><TT>_______________________________________________
NSRCA-discussion mailing list
<A href="mailto:NSRCA-discussion@lists.nsrca.org">NSRCA-discussion@lists.nsrca.org</A>
<A href="http://lists.nsrca.org/mailman/listinfo/nsrca-discussion" target=_blank>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</A>
</TT></PRE></div>
<!-- end of AOLMsgPart_3_ce239982-3c3f-4c27-9d4f-3ccd13829666 --></div>
</font>