<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.6000.16546" name=GENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
<DIV><FONT face=Arial>Some thoughts that might help unburden judges and improve
accuracy. These are outside the box a bit - process box, not flight box. As
some have mentioned, replacing human judges with some form of computer scoring
system is the ultimate answer. I hope I live long enough to see that work, not
that it's impossible now - just no one with the interest / skills / finances has
approached it. Much time has been spent discussing ways to transfer the score
from the judges mind to paper - but, guess what, a pencil and paper works just
fine! (It's not even too hard to process scores with a calculator!)</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>F3A rules preclude the use of means to define box
violations other than the judge's observation. Various wording in both AMA &
F3A rules have prohibited judging "aids". This seems contradictory to the
purpose! A pilot is supposed to demonstrate skill in flying an aircraft within
the constraints of the box with perfection being the goal - while being judged
by a bunch of ill-positioned folks who vary in being able to determine distance
+- 50 meters? In the days of interrogated circuits, dual conversation RX, and
giggle Hertz freq we still choose to rely on guesstimates for distance!
Nonsense. Very little effort would be required to provide accurate excessive
distance and box excursion information. Take this burden from the judge and
apply any distance / box downgrades post flight. Sure - I don't know just what
these machines are at the moment (could be just properly placed people in major
meets) - but asking the question may get somebody thinking.</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>If the pilot is expected to display perfection in flight -
we should move into the 21st century in devising ways of accurately judging
whether or not that perfection is present. Of course it might cost some of us
judges a job - darn, I would hate to lose the income!</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>Earl</FONT></DIV>
<BLOCKQUOTE
style="PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV style="FONT: 10pt arial">----- Original Message ----- </DIV>
<DIV
style="BACKGROUND: #e4e4e4; FONT: 10pt arial; font-color: black"><B>From:</B>
<A title=ejhaury@comcast.net href="mailto:ejhaury@comcast.net">Earl Haury</A>
</DIV>
<DIV style="FONT: 10pt arial"><B>To:</B> <A
title=nsrca-discussion@lists.nsrca.org
href="mailto:nsrca-discussion@lists.nsrca.org">Discussion List, NSRCA</A>
</DIV>
<DIV style="FONT: 10pt arial"><B>Sent:</B> Wednesday, October 17, 2007 10:29
AM</DIV>
<DIV style="FONT: 10pt arial"><B>Subject:</B> [NSRCA-discussion] Judging</DIV>
<DIV><BR></DIV>
<DIV><FONT face=Arial>Allow me to start a new thread for the purpose of
analyzing the issues Jim and others highlighted in the D3 thread. Most of
us have been frustrated over the years by inaccurate scoring, both high
and low. Forget the notion that it doesn't matter if an inaccurate judge is
consistent - that person is just consistently wrong and it does matter, the
rules require both accurate and consistent scoring. I also don't believe
that judging ability depends entirely on class flown, masters and FAI
folks aren't inherently smarter than others. Experience does improve accuracy
and it's important to know what maneuvers / schedule that will be judged
(called preparation). However, it's not important what class is flying the
maneuver - a half loop or immelman or stall turn is the same in intermediate
as F3A. The NSRCA Judge Cert program has improved the quality of judging
immensely! So - now that I've gotten these generalities out of my system,
let's take a look at why F3A scores may vary a lot by judge /
region.</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>I've heard the opinion expressed that the whole point
scoring in F3A dictates that no downgrade is applied until an error of 15 deg
or more is observed. Conversely, others feel that any error in F3A requires,
at least, a one point downgrade. Hmmmm - that'll make a difference! F3A
adopted whole point scoring in an effort to force judges to use the entire 10
- 0 range of scores, rather than the upper 3 or so as was typical. This is
probably where the 1pt for any error notion comes from. But it was
difficult to quantify how much to downgrade many errors, and a wide variation
occurred between judges of equal skill, some saw a 5 deg error worthy of a
point deduction - others would see a 30 deg error as worthy of only a point.
Probably the most useful metric available to judges is the 1/15 rule!
</FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>However, the 1/15 rule fails to define just what should
be done for errors of less than 15 deg. Honest differences of opinion exist
and these become more important the better a maneuver is flown. I suspect that
some evaluation of the wording of the rules might help. F3A requires "marking"
(scoring) in whole points, but uses the word "downgrade" regarding the judges
assessment of the "mark". While examples of egregious errors are noted in
whole points, there is no exclusion forbidding the judge to use smaller
downgrade increments to arrive at the whole point score. So why is the
downgrade for errors smaller than 15 deg undefined? Well - pattern folks are
certainly smart (or we wouldn't be doing this - right?) and have no problem
recognizing the downgrade applicable to 30, 45, etc. errors basis the 1/15
metric, there shouldn't be any difficulty in the other direction either
as 5 deg = 1/3 pt, 7.5 = 1/2 pt, etc. </FONT></DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial>A problem arises when a judge is between whole points
with the proper downgrades. (Things would be a lot easier if F3A adopted 1/2
pt scoring - I've made the arguments and some are listening - but don't expect
a change any time soon.) The scenario might be a simple turnaround maneuver
with a slight 5 deg error of some sort which deserves a 1/3 pt downgrade. Some
will score this a 10, others a 9. The F3A rules dwell on major defects and
leave these situations nearly unaddressed. Consider that a 9 is unfair - might
as well make a 15 deg error. Some will say a 10 is unfair as the maneuver is
imperfect and we are striving for perfection. OK - the F3A rules state "<SPAN
lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=3>A high score should be given only if no major defects are found and the
maneuver is well positioned." <FONT face=Arial>You decide - I would probably
go with a 10, as there's no "major" defect, and feel comfortable rounding to
the nearest whole number. Unfortunately, unintentional bias (basis pilot
reputation, quality of current flight, etc.) can slip in here and result in
like maneuvers being rounded up and down for different pilots - here's a place
where judge consistency must be applied. </FONT></FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=3><FONT face=Arial></FONT></FONT></SPAN></FONT> </DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=+0><FONT face=Arial><FONT size=3>Centering is another area where
downgrades often vary in that some focus on a center "key point" of a maneuver
and downgrade heavily if that point is missed. F3A rules state "<SPAN
lang=EN-GB><FONT face="Times New Roman">This may be in the range of 1 to 4
points subtracted" </FONT><FONT face=Arial>with regards to centering
errors, without defining a metric. Most assume 1 pt / 25%. OK - the middle of
the inverted portion of a 4 pt roll is way off - the 90 deg roll ends at the
pole - most would ding this a couple of points. But, a 4 pt roll may be 1000
ft long, so a 2 pt downgrade would be appropriate for a 500 ft error, the
example is probably less than 200 feet - so a 1pt downgrade would be more
appropriate. If the overall length of a maneuver is considered, centering
downgrades are often less than scored. This concept is even more important for
"narrow" center
maneuvers.</FONT></SPAN></FONT></FONT></FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=+0><FONT face=Arial><FONT size=3><SPAN
lang=EN-GB></SPAN></FONT></FONT></FONT></SPAN></FONT> </DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=+0><FONT face=Arial><FONT size=3><SPAN lang=EN-GB>So - there are a couple
of places where judges can disagree in scoring and these will generate large
differences in scores by flight and, possibly, by region. Our NSRCA judging
program has done and is doing a good job of ensuring we all recognize errors.
The 1/15 rule provides a good metric and works well with the AMA 1/2 point
scoring system. This same 1/15 metric leaves us hanging a bit in F3A
when used with whole point scoring. Maybe a solution is for us to use 1/2 pt.
scoring in F3A events in AMA contests. Certainly we might include direction in
our judging program to ensure folks judging F3A handle this issue
consistently.</SPAN></FONT></FONT></FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=+0><FONT face=Arial><FONT size=3><SPAN
lang=EN-GB></SPAN></FONT></FONT></FONT></SPAN></FONT> </DIV>
<DIV><FONT face=Arial><SPAN lang=EN-GB
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-GB; mso-fareast-language: FR; mso-bidi-language: AR-SA"><FONT
size=+0><FONT face=Arial><FONT size=3><SPAN
lang=EN-GB>Earl</SPAN></FONT></FONT></FONT></SPAN></FONT></DIV>
<P>
<HR>
<P></P>_______________________________________________<BR>NSRCA-discussion
mailing
list<BR>NSRCA-discussion@lists.nsrca.org<BR>http://lists.nsrca.org/mailman/listinfo/nsrca-discussion</BLOCKQUOTE></BODY></HTML>