[NSRCA-discussion] Judging

Wed Oct 17 12:19:25 AKDT 2007

In my opinion, that only says that you can't use them to indicate a
violation - nothing about using a visual reference to make a judgment. In
other words, you can't blow a whistle after the plane flies out of the box.

Jay Marshall 

-----Original Message-----
From: nsrca-discussion-bounces at lists.nsrca.org
[mailto:nsrca-discussion-bounces at lists.nsrca.org] On Behalf Of Earl Haury
Sent: Wednesday, October 17, 2007 4:02 PM
To: NSRCA Mailing List
Subject: Re: [NSRCA-discussion] Judging

Hi Georgie

In the F3A rules currently there is the sentence "Audible and visual signals
to indicate violations of the maneuvering zone are not to be employed".
Sporting Code. Section C, Part Five, specifically 5.1.8. I believe that all
such wording is now gone from AMA rules.

Earl

----- Original Message ----- 

From: george w. kennie <mailto:geobet at gis.net>  

To: NSRCA Mailing List <mailto:nsrca-discussion at lists.nsrca.org>  

Sent: Wednesday, October 17, 2007 1:06 PM

Subject: Re: [NSRCA-discussion] Judging

 <<<<<<<Various wording in both AMA & F3A rules have prohibited judging
"aids". >>>>>>

It would be very helpful to me if someone could point me to this reference
in the rulebook as I have never been able to find it.

Georgie

----- Original Message ----- 

From: Earl Haury <mailto:ejhaury at comcast.net>  

To: NSRCA Mailing List <mailto:nsrca-discussion at lists.nsrca.org>  

Sent: Wednesday, October 17, 2007 12:25 PM

Subject: Re: [NSRCA-discussion] Judging

Some thoughts that might help unburden judges and improve accuracy. These
are outside the box a bit - process box, not flight box. As some have
mentioned, replacing human judges with some form of computer scoring system
is the ultimate answer. I hope I live long enough to see that work, not that
it's impossible now - just no one with the interest / skills / finances has
approached it. Much time has been spent discussing ways to transfer the
score from the judges mind to paper - but, guess what, a pencil and paper
works just fine! (It's not even too hard to process scores with a
calculator!)

F3A rules preclude the use of means to define box violations other than the
judge's observation. Various wording in both AMA & F3A rules have prohibited
judging "aids". This seems contradictory to the purpose! A pilot is supposed
to demonstrate skill in flying an aircraft within the constraints of the box
with perfection being the goal - while being judged by a bunch of
ill-positioned folks who vary in being able to determine distance +- 50
meters? In the days of interrogated circuits, dual conversation RX, and
giggle Hertz freq we still choose to rely on guesstimates for distance!
Nonsense. Very little effort would be required to provide accurate excessive
distance and box excursion information. Take this burden from the judge and
apply any distance / box downgrades post flight. Sure - I don't know just
what these machines are at the moment (could be just properly placed people
in major meets) - but asking the question may get somebody thinking.

If the pilot is expected to display perfection in flight - we should move
into the 21st century in devising ways of accurately judging whether or not
that perfection is present. Of course it might cost some of us judges a job
- darn, I would hate to lose the income!

Earl

----- Original Message ----- 

From: Earl Haury <mailto:ejhaury at comcast.net>  

To: Discussion List, NSRCA <mailto:nsrca-discussion at lists.nsrca.org>  

Sent: Wednesday, October 17, 2007 10:29 AM

Subject: [NSRCA-discussion] Judging

Allow me to start a new thread for the purpose of analyzing the issues Jim
and others highlighted in the D3 thread. Most of us have been frustrated
over the years by inaccurate scoring, both high and low. Forget the notion
that it doesn't matter if an inaccurate judge is consistent - that person is
just consistently wrong and it does matter, the rules require both accurate
and consistent scoring. I also don't believe that judging ability depends
entirely on class flown, masters and FAI folks aren't inherently smarter
than others. Experience does improve accuracy and it's important to know
what maneuvers / schedule that will be judged (called preparation). However,
it's not important what class is flying the maneuver - a half loop or
immelman or stall turn is the same in intermediate as F3A. The NSRCA Judge
Cert program has improved the quality of judging immensely! So - now that
I've gotten these generalities out of my system, let's take a look at why
F3A scores may vary a lot by judge / region.

I've heard the opinion expressed that the whole point scoring in F3A
dictates that no downgrade is applied until an error of 15 deg or more is
observed. Conversely, others feel that any error in F3A requires, at least,
a one point downgrade. Hmmmm - that'll make a difference!  F3A adopted whole
point scoring in an effort to force judges to use the entire 10 - 0 range of
scores, rather than the upper 3 or so as was typical. This is probably where
the 1pt for any error notion comes from.  But it was difficult to quantify
how much to downgrade many errors, and a wide variation occurred between
judges of equal skill, some saw a 5 deg error worthy of a point deduction -
others would see a 30 deg error as worthy of only a point. Probably the most
useful metric available to judges is the 1/15 rule! 

However, the 1/15 rule fails to define just what should be done for errors
of less than 15 deg. Honest differences of opinion exist and these become
more important the better a maneuver is flown. I suspect that some
evaluation of the wording of the rules might help. F3A requires "marking"
(scoring) in whole points, but uses the word "downgrade" regarding the
judges assessment of the "mark". While examples of egregious errors are
noted in whole points, there is no exclusion forbidding the judge to use
smaller downgrade increments to arrive at the whole point score. So why is
the downgrade for errors smaller than 15 deg undefined? Well - pattern folks
are certainly smart (or we wouldn't be doing this - right?) and have no
problem recognizing the downgrade applicable to 30, 45, etc. errors basis
the 1/15 metric, there shouldn't be any difficulty in the other direction
either as 5 deg = 1/3 pt, 7.5 = 1/2 pt, etc. 

A problem arises when a judge is between whole points with the proper
downgrades. (Things would be a lot easier if F3A adopted 1/2 pt scoring -
I've made the arguments and some are listening - but don't expect a change
any time soon.) The scenario might be a simple turnaround maneuver with a
slight 5 deg error of some sort which deserves a 1/3 pt downgrade. Some will
score this a 10, others a 9. The F3A rules dwell on major defects and leave
these situations nearly unaddressed. Consider that a 9 is unfair - might as
well make a 15 deg error. Some will say a 10 is unfair as the maneuver is
imperfect and we are striving for perfection. OK - the F3A rules state "A
high score should be given only if no major defects are found and the
maneuver is well positioned." You decide - I would probably go with a 10, as
there's no "major" defect, and feel comfortable rounding to the nearest
whole number. Unfortunately, unintentional bias (basis pilot reputation,
quality of current flight, etc.) can slip in here and result in like
maneuvers being rounded up and down for different pilots - here's a place
where judge consistency must be applied. 

Centering is another area where downgrades often vary in that some focus on
a center "key point" of a maneuver and downgrade heavily if that point is
missed. F3A rules state "This may be in the range of 1 to 4 points
subtracted"  with regards to centering errors, without defining a metric.
Most assume 1 pt / 25%. OK - the middle of the inverted portion of a 4 pt
roll is way off - the 90 deg roll ends at the pole - most would ding this a
couple of points. But, a 4 pt roll may be 1000 ft long, so a 2 pt downgrade
would be appropriate for a 500 ft error, the example is probably less than
200 feet - so a 1pt downgrade would be more appropriate. If the overall
length of a maneuver is considered, centering downgrades are often less than
scored. This concept is even more important for "narrow" center maneuvers.

So - there are a couple of places where judges can disagree in scoring and
these will generate large differences in scores by flight and, possibly, by
region. Our NSRCA judging program has done and is doing a good job of
ensuring we all recognize errors. The 1/15 rule provides a good metric and
works well with the AMA 1/2 point scoring system.  This same 1/15 metric
leaves us hanging a bit in F3A when used with whole point scoring. Maybe a
solution is for us to use 1/2 pt. scoring in F3A events in AMA contests.
Certainly we might include direction in our judging program to ensure folks
judging F3A handle this issue consistently.

Earl

  _____  

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

  _____  

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

  _____  

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20071017/438b226c/attachment-0001.html 