[NSRCA-discussion] Judging

Wed Oct 17 10:06:29 AKDT 2007

 <<<<<<<Various wording in both AMA & F3A rules have prohibited judging "aids". >>>>>>

It would be very helpful to me if someone could point me to this reference in the rulebook as I have never been able to find it.
Georgie

  ----- Original Message ----- 
  From: Earl Haury 
  To: NSRCA Mailing List 
  Sent: Wednesday, October 17, 2007 12:25 PM
  Subject: Re: [NSRCA-discussion] Judging

  Some thoughts that might help unburden judges and improve accuracy. These are outside the box a bit - process box, not flight box. As some have mentioned, replacing human judges with some form of computer scoring system is the ultimate answer. I hope I live long enough to see that work, not that it's impossible now - just no one with the interest / skills / finances has approached it. Much time has been spent discussing ways to transfer the score from the judges mind to paper - but, guess what, a pencil and paper works just fine! (It's not even too hard to process scores with a calculator!)

  F3A rules preclude the use of means to define box violations other than the judge's observation. Various wording in both AMA & F3A rules have prohibited judging "aids". This seems contradictory to the purpose! A pilot is supposed to demonstrate skill in flying an aircraft within the constraints of the box with perfection being the goal - while being judged by a bunch of ill-positioned folks who vary in being able to determine distance +- 50 meters? In the days of interrogated circuits, dual conversation RX, and giggle Hertz freq we still choose to rely on guesstimates for distance! Nonsense. Very little effort would be required to provide accurate excessive distance and box excursion information. Take this burden from the judge and apply any distance / box downgrades post flight. Sure - I don't know just what these machines are at the moment (could be just properly placed people in major meets) - but asking the question may get somebody thinking.

  If the pilot is expected to display perfection in flight - we should move into the 21st century in devising ways of accurately judging whether or not that perfection is present. Of course it might cost some of us judges a job - darn, I would hate to lose the income!

  Earl
    ----- Original Message ----- 
    From: Earl Haury 
    To: Discussion List, NSRCA 
    Sent: Wednesday, October 17, 2007 10:29 AM
    Subject: [NSRCA-discussion] Judging

    Allow me to start a new thread for the purpose of analyzing the issues Jim and others highlighted in the D3 thread. Most of us have been frustrated over the years by inaccurate scoring, both high and low. Forget the notion that it doesn't matter if an inaccurate judge is consistent - that person is just consistently wrong and it does matter, the rules require both accurate and consistent scoring. I also don't believe that judging ability depends entirely on class flown, masters and FAI folks aren't inherently smarter than others. Experience does improve accuracy and it's important to know what maneuvers / schedule that will be judged (called preparation). However, it's not important what class is flying the maneuver - a half loop or immelman or stall turn is the same in intermediate as F3A. The NSRCA Judge Cert program has improved the quality of judging immensely! So - now that I've gotten these generalities out of my system, let's take a look at why F3A scores may vary a lot by judge / region.

    I've heard the opinion expressed that the whole point scoring in F3A dictates that no downgrade is applied until an error of 15 deg or more is observed. Conversely, others feel that any error in F3A requires, at least, a one point downgrade. Hmmmm - that'll make a difference!  F3A adopted whole point scoring in an effort to force judges to use the entire 10 - 0 range of scores, rather than the upper 3 or so as was typical. This is probably where the 1pt for any error notion comes from.  But it was difficult to quantify how much to downgrade many errors, and a wide variation occurred between judges of equal skill, some saw a 5 deg error worthy of a point deduction - others would see a 30 deg error as worthy of only a point. Probably the most useful metric available to judges is the 1/15 rule! 

    However, the 1/15 rule fails to define just what should be done for errors of less than 15 deg. Honest differences of opinion exist and these become more important the better a maneuver is flown. I suspect that some evaluation of the wording of the rules might help. F3A requires "marking" (scoring) in whole points, but uses the word "downgrade" regarding the judges assessment of the "mark". While examples of egregious errors are noted in whole points, there is no exclusion forbidding the judge to use smaller downgrade increments to arrive at the whole point score. So why is the downgrade for errors smaller than 15 deg undefined? Well - pattern folks are certainly smart (or we wouldn't be doing this - right?) and have no problem recognizing the downgrade applicable to 30, 45, etc. errors basis the 1/15 metric, there shouldn't be any difficulty in the other direction either as 5 deg = 1/3 pt, 7.5 = 1/2 pt, etc. 

    A problem arises when a judge is between whole points with the proper downgrades. (Things would be a lot easier if F3A adopted 1/2 pt scoring - I've made the arguments and some are listening - but don't expect a change any time soon.) The scenario might be a simple turnaround maneuver with a slight 5 deg error of some sort which deserves a 1/3 pt downgrade. Some will score this a 10, others a 9. The F3A rules dwell on major defects and leave these situations nearly unaddressed. Consider that a 9 is unfair - might as well make a 15 deg error. Some will say a 10 is unfair as the maneuver is imperfect and we are striving for perfection. OK - the F3A rules state "A high score should be given only if no major defects are found and the maneuver is well positioned." You decide - I would probably go with a 10, as there's no "major" defect, and feel comfortable rounding to the nearest whole number. Unfortunately, unintentional bias (basis pilot reputation, quality of current flight, etc.) can slip in here and result in like maneuvers being rounded up and down for different pilots - here's a place where judge consistency must be applied. 

    Centering is another area where downgrades often vary in that some focus on a center "key point" of a maneuver and downgrade heavily if that point is missed. F3A rules state "This may be in the range of 1 to 4 points subtracted"  with regards to centering errors, without defining a metric. Most assume 1 pt / 25%. OK - the middle of the inverted portion of a 4 pt roll is way off - the 90 deg roll ends at the pole - most would ding this a couple of points. But, a 4 pt roll may be 1000 ft long, so a 2 pt downgrade would be appropriate for a 500 ft error, the example is probably less than 200 feet - so a 1pt downgrade would be more appropriate. If the overall length of a maneuver is considered, centering downgrades are often less than scored. This concept is even more important for "narrow" center maneuvers.

    So - there are a couple of places where judges can disagree in scoring and these will generate large differences in scores by flight and, possibly, by region. Our NSRCA judging program has done and is doing a good job of ensuring we all recognize errors. The 1/15 rule provides a good metric and works well with the AMA 1/2 point scoring system.  This same 1/15 metric leaves us hanging a bit in F3A when used with whole point scoring. Maybe a solution is for us to use 1/2 pt. scoring in F3A events in AMA contests. Certainly we might include direction in our judging program to ensure folks judging F3A handle this issue consistently.

    Earl

----------------------------------------------------------------------------

    _______________________________________________
    NSRCA-discussion mailing list
    NSRCA-discussion at lists.nsrca.org
    http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

------------------------------------------------------------------------------

  _______________________________________________
  NSRCA-discussion mailing list
  NSRCA-discussion at lists.nsrca.org
  http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20071017/103a6a5d/attachment.html 