Scoring formula: Process, Human factors, cut for length

[George Kennie]

Boy Jim,
Our I-Pods are playing the same tune!
When I returned from the Nats I told my wife, "The only way this
thing is ever going to possibly be scored fairly is if everybody
flies the same identical aircraft and the judges are screened from
visual contact with the pilot and all competitors are recognized on
a number basis".
Had this been a possibility this year, there is no doubt in my mind
that the final result would have looked nothing like the actuality
(excepting Somenzini).
As long as we have human beings in the chair, biases ( real or
unrealized) will always prevail over reality.
Georgie

Jim_Woodward at beaerospace.com wrote:

>
> Hi All,
>
> The scoring formula thread brings in two camps of thoughts (from
> my reading) that each should be addressed.  Troy, Earl, and Eric
> have touched on the method or "process" that should go into
> scoring and tear sheets for FAI.  Troy had some great, valid
> points about the tear sheets in which I agree would lead to a more
> honest judging result, and create the "transparent" view of the
> contest to the contestants.  Besides process or following FAI
> rules, the other issue several people have tried to delicately
> step around is the topic that if one brings up, is typically
> attacked as having "sour-grapes" or upset in other ways at the
> outcome.
>
> So lets bring up the big question and debate as free-thinking
> adults:  How do we ensure that the flight in question (any class)
> is being judged purely on the technical quality of the flight
> compared against the known set of downgrades?  If the flight in
> question is judged with integrity and without prejudice, it is
> quickly recognizable as a less than perfect event.  The judging
> committee has gone a long way to create information sets and
> education regarding the rules and technical guide lines for the
> patterns and the individual maneuvers.  In the class though, we
> then typically say something like, " .... but you have to remember
> that judges are human."  While this catch all statement is used
> (in the class room setting) to present a case as for why there may
> be minor differences in scoring (judge-to-judge), we may not be
> giving the "human" factor the priority of education it deserves as
> compared to the affect this same "human" factor has on the outcome
> of the event.  How do we ensure that each flight is being scored
> on its quality and merits, without regard to any additional
> factors?  How do we ensure that any institutional or historical
> bias does not turn one fliers "7" into an "8", and another's "7"
> into a "6"?  We have taken the "technical" education pretty far -
> but how can human behavior  be changed?
>
> Ideally, the judges should not even "know" who is flying which is
> to say again, that the score is not based on the flyer, but the
> quality of the flight.  For example:  Should the pilots all fly
> the same plane, with the same color scheme, with the judges
> shielded from view of the identity of the pilot?  I doubt this
> would go over well for many reasons however - one reason is that
> recognition for past performance is a key factor in today's or
> future results.  Thus, each pilot with excellent or poor prior
> results is at advantage or disadvantage to have the judges to know
> that it is indeed, them who is flying.  Opposition to such on the
> high end in regards to scoring, is evidence of this "X" factor in
> the overall final results.  I'm not advocating a 1-design class,
> just using it for example.  How do we ensure that judges are
> evaluating the flight on its merits alone, and not according to
> additional factors such as (name your favorite).  The additional
> human factors if brought to the chair,  are worthy of discussion.
> Heres a question:  Do you want habitual long standing judges with
> years of experience in the FAI/Masters finals chair, or somewhat
> newer judges who are technically educated but bring less prior
> experience to the chair of this same group of competitors?
>
> Again, the point is process (in the FAI since following what rules
> are there), and human, ensuring the each flight is evaluated
> solely on the its own merit, as unique discreet events without
> regard to past performance or future expectation.  However, from
> the previous threads, it appears that "Advanced" suffers from a
> lack of technical knowledge in the chair where as the other end of
> the spectrum (FAI) can suffer from the human factor part of the
> equation.
>
> All ideas welcome,
> Jim W.
>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.f3a.us/pipermail/nsrca-discussion/attachments/20050801/22a1fecc/attachment.html