[NSRCA-discussion] Was Bakersfield contest - now discussion on normalization.

Tue May 6 13:16:53 AKDT 2014

Better late than never, maybe!
I caught bits and pieces of this discussion while in Canada last week.
I believe I was district VP in the mid 80's when we committed to the present
normalizing method. Not wanting my scores based on someone else's
performance at the time I wasn't in favor of normalizing. About that time
scores from the NATs were published, both raw and normalized. After
calculating placement based on raw scores, as I remember, 1 st through 4 th
place didn't change. I couldn't argue against normalized scoring then nor
can I now.
All things considered our present normalizing method with throwaway rounds
probably works best for most competitors.
Some of us just need to fly better.
I wish judging was as easy.
Jim

-----Original Message-----
From: nsrca-discussion-bounces at lists.nsrca.org
[mailto:nsrca-discussion-bounces at lists.nsrca.org]On Behalf Of
rcmaster199 at aol.com
Sent: Wednesday, April 30, 2014 8:15 AM
To: nsrca-discussion at lists.nsrca.org
Subject: Re: [NSRCA-discussion] Was Bakersfield contest - now discussion on
normalization.

Normalizing to average can affect every pilot in the event of a truncated
flight or two.

The low score of the short flights figure in the average, lowering it. The
guys with the flame outs of course are affected but everyone else is also
affected in a positive way, increasing their respective scores
proportionately. How large the increase is depends on number of pilots

Normalization to max score affects only the truncated flights and no one
else. I think that's what we want.

MattK

-----Original Message-----
From: Atwood, Mark <atwoodm at paragon-inc.com>
To: General pattern discussion <nsrca-discussion at lists.nsrca.org>
Sent: Wed, Apr 30, 2014 2:44 am
Subject: [NSRCA-discussion] Was Bakersfield contest - now discussion on
normalization.
The reality is that "most" of the time, all of these methods will get it
right.  All of them how outliers that demonstrate their flaws.

The question becomes what flaws are more egregious and lead to bad outcomes.
Raw scoring, and normalizing to the perfect score are virtually the same
thing and will mathematically result in the same outcome.  It does not
normalize the rounds to be equal in weight and allows for a "blowout" round
to offset multiple close rounds.   There's a myriad of reasons we no longer
use raw scores.  But even raw scores provided the right outcome "most" of
the time.

Normalizing to the highest score has been the accepted way of late. It has
the significant advantage of making each round equal in value.  It suffers
most notably when one or several pilots miss a round ( for whatever reason).
This very very rarely changes the outcome of the 1st place winner, but can
significantly alter the lower standings.  This is most noticed at nationals
when people are fighting for a position in the semi finals or finals.

Normalizing to the average does the reverse.  Still maintains equal value in
each round but Improves the accuracy of the middle outcomes at the expense
of a possibly less accurate 1st place outcome.  This is appealing if your
fighting for the finals cutoff but I think most would agree that at the end
of the day we're trying to pick the winner.

With that in mind, I think we currently have the correct normalization
process.

Mike'a concern of throwing away too much data is a very valid, albeit
completely different issue.  That to me is less of a mathematical issue than
a potentially contentious discussion about the subjectivity of our
discipline.  The argument to throw out highs and lows (I disagree with doing
this) is based fundamentally on the assumption that those scores are wrong.
The 10 was erroneous or the 0 was erroneous etc.  In practice this only
serves to call into question the quality of the judge.  It was meant to
counter the problem with regional "style" especially with regard to snaps
and spins, where a spurious Zero could ruin a round.  We've done a LOT to
reduce these problems over the years and I think our overall judging is the
best it's ever been.

Keeping rounds is slightly different in its intent.  Only part of that is
meant to address equipment reliability.  Many many many sports reward
consistency and still allow for a misstep.  Athletic sports typically play
series... Best 3of5, 4 of 7, etc.   Subjective sport that don't do this
(skating, gymnastics, etc) also don't have you perform more than once.    We
are a hybrid of the two.

My $0.02

Mark

Sent from my average intelligence  phone

On Apr 29, 2014, at 6:02 PM, "Jas S" < justanotherflyr at gmail.com
<mailto:justanotherflyr at gmail.com> > wrote:
Can the results be run normalized like normal to see if there are any
changes? Would be interesting to see if anything changes. My guess is it
probably won't, but might show any kind of difference between doing it this
way and the other way.

On Tue, Apr 29, 2014 at 4:12 PM, Del R < drykert2 at rochester.rr.com
<mailto:drykert2 at rochester.rr.com> > wrote:
Hi Mike.. While I concede it has rarely happened in the past but their have
been repeated occurrences d/t possible judge overload, Miss understanding of
maneuver being flown etc. some have earned positive scores when a zero was
deserved. My gripe with the current system especially at the NATS settings
is when the lowest scoring judge is look at with suspicion when they are
they only one who catches a zero maneuver. On the surface of the data it
appears to be the bad judge. In actuality the other judges missed the zero.
Again Compromise. I accept it is never done intentionally by anyone but when
judges work harder than pilots it makes me question the system that is at
times rewarding the wrong apple. Now I'm fairly confident in the overall
scheme of average flying over the season the consistently best pilot is
still truly the one who wins most often. My issue is the lack of
acknowledgment by those that control the direction of P.A. at times are
willing to throw out the baby with the bath water. The drive to constantly
make maneuvers more complex to challenge the pilots is rarely evaluated to
what it does to the people that are required to become familiar with the
constantly changing schedule depending on which class they are to be called
on to judge. Some have had to judge 3 and in rare cases 4 other classes
besides the one they are flying in. That isn't ideal either. With everything
being purely voluntary and some have to leave part way through without
judging skews the load. In the past some would even show up late to register
to attempt to avoid judging at all. If judging has become that difficult
then the community should be taking a hard look at how fledgling pilots in
the lower classes handle and accept those challenges. While some can accept
it quite readily others are not so adept. Do we slam the door on their
desire to compete and attend more contests. I have never seen the community
reverse any of it major changes when it has hurt attendance. The priority is
always for the betterment of FAI pilots in the WORLDS. While those are
admiral goals of the organization my contention is if they want to encourage
participation they need to change some of their priorities a little.

    Del
----- Original Message -----
From: Dr. Michael Harrison, DDS <mailto:drmikedds at sbcglobal.net>
To: 'General pattern discussion' <mailto:nsrca-discussion at lists.nsrca.org>
Sent: Tuesday, April 29, 2014 3:14 PM
Subject: Re: [NSRCA-discussion] Bakersfield Bashscores areon the NSRCA
website...

and  data we are picking may not be the right data or we are throwing away
good data and possibly picking the wrong competitor,,,

From: nsrca-discussion-bounces at lists.nsrca.org
<mailto:nsrca-discussion-bounces at lists.nsrca.org>  [mailto:
nsrca-discussion-bounces at lists.nsrca.org
<mailto:nsrca-discussion-bounces at lists.nsrca.org> ] On Behalf Of Del R
Sent: Tuesday, April 29, 2014 12:47 PM
To: General pattern discussion
Subject: Re: [NSRCA-discussion] Bakersfield Bash scores areon the NSRCA
website...

Mike.. Over the years many of the negative down sides have been repeatedly
pointed out regarding using the current system and while I truly concur with
your opinion the ruling part sees nothing wrong with throwing away some
rounds which is against the very attribute they contend to support. If the
overall best average pilot shows up each round and does their best then they
would secure their place. Belittling any judges work manning the chair by
throwing away their work is counter-productive to the hard work they put in
with their calibrated eyeballs. It is all about what is an acceptable
compromise. In a perfect world all judges would be focused and catch every
zero but when the maneuvers are as complex as they have become it is a
daunting task for new pilots to master let alone feel comfortable about
advancing to the point they have to man the chair to compete.  The sport has
become so complex one needs professional caller and when you have maintaince
issues to tend to while having to go man the chair makes it less than
desirable. Maybe that is why many are glad to have the choice of throwing
one of their flights away.

    Del
----- Original Message -----
From: Dr. Michael Harrison, DDS <mailto:drmikedds at sbcglobal.net>
To: 'General pattern discussion' <mailto:nsrca-discussion at lists.nsrca.org>
Sent: Tuesday, April 29, 2014 11:51 AM
Subject: Re: [NSRCA-discussion] Bakersfield Bash scores areon the NSRCA
website...

My 2-1=1 cents,  There is no perfect scoring system.  The system used at
bakersfield regresses back to winning with a " BarnBurner score" on the last
flight of the contest and this suffers the dilemma of becoming the
popularity contest we used to deal with.  Normalizing normally rewards the
consistently best pilot, however there are always exceptions and extreme
examples to the contrary.  This holds true in all competitive events of any
type.

I believe where we fall short today is that we throw away too much data.
Examples would be dropping the high and low score of each flight, dropping
the worst flight at a contest, counting only 4 flights of contests with more
than 4 flights, throwing away previous flights at major events, ie.
eliminating the prelims scores at major events, etc.

With the reliability of all our equipment and the technology present today,
we should do more to retain more of the judging we have done.

More and more is being done via various methods(TBL, dropping high and low
scores, dropping worst flights, etc) to create an "average score" which is
erroneously considered the best most fair score.  I believe this is a
mistake.  I believe we should keep the high and low scores, the worst
flights, whatever.  We don't really have the statistical data to do TBL
properly, in my opinion, it should be eliminated.  It definitely has an
adverse affect on the judges ability to do his job.

I believe we should work to keep much more data.  I believe we can create
systems to do that.

On another note, I believe this is one really good contest, would love to
have been there.  Probably the premier "local" contest of the nation.

Mike
On Apr 29, 2014, at 8:19 AM, ronlock at comcast.net
<mailto:ronlock at comcast.net>  wrote

Ron Lockhart
  _____

From: "Mark Atwood" < atwoodm at paragon-inc.com
<mailto:atwoodm at paragon-inc.com> >
To: "General pattern discussion" < nsrca-discussion at lists.nsrca.org
<mailto:nsrca-discussion at lists.nsrca.org> >
Sent: Tuesday, April 29, 2014 2:01:04 AM
Subject: Re: [NSRCA-discussion] Bakersfield Bash scores are on the
NSRCA        website...
It does not normalize the rounds.  One round can be worth more than another.
Easy vs hard judges, calm vs windy conditions, etc.  Especially with throw
aways it's critical that all the round have the same "value".

That said, I can see an interest in normalizing to the average score, rather
than to the top score.  It minimizes some of the variance when a key flyer
doesn't fly the round either due to a problem or simply choosing not to fly
after winning the first 4 rounds.  Particularly in that last scenario, it
makes the remaining rounds much more valuable to the 2nd and 3rd place
flyers.

Sent from my average intelligence  phone

On Apr 28, 2014, at 8:27 PM, "Peter Vogel" < vogel.peter at gmail.com
<mailto:vogel.peter at gmail.com> > wrote:
Well, it's normalization of a sort, in that it's a percentage of perfect
within each class, so a 984 in Intermediate and a 984 in Sportsman mean the
same thing, you flew 98.4% of the perfect score (no, I haven't seen a score
that high! :-)

Overall though, I don't like the dynamic of it.  It's very clear when you
have a highly critical set of judges, everyone pretty much drops that round.
Normalization to best is definitely a superior way for people to see how
they are doing relative to the best pilot in the round for a particular set
of judges, regardless of how critical (or not) they happen to be.  An
interesting experiment, but I wouldn't recommend it for future contests.

Peter+

On Mon, Apr 28, 2014 at 7:08 PM, John Gayer < jgghome at comcast.net
<mailto:jgghome at comcast.net> > wrote:
It is a misnomer to call what was used normalization. It is actually a
reversion to the pre-normalization scoring scheme of yesteryear. The only
difference from the old days is that by "normalizing" to a perfect score,
you can immediately see the average raw score per maneuver.
John

On 4/28/2014 6:15 PM, Anthony Romano wrote:
So why the change in normalization?

Anthony
  _____

Date: Mon, 28 Apr 2014 17:04:28 -0700
From:  vogel.peter at gmail.com <mailto:vogel.peter at gmail.com>
To:  nsrca-discussion at lists.nsrca.org
<mailto:nsrca-discussion at lists.nsrca.org>
Subject: Re: [NSRCA-discussion] Bakersfield Bash scores are on the NSRCA
website...
Final data from the contest is also available here:

http://www.patternscoring.com/560, the per-contestant report is considerably
richer than what you get on  nsrca.us <http://nsrca.us/>

I also just completed an analysis of what would have happened if we'd
normalized to best instead of to perfect, the only class where it would have
made a difference in final placement was FAI Silver (MP15) where Jon's win
in one round and 2nd place rounds were closer to Sean's scores than Dale's
2nd place rounds would have put him about 30 points over Dale.

Peter+

On Mon, Apr 28, 2014 at 11:17 AM, Peter Vogel < vogel.peter at gmail.com
<mailto:vogel.peter at gmail.com> > wrote:
That's not the scoring program, the XML says "Prelim", "1", "2" for the
flight number, but the web server's transformation is showing it as "1", "2"
and "0".

Peter+

On Mon, Apr 28, 2014 at 11:07 AM, Derek Koopowitz< derekkoopowitz at gmail.com
<mailto:derekkoopowitz at gmail.com> > wrote:
Everyone,

The NSRCA website is back up and running again... our host was doing some
maintenance on the server.  The scores for the Bakersfield contest have been
posted and just to let everyone know...

1.  Every score is normalized to PERFECT, not to best pilot
2.  Every AMA class flew 2 final rounds on Sunday using unknowns
3.  Best 3 of 4 prelim rounds were AVERAGED to produce a SINGLE round score
to carry over into the finals.
4.  Best 2 of 3 (prelim carry over + 2 final rounds = 3) count for the final
score.

In FAI F, Prelim carry over is shown as Round 1, Round 2 is first finals
round, and round 3 is the 2nd final round -- for some reason the scoring
program is showing that as round 0.

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

--
Director, Fixed Wing Flight Training
Santa Clara County Model Aircraft Skypark

--
Director, Fixed Wing Flight Training
Santa Clara County Model Aircraft Skypark

_______________________________________________ NSRCA-discussion mailing
list  NSRCA-discussion at lists.nsrca.org
<mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

--
Director, Fixed Wing Flight Training
Santa Clara County Model Aircraft Skypark

_______________________________________________

NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

  _____

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
  _____

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

--
Jason
http://jasonshangar.weebly.com/
_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion

_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org <mailto:NSRCA-discussion at lists.nsrca.org>
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20140506/6650a21a/attachment.html>