[NSRCA-discussion] Scoring Process Question

[Ed White]

I think there is one critical point that saves us on significant digits.  That is the difference between absolute accuracy and relative comparison.  If we considered the score to be an absolute number, that we compared across rounds and across contests, we would be in trouble.  However, all we are really doing here is relative comparison of flyer to flyer within a single round.  Even after we normalize and end up with fractional scores, the scores accurately represent the relative standing of each flyer for that round.

When we combine normalized scores across rounds is where we have trouble.  But the main problem is not significant digits, it is assigning the value of 1000 to the highest scoring flyer for that flight.  That makes it an quasi-absolute measurement so we can compare across rounds.  Since we are now throwing different judges into the mix among other differences, it is not an exact comparison, but I think the present method is pretty good and the best we can do considering all the confounding factors present.

Ed


glmiller3 at suddenlink.net wrote: I'm going to open a can of worms here in hopes of coming up with a better system out of the discussion.  Perhaps this has been discussed before and I'm not aware of it.  Let me preface this by saying I am not a mathematician or statistician, but I have some familiarity with both subjects and the following question has been growing in my mind for some time. 

It seems to me that we are judging our maneuvers with limited accuracy (within 1 point in FAI and X.5 points in AMA classes) we are then creating the ILLUSION of accuracy by multiplying that score by a K factor and then normalizing to a 1000 point scale.  Here is a fairly brief explanation of "Significant Digits" that I've copied from the web which will introduce you to this thought if you haven't seen it before:

****"SIGNIFICANT DIGITS

The number of significant digits in an answer to a calculation will depend on the number of significant digits in the given data, as discussed in the rules below. Approximate calculations (order-of-magnitude estimates) always result in answers with only one or two significant digits. 

When are Digits Significant? 

Non-zero digits are always significant. Thus, 22 has two significant digits, and 22.3 has three significant digits. 

With zeroes, the situation is more complicated: 

Zeroes placed before other digits are not significant; 0.046 has two significant digits. 
Zeroes placed between other digits are always significant; 4009 kg has four significant digits. 
Zeroes placed after other digits but behind a decimal point are significant; 7.90 has three significant digits. 
Zeroes at the end of a number are significant only if they are behind a decimal point as in (c). Otherwise, it is impossible to tell if they are significant. For example, in the number 8200, it is not clear if the zeroes are significant or not. The number of significant digits in 8200 is at least two, but could be three or four. To avoid uncertainty, use scientific notation to place significant zeroes behind a decimal point: 
8.200 ´  has four significant digits 
8.20 ´  has three significant digits 

8.2 ´  has two significant digits

Significant Digits in Multiplication, Division, Trig. functions, etc. 

In a calculation involving multiplication, division, trigonometric functions, etc., the number of significant digits in an answer should equal the least number of significant digits in any one of the numbers being multiplied, divided etc. 

Thus in evaluating sin(kx), where k = 0.097 m-1 (two significant digits) and x = 4.73 m (three significant digits), the answer should have two significant digits. 

Note that whole numbers have essentially an unlimited number of significant digits. As an example, if a hair dryer uses 1.2 kW of power, then 2 identical hairdryers use 2.4 kW: 

1.2 kW {2 sig. dig.} X 2 {unlimited sig. dig.} = 2.4 kW {2 sig. dig.} "******

My Point is this:

I've seen many contests decided by less than 10 points on a scale of 4000 which has been expanded from (at most) 2 significant digits.  As a matter of "statistics" I think that any separation of less than 100 points (two significant digits, ie,  3X00 points) is "artificial accuracy".  Unfortunately, I don't have any great ideas about how to improve upon the current system, I'm just pointing out what I think is a scientifically valid problem with it.  

I smile when I see round scores posted to ten thousanths of a point on a scale that has been expanded from two significant digit accuracy to a 1000 point scale.  This turns a two significant digit answer into eight significant digits!  (ie, 1234.5678)    I think that scientifically, the scores would be more accurately posted as in scientific notation at   x.x  * 10 to the second power.  Most of the contests that I've been to this year have been decided essentially by random statistical "noise" rather than actual scoring decisions.  


Has anyone ever thought/talked about this before ?   

Let me add, that despite what I think are statistically invalid methods, in most cases the system seems to work pretty well.  In general the superior pilots get enough better scores to overcome the "noise" but it sure would be nice to come up with a more mathematically valid solution, IMO.

George




_______________________________________________
NSRCA-discussion mailing list
NSRCA-discussion at lists.nsrca.org
http://lists.nsrca.org/mailman/listinfo/nsrca-discussion
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nsrca.org/pipermail/nsrca-discussion/attachments/20070627/40c03d93/attachment.html