To Matrix or not to Matrix

Sat Nov 13 16:20:07 AKST 2004

..........Since my name was mentioned..................I've done the numbers and make no claims - only a conclusion which is based on real numbers.  I do not make the same conclusion as Ron Van Putte.

"Vastly superior" and "far superior" are obviously narrative terms and subject to debate.  I can't attach the rather lengthy spreadsheet I have (which does have very specific numbers for a number of scenarios), but I can pretty easily provide a narrative summary of the assumptions and final results of one possible scenario.  I've not presented much in the way of decimal points - if I did, the results don't change anyway.

Assumptions:
72 total K for pattern
9.0 average score for #1 seeded pilot (Pilot A)
8.9 average score for #2 seeded pilot (Pilot B)
8.8 average score for #3 seeded pilot (Pilot C)
8.7 average score for #4 seeded pilot (Pilot D)

8.5 average score for unseeded pilot (Pilot X)

At the end of the 6 round preliminaries, the results for top 4 pilots will be:
Pilot A    4000    1000 average
Pilot B    3989    997  average
Pilot C    3965    991  average
Pilot D    3921    980  average

So what about Pilot X?  The score for Pilot X is dependant upon which flight group Pilot X is placed in:
"X" in Group A    3776    944
"X" in Group B    3809    952
"X" in Group C    3833    958
"X" in Group D    3833    958

So based on essentially nothing other than random luck, 2 unseeded pilots of equal skill may see a 57 point differential.  As points of reference from the 2004 US Nationals:
- The point differential in the F3A prelims between 8th (3809, last man into the finals) and 11th was 56 points.
- The point differential in the Masters prelims between 8th (3863, last man into the finals) and 12th was 53 points.

The most extreme case (where Pilot A is "vastly superior") results in Pilot A finishing the prelims in 1st, the #5 seed finishing in 8th, and Pilots 9, 13, 17, 21, 25, and 29 finishing 27, 28, 29, 30, 31, and 32nd places, respectively.

I don't have the full raw data sets from past Nationals contests to complete evaluations, but I am quite certain one or more pilots in my flight group in the 1998 US NATs Masters prelims were pushed down in the standings (and possibly out of the finals) as I was the strongest flyer (and eventual winner) in the prelims by a substantial margin.  And, I am equally certain that several years later I was pushed out of the US NATs F3A finals as I was flying in the group with the strongest flyer in the prelims.  For anyone out that has full data sets (and the time), it might be interesting to complete an analysis of what happens when a given pilot is moved into different flight groups - I think my above narrative demonstrates what can happen.  If you think my numbers/scenario are unrealistic, maybe you will have a different conclusion.

Regards,

Dave Lockhart
DaveL322 at comcast.net

  ----- Original Message ----- 
  From: Ron Van Putte 
  To: discussion at nsrca.org 
  Sent: Saturday, November 13, 2004 6:10 PM
  Subject: Re: To Matrix or not to Matrix

  On Nov 13, 2004, at 3:54 PM, George Kennie wrote:

    <<<BTW, the way the matrix system is implemented now, the top eight entrants are seeded, based on the previous Nats, and the
    top four are distributed among groups 1, 2, 3, 4, and the next four are distributed among groups 1, 2, 3, 4.>>>

    This is confusing to me. To say that the top 4 are distributed in groups 1,2,3&4 and the next 4 are distributed in groups 1,2,3&4 sounds the same as the top 8 are distributed in groups 1,2,3&4 ?????? Please clarify.

  Pilots are seeded 1 thru 8. Pilots seeded 1 and 5 are in group 1, pilots seeded 2 and 6 are in group 2, pilots seeded 3 and 7 are in group 3 and pilots seeded 4 and 8 are in group 4. First day is group 1 vs 2 and group 3 vs 4. Second day is group 1 vs 3 and group 2 vs 4. Third day is group 1 vs 4 and group 2 vs 3. If things go according to the seeding, at the end of the third day, pilots 1 thru 3 should each have won at least two rounds. Pilot 1 will have won six times, pilot 2 will have won four times and pilot 3 will have won twice. Pilot 4 will not have won a round, but should be close for the day against pilot 3. We keep the best score from each day, plus the next best score not previously used, so pilot 1 has a 4,000 normalized score, pilot 2 also has a 4,000 normalized score and pilot 3 has a score of 2,000, plus his third daily score and the next highest score not used previously. Pilot 4 has his three best daily scores, plus the highest score not previously used. 

  I have run simulated contests for 32 pilots using this seeding and, unless pilot 1 is vastly superior to pilot 2, even the worst case scenario of having pilots who are ranked 1, 5, 9, 10, 11, 12, 13, 14 in group 1, pilots who are ranked 2, 6, 15, 16, 17, 18, 19, 20 in group 2, pilots who are ranked 3, 7, 21, 22, 23, 24, 25, 26 in group 3 and pilots who are ranked 4, 8, 27, 28, 29, 30, 31, 32 in group 4, the matrix system will get EVERY finishing place correct. Try it yourself if you don't believe it. Frankly, I am tired of having people who have not "done the numbers" claim otherwise. I had a lively discussion and exchange of data with Dave Lockhart on this subject and he could demonstrate situations where the matrix system did not get the right order, but it required that the best pilot had to be far superior to all the other pilots for the matrix system to get it wrong. That hasn't happened in real life so far.

  I'll be honest in agreeing that the problem of a good pilot having an unlucky day (flameouts, equipment problems, etc) causing a pair of low scores on one day that would keep him from the finals, is not right, but I don't see a fair way around it. 

  Ron Van Putte
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.f3a.us/pipermail/nsrca-discussion/attachments/20041113/accbc6c0/attachment.html