Tuesday, April 29, 2014

Draft Forecasting vs. Draft Evaluating: A Statistical Analysis of Major Media Big Boards

There is a line in the sand that needs to be drawn, for without it the boundaries regarding draft analysis and evaluation against draft projection and prediction will seamlessly blend together to create the mass chaos many passionate NFL Draft enthusiasts are currently encountering.

Doug Kyed of NESN.com (@DougKyedNESN) has put together a composite ranking of the top 100 players in the 2014 NFL Draft by combining 9 highly regarded draft analyst's big boards, available here: http://nesn.com/2014/04/jadeveon-clowney-no-1-in-composite-top-32-2014-nfl-draft-expert-rankings/.  The 9 analysts included Mel Kiper Jr and Todd McShay of ESPN, Gil Brandt and Daniel Jeremiah of NFL Network, CBS Sports' Rob Rang and their NFL Draft Scout affiliate page, Bleacher Report's Matt Miller, Rotoworld's Josh Norris and NEPatriotsDraft.com expert James Christensen.  You can visit the original article from Doug Kyed to see the end result.

This collection appears to be a very well filled out collection of rankings that should in fact reflect a more specific distribution of the talent in the 2014 NFL Draft.  But there's a problem here, the pot is mixed.  There are talent evaluators, who watch the prospects on film, investigate their skill sets and their background and personality to determine how successful they should be (in theory) in the NFL and there draft forecasters: those who are taking their own observations and pair that with the inside information they have available to them as highly ranked media personnel to try to as correctly as possible project where players will be falling on draft day, not where they SHOULD be falling.  The lines are pretty firmly drawn and as you go higher up the ladder, the heavier it leans to the latter. 

ESPN: The World Wide Leader in Sports

ESPN is the go to leader in all things sports, but when it comes to the NFL Draft, they're gunning to forecast, they aren't looking to provide an audience with thought provoking in depth analysis.  Take a look at Mel Kiper Jr and Todd McShay's big boards side by side, along with their "deviation" per prospect.  Deviation is the difference between where one has a player ranked in comparison to where the other has the same player ranked.  If I ranked Jadeveon Clowney #5 and Todd McShay ranked Jadeveon Clowney at #1, we would have a deviation of 4. 

PlayerPos.CollegeMel KiperTodd McShay
ESPN H2H
1Jadeveon ClowneyDES. Carolina110
2Khalil MackOLBBuffalo330
3Greg RobinsonOTAuburn422
4Sammy WatkinsWRClemson242
5Jake MatthewsOTTexas A&M752
6Mike EvansWRTexas A&M671
7Teddy BridgewaterQBLouisville16151
8Aaron DonaldDTPittsburgh11101
9C.J. MosleyLBAlabama981
10Taylor LewanOTMIchigan561
11Johnny ManzielQBTexas A&M18162
12Eric EbronTEUNC8113
13Anthony BarrOLBUCLA22308
14Blake BortlesQBUCF1798
15Zack MartinOTNotre Dame10144
16Justin GilbertCBOkla. St.14217
17Odell BeckhamWRLSU13196
18Ha Ha Clinton-DixSAlabama19127
19Darqueze DennardCBMich. St.15172
20Brandin CooksWROregon St.12186
21Ryan ShazierLBOhio St.20299
22Calvin PryorSLouisville21201
23Marqise LeeWRUSC23252
24RaShede HagemanDTMinnesota352213
25Louis NixDTNotre Dame412417
26Kony EalyDEMissouri433112
27Kyle FullerCBVirginia Tech402812
28Jason VerrettCBTCU372710
29Bradley RobyCBOhio St.392613
30Timmy JerniganDTFSU332310
31Dee FordDEAuburn453411
32Xavier Su'a-FiloOGUCLA42431
33Derek CarrQBFresno St.2611892
34Cyrus KouandijoOTAlabama32331
35Jimmie WardSNIU50482
36Morgan MosesOTVirginia381325
37Stephon TuittDTNotre Dame28379
38Kelvin BenjaminWRFSU27325
39Jace AmaroTETexas Tech744628
40Carlos HydeRBOhio St.30399
41Kyle Van NoyLBBYU246036
42Austin Seferian-JenkinsTEWashington634914
43Dominique EasleyDTFlorida986434
44Cody LatimerWRIndiana294011
45Scott CrichtonDEOregon St.12545N/A
46Joel BitonioOLNevada673631
47Jimmy GaroppoloQBE. Illinois466216
48Davante AdamsWRFresno St.315120
49Jeremiah AttaochuOLBGeorgia Tech533518
50Troy NiklasTENotre Dame665610
51DeMarcus LawrenceDEBoise State49418
52Deone BucannonSWSU684226
53Allen RobinsonWRPenn St.83830
54Weston RichburgCColorado St.57592
55Jordan MatthewsWRVanderbilt475710
56Marcus MartinCUSC445410
57Ja'Wuan JamesOTTennessee52502
58Chris BorlandLBWisconsin253813
59Lamarcus JoynerCBFSU54477
60Marcus SmithDELouisville8410420
61Tre MasonRBAuburn557722
62Jarvis LandryWRLSU60528
63Martavis BryantWRClemson657510
64Bishop SankeyRBWashington48535
65Gabe JacksonOGMIss St.348652
66Donte MoncriefWROle Miss769014
67Jack MewhortOTOhio St.64631
68DaQuan JonesDTPenn St.957223
69Trent MurphyLBStanford589234
70Terrence BrooksSFSU80719
71Bruce EllingtonWRS. Carolina566812
72Jeremy HillRBLSU69672
73Paul RichardsonWRColorado936528
74Will SuttonDTArizona St.12595N/A
75Bashaud BreelandCBClemson12570N/A
76Keith McGillCBUtah70744
77David YankeyOGStanford367842
78Antonio RichardsonOTTennessee61583
79Telvin SmithLBFSU789921
80Pierre DesirCBLindenwood91976
81Kareem MartinDEUNC12582N/A
82Zach MettenbergerQBLSU9917576
83Stanley Jean-BaptiseCBNebraska724428
84AJ McCarronQBAlabama516615
85C.J. FiedorowiczTEIowa12579N/A
86Billy TurnerOTN. Dakota St.7714467
87Marcus RobersonCBFlorida12576N/A
88Carl BradfordLBASU125112N/A
89Charles SimsRBWest Virginia718817
90Ego FergusonDTLSU125117N/A
91Chris SmithOLBArkansas12596N/A
92Philip GainesCBRice125110N/A
93Caraun ReidDTPrinceton125107N/A
94Tom SavageQBPittsburgh62557
95Brandon ThomasOGClemson79736
96Jackson JeffcoatDETexas12585N/A
97Devonta FreemanRBFSU12591N/A
98Ka'Deem CareyRBArizona125123N/A
99Terrance WestRBTowson125134N/A
100Robert HerronWRWyoming82875
1151
Deviation per player13.54117647
Players within 10 slots64%
Of the 85 similarly ranked players illustrated between Mel Kiper Jr and Todd McShay, every prospect is separated by an average of 13.5 positions their respective big boards.  And 64% of the 85 ranked prospects are within 10 slots of one another.  These figures will be shown again later, but keep them in the back of your mind.

NFL Network: All Football, All the Time

The NFL Network had 2 experts of their own included in this Composite Big Board: Daniel Jeremiah and Gil Brandt.  Surely a network dedicated to strictly football would put more value into providing actual analysis instead of forecasting?

PlayerPos.CollegeDaniel JeremiahGil Brandt
NFL N H2H
1Jadeveon ClowneyDES. Carolina121
2Khalil MackOLBBuffalo431
3Greg RobinsonOTAuburn341
4Sammy WatkinsWRClemson264
5Jake MatthewsOTTexas A&M550
6Mike EvansWRTexas A&M770
7Teddy BridgewaterQBLouisville11132
8Aaron DonaldDTPittsburgh14173
9C.J. MosleyLBAlabama82315
10Taylor LewanOTMIchigan1091
11Johnny ManzielQBTexas A&M12111
12Eric EbronTEUNC13121
13Anthony BarrOLBUCLA6104
14Blake BortlesQBUCF20812
15Zack MartinOTNotre Dame19145
16Justin GilbertCBOkla. St.9112
17Odell BeckhamWRLSU17152
18Ha Ha Clinton-DixSAlabama16182
19Darqueze DennardCBMich. St.213211
20Brandin CooksWROregon St.22166
21Ryan ShazierLBOhio St.25196
22Calvin PryorSLouisville15216
23Marqise LeeWRUSC23203
24RaShede HagemanDTMinnesota26293
25Louis NixDTNotre Dame31343
26Kony EalyDEMissouri322210
27Kyle FullerCBVirginia Tech30255
28Jason VerrettCBTCU36404
29Bradley RobyCBOhio St.28357
30Timmy JerniganDTFSU183113
31Dee FordDEAuburn33363
32Xavier Su'a-FiloOGUCLA442717
33Derek CarrQBFresno St.35269
34Cyrus KouandijoOTAlabama422418
35Jimmie WardSNIU43441
36Morgan MosesOTVirginia29301
37Stephon TuittDTNotre Dame34286
38Kelvin BenjaminWRFSU27336
39Jace AmaroTETexas Tech375013
40Carlos HydeRBOhio St.244319
41Kyle Van NoyLBBYU74
42Austin Seferian-JenkinsTEWashington39467
43Dominique EasleyDTFlorida416423
44Cody LatimerWRIndiana58
45Scott CrichtonDEOregon St.61
46Joel BitonioOLNevada38
47Jimmy GaroppoloQBE. Illinois37
48Davante AdamsWRFresno St.497930
49Jeremiah AttaochuOLBGeorgia Tech55
50Troy NiklasTENotre Dame60
51DeMarcus LawrenceDEBoise State40150110
52Deone BucannonSWSU53
53Allen RobinsonWRPenn St.46493
54Weston RichburgCColorado St.476720
55Jordan MatthewsWRVanderbilt42
56Marcus MartinCUSC68
57Ja'Wuan JamesOTTennessee503911
58Chris BorlandLBWisconsin557318
59Lamarcus JoynerCBFSU5411763
60Marcus SmithDELouisville458035
61Tre MasonRBAuburn62
62Jarvis LandryWRLSU3810062
63Martavis BryantWRClemson75
64Bishop SankeyRBWashington52511
65Gabe JacksonOGMIss St.104
66Donte MoncriefWROle Miss71
67Jack MewhortOTOhio St.72
68DaQuan JonesDTPenn St.81
69Trent MurphyLBStanford57
70Terrence BrooksSFSU109
71Bruce EllingtonWRS. Carolina89
72Jeremy HillRBLSU47
73Paul RichardsonWRColorado517625
74Will SuttonDTArizona St.150
75Bashaud BreelandCBClemson489951
76Keith McGillCBUtah54
77David YankeyOGStanford101
78Antonio RichardsonOTTennessee118
79Telvin SmithLBFSU96
80Pierre DesirCBLindenwood150
81Kareem MartinDEUNC66
82Zach MettenbergerQBLSU93
83Stanley Jean-BaptiseCBNebraska78
84AJ McCarronQBAlabama52
85C.J. FiedorowiczTEIowa95
86Billy TurnerOTN. Dakota St.90
87Marcus RobersonCBFlorida70
88Carl BradfordLBASU56
89Charles SimsRBWest Virginia103
90Ego FergusonDTLSU45
91Chris SmithOLBArkansas85
92Philip GainesCBRice102
93Caraun ReidDTPrinceton121
94Tom SavageQBPittsburgh41
95Brandon ThomasOGClemson5315097
96Jackson JeffcoatDETexas98
97Devonta FreemanRBFSU116
98Ka'Deem CareyRBArizona86
99Terrance WestRBTowson91
100Robert HerronWRWyoming123
793
Deviation per player14.41818182
Players within 10 slots
62%
Well...maybe not.  The 2 NFL Network analysts only similarly ranked 55 prospects, but their prospect deviation (average separation between a single prospect) was less than 1 spot on the board greater than ESPN's head to head comparison; while the percentage of player ranked landing within 10 spots of one another was 2% lower than ESPN's.  

CBS Sports/NFL Draft Scout

Perhaps a scouting entity that is contracted by a mass media powerhouse can provide some real substance with their projections and analysis?

PlayerPos.CollegeNFL Draft ScoutRob Rang
CBS H2H
1Jadeveon ClowneyDES. Carolina110
2Khalil MackOLBBuffalo231
3Greg RobinsonOTAuburn321
4Sammy WatkinsWRClemson451
5Jake MatthewsOTTexas A&M541
6Mike EvansWRTexas A&M963
7Teddy BridgewaterQBLouisville1495
8Aaron DonaldDTPittsburgh1073
9C.J. MosleyLBAlabama231112
10Taylor LewanOTMIchigan8168
11Johnny ManzielQBTexas A&M71710
12Eric EbronTEUNC17203
13Anthony BarrOLBUCLA1183
14Blake BortlesQBUCF6104
15Zack MartinOTNotre Dame12120
16Justin GilbertCBOkla. St.13141
17Odell BeckhamWRLSU15249
18Ha Ha Clinton-DixSAlabama18235
19Darqueze DennardCBMich. St.16133
20Brandin CooksWROregon St.20222
21Ryan ShazierLBOhio St.22157
22Calvin PryorSLouisville21183
23Marqise LeeWRUSC27270
24RaShede HagemanDTMinnesota334714
25Louis NixDTNotre Dame24195
26Kony EalyDEMissouri352114
27Kyle FullerCBVirginia Tech254217
28Jason VerrettCBTCU26282
29Bradley RobyCBOhio St.32342
30Timmy JerniganDTFSU19267
31Dee FordDEAuburn29301
32Xavier Su'a-FiloOGUCLA30311
33Derek CarrQBFresno St.28291
34Cyrus KouandijoOTAlabama31354
35Jimmie WardSNIU34362
36Morgan MosesOTVirginia37392
37Stephon TuittDTNotre Dame47407
38Kelvin BenjaminWRFSU587315
39Jace AmaroTETexas Tech496011
40Carlos HydeRBOhio St.446319
41Kyle Van NoyLBBYU563224
42Austin Seferian-JenkinsTEWashington55572
43Dominique EasleyDTFlorida61698
44Cody LatimerWRIndiana45450
45Scott CrichtonDEOregon St.463313
46Joel BitonioOLNevada41374
47Jimmy GaroppoloQBE. Illinois48502
48Davante AdamsWRFresno St.674423
49Jeremiah AttaochuOLBGeorgia Tech526614
50Troy NiklasTENotre Dame714922
51DeMarcus LawrenceDEBoise State382513
52Deone BucannonSWSU65596
53Allen RobinsonWRPenn St.36415
54Weston RichburgCColorado St.60528
55Jordan MatthewsWRVanderbilt439350
56Marcus MartinCUSC66615
57Ja'Wuan JamesOTTennessee39467
58Chris BorlandLBWisconsin81865
59Lamarcus JoynerCBFSU517827
60Marcus SmithDELouisville57489
61Tre MasonRBAuburn54551
62Jarvis LandryWRLSU1157243
63Martavis BryantWRClemson537724
64Bishop SankeyRBWashington50544
65Gabe JacksonOGMIss St.774334
66Donte MoncriefWROle Miss40382
67Jack MewhortOTOhio St.1059114
68DaQuan JonesDTPenn St.63585
69Trent MurphyLBStanford80811
70Terrence BrooksSFSU69681
71Bruce EllingtonWRS. Carolina766214
72Jeremy HillRBLSU68746
73Paul RichardsonWRColorado90846
74Will SuttonDTArizona St.865135
75Bashaud BreelandCBClemson91943
76Keith McGillCBUtah64N/A
77David YankeyOGStanford1517972
78Antonio RichardsonOTTennessee92857
79Telvin SmithLBFSU1078819
80Pierre DesirCBLindenwood749218
81Kareem MartinDEUNC735617
82Zach MettenbergerQBLSU1018714
83Stanley Jean-BaptiseCBNebraska78N/A
84AJ McCarronQBAlabama1249529
85C.J. FiedorowiczTEIowa1068917
86Billy TurnerOTN. Dakota St.72764
87Marcus RobersonCBFlorida825329
88Carl BradfordLBASU70655
89Charles SimsRBWest Virginia117N/A
90Ego FergusonDTLSU756411
91Chris SmithOLBArkansas79N/A
92Philip GainesCBRice62719
93Caraun ReidDTPrinceton94N/A
94Tom SavageQBPittsburgh429755
95Brandon ThomasOGClemson237N/A
96Jackson JeffcoatDETexas89N/A
97Devonta FreemanRBFSU130N/A
98Ka'Deem CareyRBArizona1049014
99Terrance WestRBTowson88968
100Robert HerronWRWyoming111N/A
977
Deviation per player10.73626374
Players within 10 spots67%

Well, maybe not.  Although CBS Sports and NFL Draft Scout may have a perfectly feasible explanation for this: The CBS Sports board very well be an average boards that includes Rob Rang's board; these may not be 2 separate entities but rather Rang's board is a component that creates the larger CBS board.  Because of this, we won't focus too closely on Rob's individual board, but rather the CBS Sports board overall.  

I know what you're likely thinking: These boards are all similar because they're all professionals!  Of course they're all going to see things similarly! Well with that in mind, here is ESPN's Todd McShay board in comparison to Bleacher Report's Matt Miller and Rotoworld's Josh Norris: 

PlayerPos.CollegeTodd McShayMatt MillerTodd McShayJosh Norris
Miller vs. McShayMcShay vs Norris
1Jadeveon ClowneyDES. Carolina121121
2Khalil MackOLBBuffalo352330
3Greg RobinsonOTAuburn242286
4Sammy WatkinsWRClemson431451
5Jake MatthewsOTTexas A&M561561
6Mike EvansWRTexas A&M7817125
7Teddy BridgewaterQBLouisville1511415114
8Aaron DonaldDTPittsburgh101111091
9C.J. MosleyLBAlabama8102871
10Taylor LewanOTMIchigan6221662721
11Johnny ManzielQBTexas A&M16124162812
12Eric EbronTEUNC11132112312
13Anthony BarrOLBUCLA30141630228
14Blake BortlesQBUCF9211292920
15Zack MartinOTNotre Dame142612142410
16Justin GilbertCBOkla. St.21714214019
17Odell BeckhamWRLSU1917219145
18Ha Ha Clinton-DixSAlabama1216412102
19Darqueze DennardCBMich. St.1723617258
20Brandin CooksWROregon St.183113183113
21Ryan ShazierLBOhio St.29367291316
22Calvin PryorSLouisville20299204222
23Marqise LeeWRUSC25338253914
24RaShede HagemanDTMinnesota22351322166
25Louis NixDTNotre Dame243814241113
26Kony EalyDEMissouri31922314817
27Kyle FullerCBVirginia Tech28181028217
28Jason VerrettCBTCU2720727207
29Bradley RobyCBOhio St.2632626260
30Timmy JerniganDTFSU23252236441
31Dee FordDEAuburn342410341915
32Xavier Su'a-FiloOGUCLA43518432518
33Derek CarrQBFresno St.118151031183088
34Cyrus KouandijoOTAlabama33285335522
35Jimmie WardSNIU483018481731
36Morgan MosesOTVirginia134128135340
37Stephon TuittDTNotre Dame37436379154
38Kelvin BenjaminWRFSU32471532386
39Jace AmaroTETexas Tech463313461828
40Carlos HydeRBOhio St.392712396930
41Kyle Van NoyLBBYU60372360528
42Austin Seferian-JenkinsTEWashington4945449436
43Dominique EasleyDTFlorida64194564460
44Cody LatimerWRIndiana40995940455
45Scott CrichtonDEOregon St.45369453411
46Joel BitonioOLNevada361046836448
47Jimmy GaroppoloQBE. Illinois62584627513
48Davante AdamsWRFresno St.51543517221
49Jeremiah AttaochuOLBGeorgia Tech35885335323
50Troy NiklasTENotre Dame567519563719
51DeMarcus LawrenceDEBoise State41702941498
52Deone BucannonSWSU425210427634
53Allen RobinsonWRPenn St.83344983852
54Weston RichburgCColorado St.59771859509
55Jordan MatthewsWRVanderbilt573720577922
56Marcus MartinCUSC54595548329
57Ja'Wuan JamesOTTennessee50132825011565
58Chris BorlandLBWisconsin387436389355
59Lamarcus JoynerCBFSU477124476316
60Marcus SmithDELouisville10490141043371
61Tre MasonRBAuburn774631779619
62Jarvis LandryWRLSU5244852575
63Martavis BryantWRClemson75611475714
64Bishop SankeyRBWashington53671453175122
65Gabe JacksonOGMIss St.86959866719
66Donte MoncriefWROle Miss90801090864
67Jack MewhortOTOhio St.63491463585
68DaQuan JonesDTPenn St.728311728412
69Trent MurphyLBStanford9234589211119
70Terrence BrooksSFSU718211714130
71Bruce EllingtonWRS. Carolina6813163685612
72Jeremy HillRBLSU677366710336
73Paul RichardsonWRColorado6585206510843
74Will SuttonDTArizona St.956431951580
75Bashaud BreelandCBClemson706010704624
76Keith McGillCBUtah74817749723
77David YankeyOGStanford784038789820
78Antonio RichardsonOTTennessee5868105811456
79Telvin SmithLBFSU998910993564
80Pierre DesirCBLindenwood976334977720
81Kareem MartinDEUNC821284682875
82Zach MettenbergerQBLSU1754213317574101
83Stanley Jean-BaptiseCBNebraska44621844175131
84AJ McCarronQBAlabama66922666175109
85C.J. FiedorowiczTEIowa7912546795920
86Billy TurnerOTN. Dakota St.14494501446282
87Marcus RobersonCBFlorida7665117614670
88Carl BradfordLBASU11210391129220
89Charles SimsRBWest Virginia88533588957
90Ego FergusonDTLSU1171051211717558
91Chris SmithOLBArkansas9650469611317
92Philip GainesCBRice1102261161106149
93Caraun ReidDTPrinceton107126191074760
94Tom SavageQBPittsburgh5529023555175120
95Brandon ThomasOGClemson739825733637
96Jackson JeffcoatDETexas8548378517590
97Devonta FreemanRBFSU9110716915437
98Ka'Deem CareyRBArizona123695412315330
99Terrance WestRBTowson1341092513411618
100Robert HerronWRWyoming87101148713144
23982822
Deviation per player
23.98
Deviation per player
28.22
Players within 10 spots39%Players within 10 spots32%
The deviation per player in comparing Matt Miller's board is more than 66% greater than ANY difference illustrated when comparing same Network analysts.  The 39% of players within 10 slots on their respective boards is also approximate 66% greater than any illustrated in same Network analyst's boards.  And Josh Norris of Rotoworld shows even GREATER variance then the comparison between McShay's ESPN board and that of Miller from Bleacher Report.  

For the sake of space and data, I will not post the final charts, but I can tell you that in comparing 2 major media outlets head to head: ESPN's Todd McShay and CBS Sports' overall board, there was a deviation per player of 18.31 spots per player and 53% of players ranked within 10 spots on their respective boards.  These 2 figures (18.31 spots and 53%) are NOT statistically significant (a statistical measure to determine if a number has and weight to it)...so ESPN, NFL Network and CBS Sports' overall board are all, statistically speaking, related and there isn't any significant difference between them.  Sure, the numbers are different but from a statistics standpoint they're extremely similar.  That is NOT a criticism of the evaluation abilities of any of these 6 men, I have the utmost respect for all of their abilities.  But it IS an illustration of the priorities of their employers: they're rather know who is going where instead of who SHOULD be going where.  The boards of Josh Norris, Matt Miller, James Christensen?  Authentic, unique and insightful.  

Bottom line: If you want a unique, sincere talent evaluation, read more Josh Norris, Matt Miller, James Christensen (and Kyle Crabbs of NDT Scouting of course!)

If you want a projection game and a draft forecast, stick to the major media networks.

Did you enjoy this article?  Do you have any questions?  Reach Kyle Crabbs, founder of NDT Scouting, on Twitter via @NFLDraftTracker and let him know.  Also for your consideration, swing by NDT Scouting's homepage at www.NDTScouting.com and take a look around.  There are position rankings, a top 75 prospect big board, an explanation of NDT Scouting's scoring methods, multimedia and links to our latest content as well as the opportunity to purchase your copy of the NDT Scouting 2014 NFL Draft Guide for just $10!

No comments:

Post a Comment