 |
| Scary little guys |
In spite of the recent recruiting success, our friend ProveIt has come to share some knowledge with the rest of us in form of a guest post. Be sure to tell him what you think of his article!
You hear claims that the recruiting boards are biased, but never any details. Are the boards truly biased and deserved to be cybernetically torched at every mention, or is this just sour grapes from fans in conferences with lower rankings?
A quick internet search of various analysis of their rankings show Scout and Rivals are notably more accurate than the others. Rivals nominally edges out Scout in accuracy, but not enough to herald them king (usually only a couple of players a year differentiate the 2, and some criteria is questionable).
I don’t consider subjective as less accurate or more biased than objective. An objective plus subjective analysis has the capacity to be much more accurate than a purely objective analysis. An objective formula can be biased if the criteria are slanted.
Proving bias in a subjective analysis is difficult. I looked at the objective rankings.
Scout MethodologyPlayer Rankings are purely subjective. Starting at a local level, recruits are ranked by position and overall ranking (all positions combined). These lists are passed up and merged through a regional and finally national level. Stars are awarded based on the player’s position in the “All positions” list:
1-50: 5 star
51-300: 4 star
…and so on…
Team rankings are based on the top 25 players in the recruiting class. If a program goes over 25, only the top 25 recruits are used. If a program is under 25, they get 0 points for the missing recruits. Each player commitment adds points to the team’s recruiting class.
The points a player adds are based on the number of stars:
5 Star = 200 points
4 Star = 120 points
3 Star = 40 points
2 Star = 20 points
4 Star = 120 points
3 Star = 40 points
2 Star = 20 points
A bonus is awarded if the player is ranked in the top 100 at his position. The #1 player at a position is worth an additional 100 points, the #100 player is worth an additional 1 point.
Rival’s description is cryptic, but claims an objective and subjective element.
Objective Component I believe each program is given a recruiting ranking based on the weighted average of their recruitment classes over the previous several years.
Using this program recruiting ranking, a value is added to each athlete they present with a formal offer. The better the program has been at recruiting, the higher the value of their offers.
Recruits are ranked by the total value of the offers they receive (program interest). Each offer increases the recruit’s total by an amount based on the program’s recruitment ranking.
Using this method, Rivals cleverly incorporates the opinions of thousands of coaches and college scouts nationwide into their calculation. This can be considered their rough starting list.
Note: This is just my best guess based on Rivals description. It is understandable Rivals wants to keep the details of their calculation confidential and proprietary.
Subjective Component – I believe Rivals staff takes the rough starting list and tweak it to their scouting opinions. The result is the final player ranking, listed in both position rankings 1-100 and overall ranking of all positions.
Like Scout, Rivals’ team rankings for the year are based on the value of the recruits they have signed in the current class.
Unlike Scout, Rivals does not have a fixed number of 5 star, 4 star, etc. recruits but attempts to gage their quality over an extended period of time. Rivals typically awards about ½ as many 5 star rankings, but nearly the same number of 4 star rankings as Scout.
Rivals establishes a subjective star ranking over several years. This provides a consistency in comparing star rankings of recruits from different years.
Scout issues a fixed number of stars based on the ranking of all recruits at all positions. Because of the broad base (all positions and all players nationwide) this should also be reasonable consistency year to year.
Position rankings do not provide a good comparison across multiple years.
Neither site ranks junior college recruits other than awarding a star rating. Both include Junior College recruits in the team rankings, based entirely on their star ranking.
In at least the Scout team ranking calculation (and perhaps both) this creates a bias against programs which cultivate junior colleges – the position rank can increase the value of a recruit in the team calculations by over 50%, but JC recruits do not get this bump.
I did not count the number of signed JC recruits to determine if this results in a regional or conference bias.
Scout establishes a broad point differential for team rankings based on the stars rating of the school.
For example, the 28th ranked RB had 4 stars and added 201 points.
The 29th ranked RB had 3 stars adding 120 points.
The 81 point difference between the 28th and 29th RB is greater than the difference between the 29th RB and 113th ranked RB – hardly a scale that truly represents the differential.
This exaggerates minor differences between programs, but does not present a regional bias.
Number of Signed Recruits Bias
The recruiting boards do not adjust based on the number of recruits signed. Scout counts the 1st 25 recruits (which few programs sign). Rivals may count every recruit.
If a program signs less than this number, Scout gives zero points for the unfilled positions. If a program signs more than this number, only the best 25 recruits are used, the rest are discarded. Rivals might just keep adding forever.
This is the equivalent to playing stud poker where you are dealt 5 cards, while your opponent is dealt 6 and gets to discard 1. Reality it is even worse than this analogy - in comparing 2011 South Carolina and PSU, South Carolina was dealt 6 cards (32 recruits) and PSU was dealt 3 (16 recruits).
This could be easily remedied – just decrease the number of players considered, or diminish the value of additional recruits in large classes. Either would place the weight of the calculation where it belongs - on the top 17 or so players. Instead the recruiting services acknowledge but continue to use this extreme bias.
This creates a bias against conferences which sign fewer players on average. Recent limitations such as the SECs limit of 28 signed recruits will do little to correct this since very few programs sign 28 recruits a year.
This creates a bias against programs more likely to red shirt athletes since they have fewer scholarships to award. The irony being the red shirts may be a result of past recruiting success providing depth.
In a useless gesture, the B1G limits programs to 28 or less formal scholarship offers to undecided and verbal recruits. Without the limit, programs in other conferences offer everyone and sort them out when the recruits try to accept (a complaint levied by Georgia’s Richt).
Rivals recruits rating is partially based on the number of offers they receive. This creates 2 biases.
1. Regional bias based on formal offers. The B1G region has a large population, but not nearly as many majors in the region. With other regions having a lot more programs, each awarding approx. twice the number of formal offers, it is no surprise the Rivals calculation determines there is more interest in these recruits based on more offers. As a result, these athletes start out higher on the Rivals initial list.
2. Regional bias based on early verbal commitments. With fewer majors in the region and conference rules limiting commitments, B1G programs are more likely to sign better recruits early. Recruits that commit early receive fewer formal offers decreasing their value in Rivals’ calculation, which decreases the program ranking, when in reality the program is demonstrating recruiting excellence.
These biases are multiplied. When these recruits commit to a local program, this bias boosts the local program’s ranking. Over time this boosts the value of the program’s scholarship offers, which boosts the recruits rankings, which boosts the program rankings, which boosts the athlete’s rankings, and so on…
These biases becomes apparent when you compare the teams rankings of Rivals and Scout – the B1G programs are around 4 rankings lower in Rivals than Scout. The real drop is really greater when you factor in Scout’s bias towards conferences that sign larger classes.
From the hidden regional bias introduced by Rivals’ basing their calculation on offers, to Scouts’ overt bias for signing larger classes, there is a definite bias against the B1G in the recruiting service rankings.
This comes just looking at the objective part of the calculations – the subjective may not exist, or it could represent even more bias.
The sites aren’t incredibly inaccurate, and watching a coach build a recruiting class can add a new dimension to enjoying college football. I would pay good money to be taken through a typical recruit’s campus visit, and would love to sit in on a top coach making a pitch to a recruit and his parents.
You just need to know how to look past the biases…
Team Rankings and Conference Rankings are better viewed by sorting on average star rating to get around the class size bias. Recognize teams separated by 1/10th of a star or less are about equal.
Some like to look at the number of commitments from Top 100. At Rivals I consider this a little bit too regionally biased. For both I consider this a bit too limited to get an accurate picture of the entire class.
Some like to look at the number of 5 star recruits signed. With only 25-50 recruits a year with this ranking, this makes the field a bit too narrow to get an accurate picture of the entire class.
I like to look at the total number of 4 plus 5 star recruits if I am looking at top recruiting programs. These are the athletes expected to be future stars. If I am looking at a middle or lower program, I also look at the number of 3 star recruits.
I personally prefer Scout. There isn’t a notable difference in accuracy, and at least its biases are somewhat predictable. I also find it easier to navigate.
Player Rankings depends on the priority. If I am looking at overall roster strength, I look at the star ranking since it is more constant in comparing players from different years. If I am looking at must fill position needs (2012 OSU and the OL), I look at the position rank since this is the only options available to my program.
Hint: When looking at the position, keep in mind the number of athletes on the field at any given time and the length of time he is expected to start. For example, the 30th rank QB is probably around 60th for 2 upper classmen classes – with only 1 on the field at any given time, this is low for a major and average for all FBS teams. On the other hand, with 3 typically on the field, the 30th ranked WR is probably among the top 1/3 starters for majors.
Disclaimer My interest in recruiting is sporadic at best - I don’t subscribe to the sites. Without full access, my information on how the rankings are calculated might be off, particularly for Rivals. On the other hand, with this much bias, why would I want to pay for a subscription?
Thanks for reading. Any comments or corrections are greatly appreciated.
