Methodology for Free Agents Will Break Your Heart:

FREE AGENTS:

I used the MLBTR transaction tool and ESPNs free agent tracker to download a list of all free agents from 2007 to 2014 (and Carlos Beltran in 2005) that have completed their contract and earned an annual salary of at least five million. I used a minimum annual salary of five million because casual fans are only interested in top free agents.

I put free agents into categories based on whether they were position players, starting pitchers and relievers. I did not split up free agents by offensive position because there are a limited number of position free agents in the sample and no reason to presume that there will be major differences in expected value for position.

There were 116 position players, 73 starting pitchers and 43 relief pitchers free agents in this study.

PROSPECTS:

I looked at all prospects ranked by Baseball America from 1998 to 2007. I used this time frame because my research has shown that pitching prospect value has improved over time. This sample size is still robust while not relying on obsolete data. I didn’t use any prospects ranked from 2008 to 2014 because I wanted to give prospects time to reach the majors and ensure that they probably were no longer under team control.

I measured a prospects performance based on the years that he would be under team control (players that entered a season with less than six years of service time) without signing an extension. Players like Chris Tillman and Chris Davis are good examples of why I did this. Chris Davis first made it into the majors in 2008 but isn’t eligible for free agency until after the 2015 season while Chris Tillman first made it into the majors in 2009 and has become arbitration eligible this year for the first time. Both players saw considerable playing time in their first year (whether it was 2008 and 2009) and therefore looking at their six years in the majors would severely discount their value. As a result, it is necessary to measure their performance while under team control as opposed to measuring their performance over their first six years in the majors.

I attempted to use salary information to determine when a prospect was eligible for free agency. Players see a large increase in salary when they become arbitration eligible and it is theoretically possible to use this information to determine when a player was under team control. This method shouldn’t be expected to have a 100% success rate. Fortunately each prospect has their service time listed on their Baseball Reference page. For the majority of relevant prospects I ended up using their page as a reference to determine when they reached six years of service time. This manual check should ensure accuracy.

In order to determine prospect average value, I added up the total number of fWAR that they produced while under team control and divided by the smaller of the number of seasons that they were under team control or six. A prospect that produces six wins in one season and then suffers a career-ending injury is considerably more valuable than a prospect that produces six wins in six seasons. It is true that most successful prospects play in at least seven seasons. I didn’t think it was fair to determine average value using total seasons because a prospect is usually brought up in the middle of a year. Also, any fraction of a year was likely purchased at minimal cost and therefore should be considered an extra bonus.

RANKING:

I grouped prospects based on whether they were position or pitching prospects and by ranking. The four rankings that I used were 1-10, 11-25, 26-50 and 51-100. I used those rankings based on other research that I’ve completed suggesting that this is the ideal grouping. I considered each time that a prospect was ranked as a separate rating. Just like players can be free agents multiple times and therefore count multiple times in this sample, I believe that prospects should be judged the same way. If Baseball America ranks a prospect in two different years then his value should count twice.

I put players into five categories: Bust, Below Average, Average, Above Average and Star. Busts were worth less than .5 fWAR per year. Below Average players were worth .5 to 1.49 fWAR. Average players were worth 1.5 to 2.49 fWAR. Above Average players were worth 2.5 to 3.49 fWAR. Stars were worth 3.5 fWAR or more. These categories have been used by a number of others. I feel that these categories give us an idea of how much success a prospect has had in the majors.

I also put players into three more general categories: Bust, Success and Superior. Busts are worth less than 1.5 fWAR per year. Successes are worth more than 1.5 fWAR per year and Superior players are worth more than 2.5 fWAR per year. These categories have also been used in a number of other papers.