THIS IS INTENDED TO REFLECT THE REALITY OF WHAT HAPPENS DURING A SEASON.
What is a number 2 starter? What is a number 5? These terms get bandied about a ton, but really what is the expected performance for these roles? I'm not talking theoretically, but the hard reality of a 162 game season and as it turns out, it's far worse than most people will intuitively guess.
THIS IS INTENDED TO REFLECT THE REALITY OF WHAT HAPPENS DURING A SEASON.
Fact is, pitchers get hurt. A lot. And because of that, teams end up giving a lot of starts to woefully inadequate pitchers. So any measurement that attempts to quantify the performance of starting pitchers into buckets of a rotation needs to take this into proper consideration.
For the methodology of this exercise, I turned to a pair of articles for inspiration. Chris Jaffe published an article attempting to figure this out back in the winter of 2006. His method was to assign slots for each team based on their rotation coming into the season and hold those as fixed as pitchers were swapped in and out. A single day before Jaffe's article was published, Jeff Sackmann had the first of his series of article published on The Hardball Times about rotations. His method was to take all the starts for a team and just group the best 32 as #1, the next best 32 as #2 and so on.
Those are both decent methods, but what stuck out to me was the fixture of doing the rotation slots team by team. It doesn't make a lot of sense to me to have, for example, the Nationals' best 32 starts this year grouped in with the Blue Jays' best 32 starts as those made by #1 pitchers. I want a measurement whereby if you have a #1 pitcher, you can say that pitcher is among the top pitchers in the league regardless of team.
In that essence here's what I did. I took every pitcher who logged a start in 2007 and sorted then in descending order by tRA+. Then I just separated the list into five parts, one for each rotation slot. The cutoffs for each slot were based on an offshoot of the expected innings pitched profile that Jaffe outlines with some minor tweaks to move from actual innings to expected innings.
#1 starters are expected to log 200 xIP (600 xOuts). Therefore, the group of #1 starters was the found by taking the sorted list and, starting from the highest tRA+ and working down, finding where the cumulative xOuts totaled as close as I could get to 18,000 (600 * 30 teams).Then the xOuts and xRuns were summed for that group and from those two numbers, the tRA was computed.
#2 starters was the same process except for 192 xIP, #3 starters were slated at 183 xIP, #4s at 175 xIP and #5s took up the rest, around 166 xIP. This process was then repeated for 2008. I did it separately for 2007 and 2008 because I wanted to see how much variation there was between the two years. If there was considerable change between them then I would suspect that it wasn't a stable enough measurement to be useful and would require a bigger sample. As it turned out, the rankings were incredibly similar and so I feel comfortable enough to go forward.
TO BE PERFECTLY CLEAR: By this definition, a #5 starter is probably not what most people term a #5 starter. I assume that when most people talk about a #5 starter, they talk about some mythical rotation that almost never misses a start and this person being the worst pitcher on it. For the most part, those rotations do not happen. A #5 as defined below represents the combined worst starters to have actually pitched. In my opinion, this is the more useful definition, because this way, having health is properly weighted and you get a notion for the level of scarcity that exists.
By traditional measures, a 90 tRA+ pitcher is terrible, a #5 or so. But a 90 tRA+ pitcher that stays healthy enough to log 180 innings is valuable because he keeps you from having to turn to those replacement 80 tRA+ pitchers for spot starts and that's what I want to be apparent here.
That's not to say he is valuable enough to warrant paying much for. As you can see below, the bottom 40% of starters are pretty well below average. It's generally the performance level that you can find off the scrap heap. So if you cannot build a Toronto or Chicago (AL) staff, use Ryan Feierabend for those back end innings; don't pay Carlos Silva. Use Dustin Moseley; don't pay Jon Garland.
AVERAGES. (based on 2007 and 2008 data)
#1 STARTERS: 130 tRA+
#2 STARTERS: 112 tRA+
#3 STARTERS: 100 tRA+
#4 STARTERS: 91 tRA+
#5 STARTERS: 76 tRA+
MARKERS.
These represent the breakpoints between each slot. For example, between a 95 and a 106 tRA+ would be rated as a #3 starter. Below 86 and you're a #5, above a 118 and you're a #1.
1 -- 2 BARRIER: 118 tRA+
2 -- 3 BARRIER: 106 tRA+
3 -- 4 BARRIER: 95 tRA+
4 -- 5 BARRIER: 86 tRA+