FanPost

Quantifying Opener Effectiveness, Part Two

Over the course of the 2019 season, I spent time tracking the Mariners’ usage of openers, in an attempt to determine if implementing the strategy helped to improve the team. After a previous fanpost, I realized that just looking at the Mariners wouldn’t provide a definitive answer to this question. I decided to wait until the season was over, and then to collect and analyze data from all the teams in the league which used openers. In an attempt to find a numerical value for the effectiveness of openers, several variations to common sabermetric equations were made. This data is summarized in a new and improved opener tracking sheet.

Data Collection

Since I had been collecting data on games where the Mariners used an opener, that data readily available. When it came to collecting data on the rest of the league, going through every game played through the year, in order to pick out the games with an opener was impractical, requiring a different method of data collection. The solution was to use game data from fangraphs.com. In order to narrow down the team by team data, a "formal" definition of an opener was needed. After a quick search, I came across this article, in which the author lays out two basic rules: the opener goes no more than two innings pitched or nine batters faced, while the following pitcher goes at least four innings or face more than 18 batters. Compared to the already collected Mariner opener data, this definition held in almost all cases.

With this definition in hand, the data collection could begin. Setting the Fangraphs’ pitching leader page to starters, with minimum innings pitched to zero, 368 names appear. This data was exported and narrowed by calculating the innings pitched per games started, and picking the pitchers with a value less than or equal to 2.0. I compared the results for the Mariners, and found that this method missed Kikuchi as an opener, and gave Reggie McClain two opens vs just the one he had on the year. The second start was a bullpen day, not meeting the second part of the definition. This led me to go team by team to verify the identified pitchers as having starts that met the definition above. The most difficult team to analyze was the Rays, since they had multiple bullpen days, which had to be omitted from the openers’ stats before adding them to the sheet. The end result was a list of 52 pitchers and 127 games. Only fifteen teams were identified as using openers.

Before getting into the data analysis, there are two exceptions to the definition that can be found in the data sheet. The first is Yusei Kikuchi’s appearance as an opener. Yusei did only pitch a single inning while facing just three batters, but Justus Sheffield followed with only 3.0 innings pitched and 14 batters faced. As this was designated an open for Kikuchi by the team, the data was included in the analysis. The other exception comes from our old friend Nick Vincent. The Giants declared Vincent was the opener for a start on May 14th, where he did go one inning, facing seven batters, but was followed by Tyler Beede, who pitched just 2.1 innings against 14 batters. A last note on the data collection; the filtering method presented above would have missed Yusei Kikuchi’s open, since he had an average of 5.05 inning pitched per start. The opener sheet will be updated if I discover any other starters that went out for an open.

Method 1: Nine Innings Pitched

When trying to come up with a method to mathematically determine opener effectiveness, the first choice was to simply use the common formulas for ERA, FIP and xFIP, focusing on the opener data set. Since we are looking at opener-specific data, the calculated values are listed as "oERA," "oFIP" and "oxFIP." oFIP and oxFIP constants were generated from oERA, and the results were tabulated in the sheet. The values for oERA were all over the place, ranging from 0.00 to 135.00. oFIP and oxFIP also both had wide ranges, 0.85 to 52.85 and 0.35 to 47.60 respectively. The large variation of values seems to have been generated by the small amount of innings pitched.

With these values in hand, we can move on to generating values for opener-specific WAR, or oWAR. Following the process on fangraphs.com, values for all 52 pitchers can be determined. The top three openers from this metric are Wilmer Font (TOR, 0.57), Ryne Stanek (TBR, 0.46) and Matt Wisler (SEA, 0.41). The bottom three of the list consisted of Carson Fulmer (CWS, -0.08), Jose Rodriguez (LAA, -0.12) and Stephen Tarpley (NYY, -0.22). The team which had the most oWAR was the Rays, with 0.75, while the Yankees had the lowest value at -0.16. Two decimal places were used to hep show the spread of values, since many are close to zero, another result of the low number of innings pitched by the openers.

Method 2: Two Innings Pitched

In an attempt to mitigate the small sample size issues with using the standard calculation methods, an adjustment was made to have oERA be based on a two innings instead of the traditional nine. The decision was made since the adopted definition of an opener would be a pitcher going at most two innings. Since the nine-inning game is considered a complete game for pitchers, it stood to reason that opener-specific data could be scaled to this lower value. After applying the adjustment, the range of oERA values were compressed to 0.00 to 30.00.

A new set of oFIP and oxFIP constants were created using the normal formulas, and this two-inning oERA. The resulting values were more spread out for each than in the first method, including several openers who posted negative values. With these basic values in hand, a second set of oWAR values was also determined. This was calculated using the same equations in method one as a base, making a couple changes to account for the smaller inning base. The changes were using the two-inning ERA, and switching the 18 in the "Dynamic Runs per Win (dRPW)" equation to a four, representing the four pitcher innings in a game with openers.

These changes to the formula resulted in the top three openers being Ryne Stanek (TBR, 6.9), Matt Wisler (SEA, 5.9) and Wilmer Font (TOR, 5.4). The bottom three using this method were determined to be Austin Adams (SEA, -0.7), Stephen Tarpley (NYY, -0.8) and Matt Carasiti (SEA, -0.9). From a team perspective, the Rays once again had the highest oWAR at 8.8 and the Yankees -0.16 represented the lowest team value. Most openers are still experiencing fluctuation based on the small amount of innings pitched.

Method 3: Nine Batters Faced

Most of the openers had innings pitched between 1.0 and 3.0. There were three pitchers who only threw 0.1 innings, and the highest amount was Wilmer Font, with 24.1 innings pitched. With this in mind, a third method was needed to help alleviate these small sample issues. The solution was to base the math on the other limit in the opener definition, facing no more than nine hitters. This new method calculated oERA by dividing the number of earned runs allowed by the number of batters faced, then multiplying by nine. This calculation produced numbers between 0.00 and 7.50. After determining new constants, and dividing by batters faced, instead of innings pitched, oFIP and oxFIP values were calculated. oFIP values ranged from -0.28 to 6.63, while oxFIP produced values between -0.28 and 3.16.

Finally, oWAR was calculated using this method as well. No adjustment to the formula was needed, other than using values based on batters faced instead of innings pitched. The adjustment to dRPW from the previous method wasn’t required, since 18 batters could be faced in games with openers. Yusei Kikuchi emerges as the oWAR leader with 1.31, followed by Joshua James (HOU, 0.92) and Hansel Robles (LAA, 0.90). The low end featured Jose Alvarado (TBR, -0.25), Ryan Tepera (TOR, -0.28) and, once again, Stephen Tarpley (NYY, -0.61).

Conclusion

Any discussion about openers eventually works its way to one central question: does employing the opener strategy lead to better results? While I didn’t have a chance to gather data about headliners for all the games featuring an opener, I did track those of the Mariners. Wade LeBlanc and Tommy Milone were the pitchers who were used in both headliner and starter roles. From the "Mariners in Depth" tab on the opener sheet, you can see the stats for each pitcher in each role. From the chart:

ERA FIP xFIP
LeBlanc (Starter) 8.35 6.56 5.35
LeBlanc (Headliner) 4.09 5.05 5.10
Milone (Starter) 4.65 4.31 4.16
Milone (Headliner) 4.94 5.44 5.22

We see that while LeBlanc was slightly better as a headliner than starter, Milone put up better numbers as a starter. The similarity in the numbers is more glaring when looking at the stats from games with and without an opener. The rate statistics are basically the same in each type of game. These, along with ERA, FIP and xFIP, are shown below:

Game BB% K% ERA FIP xFIP LOB% HR/FB%
w/ Opener 8.2% 18.4% 4.45 5.40 5.07 74.9% 17.4%
w/o Opener 8.0% 20.1% 5.13 4.90 4.79 67.2% 16.1%
Overall 8.1% 19.8% 5.00 5.00 4.84 68.4% 16.4%

The numbers show that, at least for the Mariners, implementing the opener strategy didn’t lead to much of a difference in results.

When looking at openers, we are always going to run in to the issue of small sample sizes. Some openers will only have one or two appearances, leading to statistics based on less than four innings pitched. Presented here are three different methods to reduce the effects, and finding a quantitative way to measure the effectiveness of openers. For comparison, in the first two innings, the league posted an ERA/FIP/xFIP of 4.44/4.45/4.45, while the openers posted a 6.83/6.83/6.83 line, showing that overall, the openers had slightly worse stats than regular starters. This does not, however, mean that teams shouldn’t ever use openers. The results have been somewhat mixed, and other issues, such as forcing changes in opponents’ lineups and times through the order penalties have not been analyzed. The use of openers is still in its infancy, more teams may choose to use them in the future, or the strategy may just fade away.