Evaluating Starts: An Example

by Matthew on Apr 7, 2008 10:20 PM PDT

How many of you think ERA is a good way to measure pitchers? How about hits allowed?

I'm hoping few of said yes. It's been a mini-crusade of Dave Cameron, Graham, Jeff, myself and others to hammer home the point that ERA has tremendous faults in it as far as pitcher measurement metric. And for the most part, that seems to be accepted on a macro level. Here's the rub though, if you accept it on a grand scale, over a season for example, you should accept it on the smaller scale of individual games as well.

Carlos Silva's line tonight as read by most people:
7IP, 9H, 4ER, 5.14 ERA

Carlos Silva's line tonight as you should read it:
30BF, 5K, 0BB, 0HBP, 1HR, 8-8 GO/FO

If you accept that FIP and tRA and such as the way to go in terms of measuring pitchers, then stick with that. I don't have real-time tRA's set up yet, I'm working with Graham on this, but FIP is easy to measure,

one HR = +13, 0 (BB+HBP) = 0, five K = -10. +13 + 0 -10 = +3. IP = 7. 3/7 + 3.2 = 3.63.

A 3.63 FIP would have been the 7th best mark in the AL in 2007. It's a good number. Just outside ace territory.

If you want to talk about hitting his spots, or his velocity, or his stuff, or something scout-related; fine. That's legit. But if you want to talk results; Carlos Silva pitched a very good game today.

Tweet Comment 61 comments | 2 recs |

Do you like this story?

Comments

Hmm

I would have put $100 on this being about Tim Redding's line tonight: 4 IP, 7 H, 7 R, 1 ER, 2 BB, 3 K, 1 HR

ERA for the performance: 2.25
FIP for the performance: 6.45

by davidcameron on Apr 7, 2008 10:33 PM PDT reply actions

example 1B

by Matthew on Apr 7, 2008 10:36 PM PDT up reply actions

This is why

when recapping minor league games, I try to include the BIP stats for SPs, as that paints a much clearer picture of how the start went than the line score.

by Gomez on Apr 7, 2008 11:05 PM PDT reply actions

if you are truly going to crusade

about ERA.... then you need to start hammering home the point to the mainstream stat sites because until FIP and tRA are included on the stats pages for the players its never going to replace ERA for most peoples measuring tool.

There just aren't enough fans that understand the concept of it, heck most don't actually no ERA, so they're not going to go using stats that they don't have readily available. Game annoucers aren't going to use stats that the casual fans can't reference themselves either.

Midnight Baseball - No Lights - Only in Alaska!

by MfaninAlaska on Apr 7, 2008 11:32 PM PDT reply actions

It can't catch on if people don't talk about it.

...and now I'm here

by CapSea on Apr 7, 2008 11:38 PM PDT up reply actions

which is why

I recommended hammering the point home to the mainstream stats sites, to increase the talk about those stats.

Midnight Baseball - No Lights - Only in Alaska!

by MfaninAlaska on Apr 7, 2008 11:45 PM PDT up reply actions

I think their would response would involve

"Nerds computers Billy Beane clogging the bases, etc."

We don't negotiate with terrorists.

by Mariner John on Apr 8, 2008 9:53 PM PDT up reply actions

Sure

tRA>FIP>ERA for evaluating a pitcher in one game but I don't think that any of those are nearly good enough. Condensing down a bunch of useful numbers into one number when you are dealing with a small sample is probably gonna be useless 90% of the time. For example, Silva's stuff is not good enough to consistantly produce a FIP of 3.6 even if he pitched as good as he did the other night. The best single number I think out there is WPA.

It might also be interesting to try to use swinging strikes, ect to create something like FIP. This would probably effectively "increase" your sample size because you'd be able to get information from each pitch instead of each batter.

by Edgar for Pres on Apr 7, 2008 11:41 PM PDT reply actions

I think at some point you just have to accept that people are dumb, Matthew

by Graham MacAree on Apr 8, 2008 1:25 AM PDT reply actions

The proper starting point is to accept

That even with as much knowledge as you may have, you actually know very little. Then, since you have more knowledge than most others in humanity, humanity is seriously lacking in intelligence.

The user formerly known as Sec 108.

by Sec 108 on Apr 8, 2008 8:29 AM PDT up reply actions

True.

Maybe I'll start hanging out with dolphins; they seem to have it together as a species. No work, just travel (swim) and recreational sex. What's not to like?

by Matthew on Apr 8, 2008 8:39 AM PDT up reply actions

42

by Fett42 on Apr 8, 2008 9:02 AM PDT up reply actions

Tuna nets.

Nice Guys Finish Third - Hopelessly lost, but makin' good time.

by pdb on Apr 8, 2008 9:03 AM PDT up reply actions

that's how we weed out the dumb dolphins

it's a win-win!

by Matthew on Apr 8, 2008 9:26 AM PDT up reply actions

You did know, of course.....

that some of that recreational sex you mentioned is actually a group of mails gang-raping a female. And that adults have been known to often batter and eventually kill baby dolphins of not only other species, but of their own as well. You did know that, right?

by nathaniel dawson on Apr 8, 2008 6:17 PM PDT up reply actions

males, already

by nathaniel dawson on Apr 8, 2008 6:18 PM PDT up reply actions

That sounds like Matthew.

...and now I'm here

by CapSea on Apr 8, 2008 7:59 PM PDT up reply actions

I fear dolphins.

We don't negotiate with terrorists.

by Mariner John on Apr 8, 2008 9:54 PM PDT up reply actions

A counter-example

Jon Garland's first start:

8 IP, 6 H, 1 ER, 1 BB, 0 K

Jon Garland doesn't care about your statistical mumbo-jumbo. He knows how to pitch, and doesn't need to rely on the strikeout. BABIP and other spreadsheet analysis crumbles in the veteran pitching know-how of this ace. But go ahead and reduce humans to numbers, I would too if I was a suicidally depressed coffee guzzling umbrella carrying Mariner fan.

by Fett42 on Apr 8, 2008 5:49 AM PDT reply actions

what kind of real Seattlite actually carries an umbrella?

I don't even think I own one.

by Matthew on Apr 8, 2008 8:38 AM PDT up reply actions

Likewise.

by Fett42 on Apr 8, 2008 9:02 AM PDT up reply actions

I didn't buy one until I left WA. Now I have 3 in PA.

by JuliaHope on Apr 8, 2008 9:34 AM PDT up reply actions

Ditto.

I've actually never seen a guy with an umbrella.

by redwolf75 on Apr 8, 2008 9:51 AM PDT up reply actions

Umbrellas are for tourists.

by brayden04 on Apr 8, 2008 9:58 AM PDT up reply actions

it is a fairly reliable way of spotting them

at least if they're male. Women sometimes are overly dependent on them.

by Matthew on Apr 8, 2008 10:07 AM PDT up reply actions

Is it wrong that I refuse to hang out with men that use umbrellas?

The user formerly known as Sec 108.

by Sec 108 on Apr 8, 2008 11:21 AM PDT up reply actions

Don't move to SoCal

they use umbrellas when it's foggy.

by seattlebruin on Apr 8, 2008 12:48 PM PDT up reply actions

no

by Matthew on Apr 8, 2008 1:01 PM PDT up reply actions

When I moved here, I had one

I think I've seen about 3-4 situations tops in the last 2 years where one would have been of use.

by Gomez on Apr 8, 2008 12:34 PM PDT up reply actions

Me either.

...and now I'm here

by CapSea on Apr 8, 2008 12:58 PM PDT up reply actions

I taught a Washington immigrant (from the south)

You can always spot the native Washingtonians by the lack of an umbrella during a typical PNW autumn/winter/spring downpour. An umbrella doesn't stop the wind-blown rain. What's the friggin' point of an umbrella?

Ill Ligitamus Non Carberendum

by PositivePaul on Apr 8, 2008 1:34 PM PDT up reply actions

If you're sufficiently nimble, you can block splashes from passing traffic with it.

That's why I use one.

by Llewdor on Apr 8, 2008 2:32 PM PDT up reply actions

Also, you're a pansy.

And you don't live in Seattle, so you're exempt.

...and now I'm here

by CapSea on Apr 8, 2008 2:38 PM PDT up reply actions

Wow this got alot more threads off it than I expected (0)

by Fett42 on Apr 8, 2008 3:05 PM PDT up reply actions

umbrella usage is a hot topic

by Matthew on Apr 8, 2008 3:09 PM PDT up reply actions

So I ran tRA for the one start, Matthew

4.79. The line drives are a killer.

by Graham MacAree on Apr 8, 2008 6:06 AM PDT reply actions

So we can all agree

He was somewhere between replacement level (ERA) and All star (FIP) and maybe right in between around league average (tRA). Thanks guys, that really cleared up a lot of things. ;)

by Edgar for Pres on Apr 8, 2008 7:34 AM PDT up reply actions

We do what we can

by Graham MacAree on Apr 8, 2008 7:39 AM PDT up reply actions

Could you run tRA for Joe Saunders last night?

At 8 IP, 1HR, 2BB, 5K, 7-11GO-AO, I can only assume FIP probably rates Silva's start as better. However, Joe's actual 4H, 1ER, 1.13 ERA suggests he was much better than Silva.

~Till the Halo burns out...

by Zu Long on Apr 8, 2008 9:20 AM PDT up reply actions

Not sure if I'm doing this right

but I'm getting 4.04.

by Jeff Sullivan on Apr 8, 2008 9:36 AM PDT up reply actions

3.09

This is what happens when you give up 1 line drive

by Graham MacAree on Apr 8, 2008 9:36 AM PDT up reply actions

NB: That's tRA, the number Jeff gave you was tRA*.

Also, these numbers won't really reflect the start completely accurately because the park factors shouldn't be halved in the case of one start. But I can't be bothered actually going into the inner workings of the spreadsheet and fixing that.

by Graham MacAree on Apr 8, 2008 9:39 AM PDT up reply actions

Could you post the formula again

Looks like the spreadsheet (which used to have it) now just has the starters/relievers, AL/NL results for 2007.

What do you have for the batted ball types? That ground out/air out thing only works for...uh, outs. ESPN's PBP isn't being all that helpful (what a shock!), so where do you get the batted ball types for the hits?

by marc w on Apr 8, 2008 9:47 AM PDT up reply actions

I drag the batted ball types out of gameday.

It's a pretty annoying process so we're working on getting that all automated.

The run values I use are here:

The formula used for tRA is = (average run value of a plate appearance against pitcher x)*batters faced/innings pitched*9. And the raw numbers are all park adjusted, etc...

tRA*'s much messier because it goes a bunch of regression calcs that hurt my brain when I have to figure them out all over again, so I won't give you that one.

Also, the fact that I divide into IP is probably not a good thing, because batters faced/IP is another thing influenced by luck... oh well. At least using ROA avoids this.

by Graham MacAree on Apr 8, 2008 10:05 AM PDT up reply actions

how are these expected

run values derived?
Why are intentional walks ignored? A pitcher got himself into the position of having to give the intentional walk why would he get a free pass on it.

I'm not trying to be an ass, but I've always wondered how the exact run values were derived.

Midnight Baseball - No Lights - Only in Alaska!

by MfaninAlaska on Apr 8, 2008 10:24 AM PDT up reply actions

Long and thorough explanation in The Book

But I'm sure Graham and/or Matthew can sum it up much better than I.

by Fett42 on Apr 8, 2008 10:32 AM PDT up reply actions

Those specific ones are from THT's 2007 Annual

As far as I'm aware, they're all experimental, and derived from the average change that each of those events causes to a run expectancy matrix over the course of a year/number of years. I haven't tested this out to see if it yields similar results, because I'm not that good at retrosheet, but it should be close.

Intentional walks aren't ignored, the distinction between BB and IBB is ignored. Which actually hurts the pitcher, because IBB are worth less than unintentional free passes. The reason I did this was just simplicity in the early stages of work - I'll get around to changing it someday.

by Graham MacAree on Apr 8, 2008 10:37 AM PDT up reply actions

Graham's explained it

but just to give an idea of scale, the numbers I've seen put the delta in run value between a BB and an intentional BB at about .09. So if a walk is worth .355, an intentional walk would be about .265. It's the -0.09 that's being left out, not the walk itself.

by marc w on Apr 8, 2008 10:41 AM PDT up reply actions

I appreciate the info

thanks guys.

Midnight Baseball - No Lights - Only in Alaska!

by MfaninAlaska on Apr 8, 2008 10:43 AM PDT up reply actions

That's sort of what I figured.

Given our discussion, of park factors earlier, I'm guessing that correcting your spreadsheet would modify the result by all of a few hundreths at most, so there isn't much point in doing so.

So far then, I have to say tRA is doing a much better job of rating single start quality, just from an observational standpoint, than ERA or FIP. Saunders pitched really well last night, and tRA reflects that. It recognizes that the high number of basehits was due to Silva's line drive rate, and penalizes him for it. Meanwhile, I'm guessing it maintain's FIP's strength of recognizing Garland's start as very fluky, and Tim Redding as having been horrific even including bad defense behind him.

~Till the Halo burns out...

by Zu Long on Apr 8, 2008 11:42 AM PDT up reply actions