Stats vs. Scouts
So I've been thinking about this post for about a year now, but I've never got around to writing it properly. Well, I have 45 minutes to kill, so it's as good a time as any. I realise that I'm probably preaching to the choir here but whatever.
"Stats vs. scouts" is one of the central arguments in baseball circles, leading to mildly horrifying levels of vitriol leveled at members of either side. It's one of Moneyball's lasting legacies, as anything that portrays the old guard as curmudgeonly incompetents doomed to fall under the armies of cleverness is going to provoke a reaction from said curmudgeonly incompetents (NB: I am not calling the old guard curmudgeonly incompetents), and rightly so.
The problem is of course that statistical analysis and scouting are by no means natural enemies - in fact, they're closely allied. Advanced statistics are proxies for good scouting. Sometimes they pick up things that scouts miss, and sometimes scouting will tell you things that stats will not, but they are complementary. A team's front office doesn't have to be one or the other - the point is that both methods are tools to evaluate players.
Interestingly, the argument doesn't even seem to be about player evaluation, and the dogmatic camps generally make absolutely no sense at all. Those in the "scouts" camp spout off about RBI and batting average (which are statistics last time I looked), and people trying to be "statsy" will misuse numbers they don't understand to come to stupid conclusions. Does anyone* seriously think that 'gritty' would be the main point in a real scouting report? Or that analysts throw random numbers at the wall until the come up with their stats?
No, this isn't about statistics against scouting. Not at all. It's about people believing that they're already experts on baseball player evaluation. Imagine if people took the same attitude towards, say, engineering as they did baseball. You'd see people looking at construction sites with total disdain:
"I've walked through a lot of buildings and I can say for sure that using an eccentric braced frame for the lateral force resisting system is completely stupid. SCBFs are much better in the clutch."
Watching a lot of baseball doesn't mean you know anything. Playing a lot of baseball doesn't mean you know anything. Watching a lot of baseball while listening to the opinions of people who've played a lot of baseball doesn't mean you know anything. But it's amazing how many people seem to think it does.
It's not stats against scouts, and it never has been. It's acknowledging one's own ignorance against the belief that one already knows everything.
Because if you already know everything, why would you ever need to think about it?

*Well, yes I'm sure someone does, but if that person is you and you feel inclined to comment on it you're going to end up banned so don't.
13 recs |
47 comments
Comments
Rec'd and bookmarked to I can link it to other sites whenever this topic comes up.
Every day I hear about Seattle sports' failures. Every night I fall asleep to the sound of my own tears.
by Benne on
Nov 21, 2008 1:31 PM PST
reply
actions
0 recs
Calibrating judgment
My mantra is that the purpose of quantitative information is to calibrate judgment. Even in structural engineering, where most of the design may done with calculations, at some point an experienced person needs to look at the output and decide, “does this make sense?”
Part of the process is assessing the quality of the numeric information with which we are working. The better the quantitative data, the more weight we give to the numbers.
by Steve Nelson on
Nov 21, 2008 1:59 PM PST
reply
actions
0 recs
True of engineering, but I'm not sold that it's true here
Baseball player evaluation is a lot more fluid than structural design. Ultimately I think both statistical analysis and scouting need to be used as checks for each other.
by Graham on
Nov 21, 2008 2:05 PM PST
up
reply
actions
0 recs
If they clash, which do you trust?
It’s an interesting philosophical question – if you’re breaking new ground and your results clash with established wisdom, does that mean your formula fails the reasonableness test, or that you’re revealing faults in the established wisdom?
Nowadays it seems there’s enough work that there are baselines from which to start, but I wonder how Bill James and the other early trailblazers handled questions like those. If you’re Bill James and the late 1970s version of Raul Ibanez is racking up RBIs and is well-respected by virtually everybody in baseball (left-handed sock!), what do you think when your formulas show he’s essentially of no incremental worth?
There’s no baseline at that point. Do you just toss out the formulas as not passing the smell test, or do you take your result as gospel and start telling the baseball establishment they don’t know how to value players?
by Chris Hafner on
Nov 21, 2008 3:05 PM PST
up
reply
actions
0 recs
Well
Back in the day that definitely would have been an issue. The reason we’re seeing a lot of progress now is the vast improvement in ability to collect data. When evaluating Raul’s defense, for example, now we have ball-in-play data to figure out how many plays an average defender makes vs. how many he made.
by Milendriel on
Nov 23, 2008 1:22 AM PST
up
reply
actions
0 recs
I'm always learning
I remember when I though FRAR was worth a damn!
Furcal
by JI on
Nov 21, 2008 2:20 PM PST
reply
actions
0 recs
People cling to FRAR because it's available historically.
They’re not willing to accept that we just can’t measure defense from 1960, so they’re going to use something that purports to.
I like using semi-colons; they make me feel smart.
by Llewdor on
Nov 21, 2008 3:21 PM PST
up
reply
actions
0 recs
My introduction to sabermetrics was WARP.
by Teej on
Nov 21, 2008 3:43 PM PST
up
reply
actions
0 recs
I got into sabermetrics from FJM.
So yeah, WARP, VORP and EqA.
by Teej on
Nov 21, 2008 3:56 PM PST
up
reply
actions
0 recs
God you all make me feel old.
My intro was most likely defensive average in 1993. I honestly do not remember who devised it.
by Sec 108 on
Nov 21, 2008 4:06 PM PST
up
reply
actions
0 recs
Maybe if you cared more about it you'd be running this website and I'd have more free time in my life
by Jeff on
Nov 21, 2008 4:06 PM PST
up
reply
actions
0 recs
If I was running this website I would not be married to the coolest woman in the world.
by Sec 108 on
Nov 21, 2008 4:14 PM PST
up
reply
actions
0 recs
Let's try this then.
If I was running this website no one would read it.
Therefore you would be running some other website with tons of traffic.
by Sec 108 on
Nov 21, 2008 4:18 PM PST
up
reply
actions
0 recs
I started at the beginning of '04 with USSM
by Graham on
Nov 21, 2008 4:06 PM PST
up
reply
actions
0 recs
Me too.
That was the Eric Hinske issue, right?
I like using semi-colons; they make me feel smart.
by Llewdor on
Nov 21, 2008 4:21 PM PST
up
reply
actions
0 recs
And I still like EqA.
An offensive metric that includes stolen bases. Nice.
I like using semi-colons; they make me feel smart.
by Llewdor on
Nov 21, 2008 4:22 PM PST
up
reply
actions
0 recs
It doesn't, but it does include times reached base via error
which is a sort-of proxy for speed.
by Matthew on
Nov 21, 2008 9:34 PM PST
up
reply
actions
0 recs
Bill James Baseball Abstract . . . 1984?
I think it was ‘84. I learned to use spreadsheets setting them up to calculate RC and RC/G; and I’ve lost count of how many players I cranked through the Brock2 spreadsheet. (I even took to inventing players according to certain types to see how they aged.)
by The Ancient Mariner on
Nov 21, 2008 6:30 PM PST
up
reply
actions
0 recs
I have the re-released Bill James Abstract.
Lots of good stuff in there. What a writer. I keep it by my bed and crack it every couple of weeks.
by Teej on
Nov 21, 2008 8:22 PM PST
up
reply
actions
0 recs
It's impossible for me to read straight through.
He digresses or mentions other players and seasons, I start skipping around.
by dpseadv on
Nov 22, 2008 2:40 AM PST
up
reply
actions
0 recs
Rec'd
It’s about time we stop with the retarded squabbling.
Fans are typically idiots.
by The Typical Idiot Fan on
Nov 21, 2008 4:09 PM PST
reply
actions
0 recs
As it comes down to in many entrenched debates,
it’s not actually about Side A vs Side B, it’s about closed vs open minds (which will exist on both sides).
by Matthew on
Nov 21, 2008 9:35 PM PST
reply
actions
0 recs
Am I wrong to be this excited about Zduriencik?
The fact that a guy with such a scouting background has come in and done this is blowing my mind. I don’t know how much of that is based on how shitty his predecessor was, but I have more faith than I’ve had in years.
by Teej on
Nov 21, 2008 10:33 PM PST
up
reply
actions
0 recs
No, Zduriencik has been awesome so far
Yeah, he’s not the 30-year-something Ivy League hotshot we’ve all been hoping for, but I’m absolutely psyched that our GM is embracing both traditional scouting and sabermetrics.
Every day I hear about Seattle sports' failures. Every night I fall asleep to the sound of my own tears.
by Benne on
Nov 22, 2008 1:59 AM PST
up
reply
actions
0 recs
Maybe he'll hire
some 3020-year something Ivy League hotshot(s)…
by Matthew on
Nov 22, 2008 12:12 PM PST
up
reply
actions
0 recs
This might be better
Not to put down the younger guys by any stretch, but when a veteran scout with a record of success like Zduriencik’s comes in and sets up his most trusted assistant to build an entire research department — especially given how good Zduriencik has proven himself to be at dealing with people over the course of his career — if he shows himself to be as open-minded in listening
to that department as he has in creating it in the first place, that could be the best possible combination.
by The Ancient Mariner on
Nov 22, 2008 12:14 PM PST
up
reply
actions
0 recs
This is pretty much
exactly what I’d been thinking about this whole thing, too. Bottom line is, there are people who want to be right and approach new information objectively, and then there are people who don’t want to admit they’re wrong—which is necessary to eventually be right; none of us were any good at evaluating from the outset.
by Milendriel on
Nov 22, 2008 5:02 AM PST
reply
actions
1 recs
Very well put
You can say pretty much the same thing in any part of life — and that says it about as well as I’ve ever seen.
by The Ancient Mariner on
Nov 22, 2008 12:15 PM PST
up
reply
actions
0 recs
Wow, that ~$1700, and it's only the weekend.
I’m surprised this post hasn’t been rec’d enough to make the recommended fanpost list.
by dpseadv on
Nov 22, 2008 10:04 PM PST
reply
actions
0 recs
Front page post.
It's hard to convince people to let you eat them if you're an asshole. - Thingray
by Faux on
Nov 23, 2008 3:00 AM PST
up
reply
actions
0 recs
And I put this comment in the wrong post anyway, what a dummy.
I was experimenting with multiple open windows.
by dpseadv on
Nov 23, 2008 10:20 AM PST
up
reply
actions
0 recs
This is nice to hear...
I think that evaluation of players is not unlike American laws; Congress writes the laws (stats) which are then interpreted by judges (scouts). Enforcing all laws to the letter would make as much sense as ranking players statistically and then aquiring them in that orders regardless of intangible circumstances. As usual, extremism for the loss.
by Azimeir on
Nov 23, 2008 8:47 PM PST
reply
actions
0 recs







