Stats vs. Scouts
So I've been thinking about this post for about a year now, but I've never got around to writing it properly. Well, I have 45 minutes to kill, so it's as good a time as any. I realise that I'm probably preaching to the choir here but whatever.
"Stats vs. scouts" is one of the central arguments in baseball circles, leading to mildly horrifying levels of vitriol leveled at members of either side. It's one of Moneyball's lasting legacies, as anything that portrays the old guard as curmudgeonly incompetents doomed to fall under the armies of cleverness is going to provoke a reaction from said curmudgeonly incompetents (NB: I am not calling the old guard curmudgeonly incompetents), and rightly so.
The problem is of course that statistical analysis and scouting are by no means natural enemies - in fact, they're closely allied. Advanced statistics are proxies for good scouting. Sometimes they pick up things that scouts miss, and sometimes scouting will tell you things that stats will not, but they are complementary. A team's front office doesn't have to be one or the other - the point is that both methods are tools to evaluate players.
Interestingly, the argument doesn't even seem to be about player evaluation, and the dogmatic camps generally make absolutely no sense at all. Those in the "scouts" camp spout off about RBI and batting average (which are statistics last time I looked), and people trying to be "statsy" will misuse numbers they don't understand to come to stupid conclusions. Does anyone* seriously think that 'gritty' would be the main point in a real scouting report? Or that analysts throw random numbers at the wall until the come up with their stats?
No, this isn't about statistics against scouting. Not at all. It's about people believing that they're already experts on baseball player evaluation. Imagine if people took the same attitude towards, say, engineering as they did baseball. You'd see people looking at construction sites with total disdain:
"I've walked through a lot of buildings and I can say for sure that using an eccentric braced frame for the lateral force resisting system is completely stupid. SCBFs are much better in the clutch."
Watching a lot of baseball doesn't mean you know anything. Playing a lot of baseball doesn't mean you know anything. Watching a lot of baseball while listening to the opinions of people who've played a lot of baseball doesn't mean you know anything. But it's amazing how many people seem to think it does.
It's not stats against scouts, and it never has been. It's acknowledging one's own ignorance against the belief that one already knows everything.
Because if you already know everything, why would you ever need to think about it?
*Well, yes I'm sure someone does, but if that person is you and you feel inclined to comment on it you're going to end up banned so don't.
47 comments
|
13 recs |
Do you like this story?
Comments
Rec'd and bookmarked to I can link it to other sites whenever this topic comes up.
Every day I hear about Seattle sports' failures. Every night I fall asleep to the sound of my own tears.
Calibrating judgment
My mantra is that the purpose of quantitative information is to calibrate judgment. Even in structural engineering, where most of the design may done with calculations, at some point an experienced person needs to look at the output and decide, “does this make sense?”
Part of the process is assessing the quality of the numeric information with which we are working. The better the quantitative data, the more weight we give to the numbers.
True of engineering, but I'm not sold that it's true here
Baseball player evaluation is a lot more fluid than structural design. Ultimately I think both statistical analysis and scouting need to be used as checks for each other.
by Graham MacAree on Nov 21, 2008 2:05 PM PST up reply actions
If they clash, which do you trust?
It’s an interesting philosophical question – if you’re breaking new ground and your results clash with established wisdom, does that mean your formula fails the reasonableness test, or that you’re revealing faults in the established wisdom?
Nowadays it seems there’s enough work that there are baselines from which to start, but I wonder how Bill James and the other early trailblazers handled questions like those. If you’re Bill James and the late 1970s version of Raul Ibanez is racking up RBIs and is well-respected by virtually everybody in baseball (left-handed sock!), what do you think when your formulas show he’s essentially of no incremental worth?
There’s no baseline at that point. Do you just toss out the formulas as not passing the smell test, or do you take your result as gospel and start telling the baseball establishment they don’t know how to value players?
by Chris Hafner on Nov 21, 2008 3:05 PM PST up reply actions
Well
Back in the day that definitely would have been an issue. The reason we’re seeing a lot of progress now is the vast improvement in ability to collect data. When evaluating Raul’s defense, for example, now we have ball-in-play data to figure out how many plays an average defender makes vs. how many he made.
People cling to FRAR because it's available historically.
They’re not willing to accept that we just can’t measure defense from 1960, so they’re going to use something that purports to.
I like using semi-colons; they make me feel smart.
God you all make me feel old.
My intro was most likely defensive average in 1993. I honestly do not remember who devised it.
Maybe if you cared more about it you'd be running this website and I'd have more free time in my life
by Jeff Sullivan on Nov 21, 2008 4:06 PM PST up reply actions
Let's try this then.
If I was running this website no one would read it.
Therefore you would be running some other website with tons of traffic.
I started at the beginning of '04 with USSM
by Graham MacAree on Nov 21, 2008 4:06 PM PST up reply actions
And I still like EqA.
An offensive metric that includes stolen bases. Nice.
I like using semi-colons; they make me feel smart.
Bill James Baseball Abstract . . . 1984?
I think it was ‘84. I learned to use spreadsheets setting them up to calculate RC and RC/G; and I’ve lost count of how many players I cranked through the Brock2 spreadsheet. (I even took to inventing players according to certain types to see how they aged.)
by The Ancient Mariner on Nov 21, 2008 6:30 PM PST up reply actions
I have the re-released Bill James Abstract.
Lots of good stuff in there. What a writer. I keep it by my bed and crack it every couple of weeks.
It's impossible for me to read straight through.
He digresses or mentions other players and seasons, I start skipping around.
Rec'd
It’s about time we stop with the retarded squabbling.
Fans are typically idiots.
by The Typical Idiot Fan on Nov 21, 2008 4:09 PM PST reply actions
As it comes down to in many entrenched debates,
it’s not actually about Side A vs Side B, it’s about closed vs open minds (which will exist on both sides).
Am I wrong to be this excited about Zduriencik?
The fact that a guy with such a scouting background has come in and done this is blowing my mind. I don’t know how much of that is based on how shitty his predecessor was, but I have more faith than I’ve had in years.
No, Zduriencik has been awesome so far
Yeah, he’s not the 30-year-something Ivy League hotshot we’ve all been hoping for, but I’m absolutely psyched that our GM is embracing both traditional scouting and sabermetrics.
Every day I hear about Seattle sports' failures. Every night I fall asleep to the sound of my own tears.
This might be better
Not to put down the younger guys by any stretch, but when a veteran scout with a record of success like Zduriencik’s comes in and sets up his most trusted assistant to build an entire research department — especially given how good Zduriencik has proven himself to be at dealing with people over the course of his career — if he shows himself to be as open-minded in listening
to that department as he has in creating it in the first place, that could be the best possible combination.
by The Ancient Mariner on Nov 22, 2008 12:14 PM PST up reply actions
This is pretty much
exactly what I’d been thinking about this whole thing, too. Bottom line is, there are people who want to be right and approach new information objectively, and then there are people who don’t want to admit they’re wrong—which is necessary to eventually be right; none of us were any good at evaluating from the outset.
by Milendriel on Nov 22, 2008 5:02 AM PST reply actions 1 recs
Very well put
You can say pretty much the same thing in any part of life — and that says it about as well as I’ve ever seen.
by The Ancient Mariner on Nov 22, 2008 12:15 PM PST up reply actions
Wow, that ~$1700, and it's only the weekend.
I’m surprised this post hasn’t been rec’d enough to make the recommended fanpost list.
Front page post.
It's hard to convince people to let you eat them if you're an asshole. - Thingray
And I put this comment in the wrong post anyway, what a dummy.
I was experimenting with multiple open windows.
This is nice to hear...
I think that evaluation of players is not unlike American laws; Congress writes the laws (stats) which are then interpreted by judges (scouts). Enforcing all laws to the letter would make as much sense as ranking players statistically and then aquiring them in that orders regardless of intangible circumstances. As usual, extremism for the loss.













