On Fan Bias
by Jeff Sullivan on Dec 1, 2009 12:17 AM PST
in Seattle Mariners Stats
Email
PrintYou may have noticed that Fangraphs is all set up for community projections now. This is the same thing USSM/we used to do, only for everybody in baseball. It's a really cool feature that's remarkably simple to use, and one neat little twist they threw in was including the rooting interest of the user inputting data. You might remember that, when we would run these things every year, there would always be some level of concern that, being Mariner fans, we might be a little optimistic about our own players. Fangraphs now allows us to investigate that possible bias.
There are five Mariner hitters who, so far, have received at least ten projections from both Mariner fans and non-Mariner fans. Here are their projected 2010 wOBAs:
| Player | M's fan | non-M's fan |
| Ichiro | 0.359 | 0.359 |
| Guti | 0.344 | 0.329 |
| Lopez | 0.327 | 0.325 |
| Branyan | 0.369 | 0.349 |
| Beltre | 0.349 | 0.346 |
No difference for three of them. Guti gets a little bit of a 2B/HR boost, while Branyan's difference comes down to 30 homers in 508 PAs versus 24 in 475. It could be bias, or it could be that M's fans know a little more about their players than people who didn't watch them as much.
(Both Griffey and Jack Wilson have a bunch of projections from Mariner fans and 5 from non-Mariner fans. There is no meaningful difference. Felix Hernandez, meanwhile, has FIP projections of 3.05 and 3.30 over equivalent innings.)
Looking through these numbers, and going through some of the early projections for players on other teams, two things become apparent:
(1) In nearly every circumstance, the fan projection is equal to or better than the non-fan projection
(2) Overall, the difference between the fan projections and the non-fan projections is small
Does optimistic fan bias exist within this exercise? It does appear to, yes. On average, fans are a bit higher on their own players than they are on the rest. But the effect of this bias is so small that, for all intents and purposes, it doesn't seem to matter. I suppose that, if you really wanted to, you could come up with a list of reasons to be skeptical of the Wisdom Of Crowds approach to player projection, but you can probably go ahead and cross "bias" right off.
32 comments
|
0 recs |
Do you like this story?
Comments
NB: In the coming days/weeks, I am going to be looking at just this sort of breakdown
for each team, so if that’s something that interests you, sit tight just a little bit.
Wonder which team has the most biased fan base!
Man, Mariner fans love to project.
by Jeff Sullivan on Dec 1, 2009 12:22 AM PST up reply actions
I hope it's some obscure team.
I need more enemies at FanGraphs.
Seattle, with its blogosphere and tech-oriented populace, has to be the most wired of fanbases.
I'd imagine it will be a team with a very large fan base and a lot of 'casual' fans who stumble upon Fangraphs and put in poorly considered projections.
Red Sox and Yankees are the obvious choices.
by EnglishMariner on Dec 1, 2009 6:32 AM PST up reply actions
Biased against
I can see the Mariners fans being optimistic in their projections. I look forward to seeing what, say, Pittsburgh Pirates or Kansas City Royals fans project. I’m guessing it’s going to be slightly more pessimistic.
MIKE JACOBS!
HE WILL SAVE THE ROYALS!
Say it with me: Washington Capitals. Capitals.
Preserved In All His Greatness - R.I.P. The Reignman 1989 to 1997
Fantastic.
When reading this, I was wondering just how much an impact LL/USSM have had with respect to tempering bias. I guess we’ll find out.
Does anyone else here choose not to take part in these projections because they feel they are not wise enough to put in a decent projection?
I don’t want to stumble along and put a projection in that is way off base when lots of smarter people will be spending a lot of time formulating their results. I know that the sample size will probably be large enough by the end of the off season whereby the dumb projections are filtered out, but it still concerns me that for lesser known players it may have an impact on the final results.
I stay away from projecting pitchers in general and relievers in particular for this reason
but hitters are quite a lot easier. Projecting veteran major leaguers is kind of a boring exercise since they have well established talent levels and it becomes a home park/aging issue. But prospects or sophomore players are a ton of fun to project and here a fan’s knowledge becomes incredibly useful. Don’t be shy because you’re not a scout, that’s what the crowd is for.
De Gutibus non disputandum est
by Bearskin Rugburn on Dec 1, 2009 7:11 AM PST up reply actions
Yes.
I will smash your face into a jelly.
by Phil Hatzenbuehler on Dec 1, 2009 7:43 AM PST up reply actions
Sometimes. If I'm feeling my oats I'll go with what I know.
If I’m not I’ll write mine down on an envelope before looking and see where I’m at. Just with the players I’m familiar with, more often than not I’m in the sheaf.
I don't think you can cross bias off
Most Mariners fans are casual. If you did a survey of the people who went to fangraphs you would probably find that they are more educated baseball fans who have a better understanding of players values. Fangraphs only uses a certain demographic that doesn’t encompass the entire fanbase. Plus 10 isn’t a big sample for each player.
But we're only worried about the community projections
And community projections are filled out by a pretty specific kind of fan.
by Jeff Sullivan on Dec 1, 2009 8:32 AM PST up reply actions
Is it bias, or are Mariners fans just smarter? Or at least the ones who read Fan Graphs.
Racer X. You have to love those amarillo hops.
p.s. fuck you angels
It's probably bias
The easy way to see this is that fans systematically project their players higher than non-fans. If fans knew more, we would expect to see them projecting some players worse than non-fans and some better than non-fans, because we already know that some players will under-perform their projections and some over-perform their projections; fans seem to think that everyone will over perform them, which is nuts.
I was joking.
Racer X. You have to love those amarillo hops.
p.s. fuck you angels
I remember a couple years ago
trying to work through some of the projection stuff. I think we tended to be about 10-20% higher than we should have been basically across the board on players. If I remember right most of it was projecting too much playing time for regulars however I think we tended to predict more bounce back years than happened (for example Richie Sexson).
Over-projection of play time is rampant
Fans just don’t realize how high the chance of injury is.
Injury is never the most likely situation
So it doesn’t make sense for someone to, say, go to Gutierrez’s page and project 50 at bats. What’s the point?
by Jeff Sullivan on Dec 1, 2009 10:45 AM PST up reply actions
What if you account for that somehow?
Like an area where you could include some sort of selection on the variance/consistency/confidence of the other numbers you are selecting. Then, when simulations are run for the whole team based on individual projections, a team with younger or less experienced players would have greater variance in their projected win total than a team with more established players.
I always felt like this was one of the weaknesses of some of the team projection systems. Sure, the Rangers or A’s could have big improvements from their rookies/young players, but their greater variance means they could also plummet down like the flaming wreckage of a 747 (Hi, Chris Davis!).
If Brad Pitt is playing Beane who do you want playing you?
JD: Eddie Guardado.
by GhettoBear04 on Dec 1, 2009 12:52 PM PST up reply actions
Fan Bias isn't hard to account for
You can factor the bias out to aggregate projections in a straight forward way. The methods for doing so aren’t perfect, but they will go a long way toward eliminating bias.
One way to do it is to compare fan’s projections of the MLB average OBP, SLG and AVG. Suppose fans projections of these figure predicts and increase over last season of .014 in each of these. That tells us a lot about the fan projections, namely that they are about .014 too high. Normalizing the entire population in some systematic way so that the average comes out to be the same as last year will account for most of the bias. I’m not sure exactly how to do this, and there’s more than one way, but almost any way of doing it will go a long way toward removing the bias from the projections.
Do you think bias can exist in Tango's fan scouting reports? Just out of interest
They are voted upon fans of the respective team, though anyone can technically vote for any player.
Also, I would think that report is largely voted upon by SABR trained community. The ratings are meant to be what is seen by the eye, but I am inclined to base my ratings on other data such as Plus/Minus and UZR ratings. This is because SABR stats essentially question the any illusions that come from personal perception. Basically, I am wondering if people voted for Zimmerman as a great fielder, due to difficulties separating their knowledge of his guady fielding stats with what they saw him as while playing, someting a great deal harder to interpret(especially from a tv.) For this reason, I usually have great difficulty voting on them
by tdot mariner fan on Dec 1, 2009 10:36 AM PST reply actions
Well, it's not just voted on by the SABR trained community - Tango solicits ratings from casual fans too.
We’ve talked about potential bias in the fan scouting reports; I’ve been worried that you’ll get 300 responses that attempt to align with Dave Cameron’s view of a player, but Tango’s made some very good arguments that there’s a lot less bias than I thought.
It’s part of the reason why I’m not as shocked by the results Jeff’s found in this post… a year or two ago I would found these results incredible.
Yeah, it's not overly optimistic/pessimistic bias...
…that I’m concerned about. In the end, they either will cancel each other out or can be corrected for. It’s groupthink that would be harder to parse out (ie, Guti being rated much higher and Jarrod Washburn being ranked much lower because everyone love Jeff).
If Brad Pitt is playing Beane who do you want playing you?
JD: Eddie Guardado.
by GhettoBear04 on Dec 1, 2009 12:55 PM PST up reply actions
There's a worry about bias generated by media
I can imagine that if a local sports writer or broadcaster is always talking about how good so and so is with a glove, it gets reflected in fans scouting report. That sort of systematic bias is harder to isolate.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Recommended FanPosts
Recent FanPosts
In Association With
Big League Stew - MLB - Yahoo! Sports
- Shane Victorino lands role on ‘Hawaii Five-0′
- Have you criticized Hanley Ramirez on Twitter? Your name may be on his bat!
- U.S. senator cries foul over Nationals plans to take back park from Phillies fans
- Photo: Comerica Park readies itself for Hockeytown
- Podcast: Dirk Hayhurst talks about Italy, the reality of the big leagues and his new book
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
