Probability vs. Certainty
Jeff's note: this is required reading. Go over it as many times as is necessary to understand that everything - everything - we do deals with probability, and that nothing is black and white.
Picture the following:
Two friends are arguing over the likely outcome of a set of ten coin tosses.One declares - quite sensibly, since these coins are known to be fair - that he expects said coins to be distributed evenly.
The other, more radical in thought, feels that they'll come up all heads but one.
The coins are flipped. Nine heads, one tails.
'Ah', says the second friend, quite happy, 'You were wrong. That's why we flip the coins!'
Pretty silly, right? Everyone knows that, cheating aside, coins have a 50-50 split between heads and tails. This obviously isn't to say that every time that 10 coins are flipped, the result is even. If we flipped 10 coins 10,000 times, we would instead see a distribution that looked something very similar to this curve (except it would be jagged):

This is known as a probability density function, and understanding what these are is vital when assessing the strength of a prediction. A PDF essentially gives the expected probability for a whole random of outcomes. In this case, it's a bell curve, but they can of course look much more complicated, and even bell curves can look very different because of what's called variance (essentially the spread of the data). Obviously, friend #1 was wise to predict 5/5, even though if you look closely a true 50/50 split will only occur 24.6% (252/1024) of the time. 9/1? 1% (10/1024). Using PDFs, you can say things like 'the number of heads is 62% likely to be 5 or less', which would be completely accurate even if it didn't turn out that way the next time you ran the experiment.
'What does this have to do with anything?', you ask yourselves. 'I hate numbers!' a heckler in the back calls out. 'Please won't someone love me?' a strange young boy cries.
Well, the thing is, every half-competant baseball analyst is in the business of thinking in terms of these PDFs. No, things are never going to be as simple as a coin toss - these are athletes playing a sport, not random numbers dancing around a spreadsheet - but that's not a prohibitive barrier with all the research that goes on these days. PECOTA? It's not giving a number, it's giving a curve. That's where things like 10%, 50%, and 75% levels come from. You've got a team's Pythag predicted accurately? Great, then you can generate a PDF and say that they have something like a 70% chance to be within +/- 4 games of that.
We don't deal with certainties when we look at this game. Sometimes, it comes across that way to people who've never seen stuff like this before and then have PECOTA dropped on them, but it's really not true at all - 'I don't think the Mariners have a high chance of making the playoffs' is NOT the same as 'We're not going to make the playoffs'. And it's not just analysts who do this - everyone who thinks about the future does, albeit subconsciously. What do you think a scout is doing when he's evaluating prospects his team might want to pick up in the draft?
Anyway, here's the crux of the matter.
We work in probabalistic terms. This means that when you tease a single number out of us, it's going to be our best guess and will probably be wrong. This does not mean that the prediction curve itself is complete bollocks (although sometimes this is in fact the case).
'That's why we play the games' is not an acceptable response to an argument about probability any more than 'that's why we flip the coins' is. An argument against a prediction must be conducted against that prediction's assumptions, rather than with a 'Well we'll just wait and see what actually happens then' because that's just not how probability works. Challenge the mechanism behind the prediction, not the expected outcome. I'm not saying the situation is as black and white as my hypothetical argument between friends, but invoking that statement means that 2 doesn't really understand that 1 was NOT stating with 100% certainty that 5 coins would come up heads.
He was, after all, perfectly correct even though he was wrong. The illusion of certainty is a ghost that many people would do well to stop chasing.
255 comments
|
1 recs |
Do you like this story?
Comments
You are a very smart and eloquent man.
Maybe Baker will link to this. I doubt it, but maybe.
Ah.
by Graham MacAree on Feb 9, 2008 2:23 PM PST up reply actions
Why you gotta be like that
I'm just mean
by Graham MacAree on Feb 9, 2008 2:51 PM PST up reply actions
You'd think this would be common sense.
by John Morgan @ Lookout Landing on Feb 9, 2008 2:30 PM PST reply actions
Totally agree there
I see where they're coming from, but I also definitely see why the average fan would tell them to go fuck themselves and enjoy some baseball.
It's the average fan's fault
And most of us rather do enjoy watching baseball.
by Jeff Sullivan on Feb 9, 2008 2:39 PM PST up reply actions
There's "fault" on both sides
Like I said below, posts like this are great because they attempt to educate with the data - which is a whole different process from presenting data and saying "this is right, and if you don't believe it you're wrong". Which is a factual statement, from a data perspective, but it's a bit off-putting from the "average fan" perspective.
by pdb on Feb 9, 2008 2:43 PM PST up reply actions
That's true.
Where I have a problem is when people come to places like this or USSM and blast us for using too many numbers. That's like waltzing into a Rufus Wainwright show and complaining that he sings too much.
by Jeff Sullivan on Feb 9, 2008 2:55 PM PST up reply actions
In the academic community...
For what it's worth, I happen to like economists and stat-heads.
Hey,
by Liebkartoffel on Feb 9, 2008 3:30 PM PST up reply actions
I don't even know what that means
My army of bio-robots will be knocking on your door shortly.
by Graham MacAree on Feb 9, 2008 3:31 PM PST up reply actions
Really?
Qualitative analysis, bitches--helping mathematically challenged people sound smart for the past two centuries.
by Liebkartoffel on Feb 9, 2008 3:56 PM PST up reply actions
Want to be my best friend?
I'm going to give them anthrax cannons
'So define "it".'
And then I drank some gin.
by Graham MacAree on Feb 9, 2008 3:59 PM PST up reply actions
What a cop out.
"Yeah, but how do you know? Isn't knowledge entirely dependent on perception? I mean, when you really think about it, couldn't you just be a figment of my imagination?"...
At which point you punch them in face and respond with:
"Yeah, well, how'd you perceive that motherfucker?"
by Liebkartoffel on Feb 9, 2008 4:10 PM PST up reply actions
He was Scottish
Never punch a Scotsman in the face.
by Graham MacAree on Feb 9, 2008 4:45 PM PST up reply actions
Ah,
by Liebkartoffel on Feb 9, 2008 5:22 PM PST up reply actions
We philosophers get a bad rap
by Uncle Ted on Feb 10, 2008 11:00 AM PST up reply actions
Social Sciences
by Edgar for Pres on Feb 9, 2008 4:36 PM PST up reply actions
Of course I would
by Graham MacAree on Feb 9, 2008 4:37 PM PST up reply actions
If we were soc majors
by Edgar for Pres on Feb 9, 2008 5:20 PM PST up reply actions
Correction:
...Or did I just prove your point?
by Liebkartoffel on Feb 9, 2008 5:16 PM PST up reply actions
Symbolic Interaction!
(Is also a Sociology major)
by tworsandtwols on Feb 9, 2008 4:43 PM PST up reply actions
I tried to stop being an arrogant bastard once
by Graham MacAree on Feb 9, 2008 2:39 PM PST up reply actions
And the world is a better place for it.
by pdb on Feb 9, 2008 2:43 PM PST up reply actions
The problem with that logic is that you
"Say you a flip a coin and it comes up heads. Ask the person you are working with what the odds are that the next flip will be tails. If they say anything greater than 50% do not do business with them. They will inevitably fail in business based on their inability to understand probability and independent results."
Higher math is not common sense
Which is why posts like this are so valuable - the concept is elusive, but if you sit down and think about it it does make good sense.
And this applies especially to sports fans - for most people, sports is an ESCAPE from things like numbers, probabilities, and bell curves. They craft a sports narrative that fits the heroic model more than it does the mathematical model, and when presented with the math side of things, they refuse to believe it because it doesn't fit their narrative.
Me, on the other hand, I largely choose to ignore the mathematics - but that doesn't mean I don't value them, or want to learn more about them than what I currently know. But, most people, as you say, deal in absolutes, so there's no room for alternative narratives in their world. This doesn't make them retarded, nor is it a weakness; people enjoy things up to the level that makes them happy, and that's fine. More power to them.
Rather than take the intolerant/elitist approach, point your non-stat friends to posts like Graham's, and some of the USSM stuff - don't demonize, educate.
by pdb on Feb 9, 2008 2:39 PM PST up reply actions
I'd definitely bring them to LL
Bringing statistics and probability into the game DOES enhance the experience in the way that rooting for players on your Fantasy Football teams does or throwing down cash at the sports book enhances the viewing of whatever your betting on.
Which reminds me: I need to gamble more often!
I think that first line
I blame the California sun.
I always love getting the line
Whenever that comes up, I just point the other person to the broken IKEA chair.
by Jeff Sullivan on Feb 9, 2008 2:58 PM PST up reply actions
Agreed
Nobody can resist a bowl full of skittles.
The two advanced stats
But then again, I was always more an M&M guy than a Skittles guy anyway.
by pdb on Feb 9, 2008 3:04 PM PST up reply actions
a-frickin-men
by pdb on Feb 9, 2008 3:07 PM PST up reply actions
Both of you had better
I strongly disagree!
I think Win Probability is more fun after a game
And peanut butter M&M's are delicious. (This is NOT a new diary topic.)
by Jeff Sullivan on Feb 9, 2008 3:10 PM PST up reply actions
You should have a
They are not nearly as good as
by Edgar for Pres on Feb 9, 2008 4:39 PM PST up reply actions
Here's the thing though...
by John Morgan @ Lookout Landing on Feb 9, 2008 3:10 PM PST up reply actions
It's uncertainty.
Plus, by holding an opinion, they're introducing a confirmation bias into their future discovery of evidence, so even if they learn the truth later they might not believe it.
This is how religion works. People don't know how the universe functions, so they believe a story that explains it all even though there's no good reason to reach that conclusion.
Statistical probability goes completely over most people's heads because it measures the uncertainty. Since they can't accept that uncertainty ever exists, any attempt to measure it must therefore be fabrication. If the outcome of the Superbowl happened as it did, then there must be an explanation as to why, and a typical person will choose an explanation that fits the outcome regardless of whether its supported by any data.
When Billy Beane said "my shit doesn't work in the playoffs", his point was that the outcome of the playoffs, from his point of view, was unexplained variance. BP's Joe Sheehan has put it better by saying, "In a short series between two playoff calibre teams, any result is reasonably likely." But since both of these contain explicit recognition of uncertainty, the average person doesn't actually understand what's being said.
Given that the average person doesn't understand any line of reasoning where the outcome is uncertain, is it any surprise when they don't understand the point Graham is making here? And isn't it scary that we let them vote?
That's the attitude right there
Just because people may not understand that line of reasoning is no reason to be arrogant. You're not a better person than me just because you grasp that concept faster than I do.
It's not arrogance if I'm right
This is demonstrably true.
Yes, it is
As I said, just because you may be better than me at grasping a particular concept does not make you a better person than me, nor does it allow you to judge my ability like you do when you say "isn't it scary that we let them vote?".
Being a "better person"
As for the voting comment, that's only assholish if you think voting is some sort of fundamental right that should never be withheld from anyone (so my suggestion that it be withheld is abhorrent). But I certainly never espoused that opinion.
Since one of the more important jobs of a government is to act as an economic steward, and most economics relies on just these sorts of probabilistic measurments, allowing people who can't understand the math behind the decisions to influence the decisions is crazy.
That's interesting
So in your world, only an educated elite should have any say in the direction of a representative democracy? Interesting.
Or, put another way: If you don't know how the engineering of a combustion engine works, should you be allowed to operate a car?
Not disagreeing with you.
Extrapolating, he's (probably) saying that economists should run the economy. The equivalent argument to the car you brought up would be "noone but economists should be able to have or use money".
Maybe I misunderstood
allowing people who can't understand the math behind the decisions to influence the decisions is crazy.
I read that as saying if I don't understand higher economic theory/mathematics, I should not be allowed to have a say in how those theories get applied (as in, I should not be able to vote). The car thing was kind of a stretch, but what i was thinking was that it is like driving - when I drive, I need to make decisions about proper speed, acceleration rate (when passing), braking, and the like.
I have no idea how, when I press the gas or brake pedal, the speed increases or decreases; I just know it does. And the fact that I don't know how the speed increases does not make me any less competent of a driver than my brother-in-law, the mechanical engineer, who probably does understand.
Voting's the same way - I don't need to know how to apply economic theory in order to be able to cast a vote. Does it help? Maybe. But should it be a requirement? Never.
Not to step on toes
-Lets say you happen to own stock in Car Company X (CCX). The board of directors recently decided they want to come up with a brand new clean-energy car line, but they're undecided as to which clean-energy to pursue so they're going to put it to a shareholder's vote. You have the right, in this case, to vote on the decision, but should you?
Or, to put it another way, you say:
when I drive, I need to make decisions about proper speed, acceleration rate (when passing), braking, and the like.
But you did go to some sort of school for that. Either driver's ed or possibly you were taught by a parent, whatever. There was training involved. We don't let people who haven't passed a driving test drive, so why do we let people who haven't passed an economics test vote on economic policy?
I think that's more of what he's saying.
Yeah, I think I get it
You could extend that argument even further, and say that if you aren't a trained educator, you shouldn't be able to vote for your school board, or if you haven't studied the law you shouldn't be able to vote for judges; before long, you're back to the "landed gentry" concept where only a small percentage of the educated elite (read: rich white men) are entitled to cast a vote.
I don't think judges should be elected
I agree, actually
Yeah, it's a "slippery slope" problem
Don't mistake my fervent defense
I'm not willing to concede that the people
So on one side you have a small group of knowledgeable people making decisions, while on the other side you have a much larger group making uninformed decisions. I can't honestly say that the democratic option looks superior.
I would say if you are part of the landed gentry
Also, just because many people choose not to educate themselves (not everyone, but American Idol does destroy CSPAN in ratings), doesn't mean the theory of democracy isn't good. You probably know that about 1/2 the population votes and the more educated a person is, the more likely that person will vote. I believe the key to a well functioning democracy is successfully educating and empowering the population.
It's not about better or worse
The problem with the landed gentry was that they had no incentive to make decisions that benefitted anyone but themselves.
This is what I like to call "human nature", and it would b ethe same today as it was then. This is the best argument for allowing everyone to vote that there is. If only one group is allowed to express opinions, of course those opinions will benefit themselves and themselves only; if every group has that right, the effect is diluted to a point where everyone seems willing to live with it.
In an ideal world.
an anonymous person
I know it's pedantic, but I find it humorous how interchangeably the term democracy is used for our republic and how we're a lot closer to an oligarchy than most people realize.
You vote in a primary, to elect delegates to the convention, who, depending on the state, may not even be required to vote per the primary results, which in turn selects the candidate, who goes into the national election (only if it's one of the two main parties) where the public votes again, but for electors who then actually vote to elect the president, and again, they may not be bound to vote as the public in their state did.
Toss in voter fraud and the screwed up system where we only have two parties, nomination within those parties are partly (I'd argue mostly) controlled by the media (see: debates) and lobbyists/PACs and really, are we anything close to a democracy?
We're pretty damn close
As for elections being a national holiday, it's a great idea - I'd go a step further, and do what Australia does. Australia requires their citizens to vote, or it's a $100 fine. They can vote for whoever they want, but they are compelled by law to do so.
god no
On what basis?
If you vote regularly anyway, what difference would it make if there were a law making it compulsory? And if you don't vote, you would now have an economic incentive to do so.
Because to me,
Forcing people to vote is the same, in my mind, as stripping people of the right to vote.
Interesting point.
Then what would happen if
Martial law
For example, in the US presidential race, if you live in a winner-take-all state that always goes Democrat, it simply does not matter for which candidate you vote. If you vote Democrat, the Democrat wins all the electoral college votes. If you vote Republican, the Democrat wins all the electoral college votes. No change; your vote made no material difference. So why did you bother?
I wonder if having
Bush: 20%
Kerry: 20%
Neither: 60%
I have argued in the past
Whenever I'm in Seattle I'm amazed at how many people spin their tires in the rain. This can only be due to a failure to understand the difference between static and kinetic friction.
I would argue that your lack of awareness (I was going to say "ignorance", but you'd probably take that as some sort of attack) of how your car works does make you a less good driver. Someone who knows how the vehicle works can improvise in unexpected situations. If your car has traction control, for example, knowing how it works can help you a lot when your wheels are experiencing disparate traction levels.
There's levels of knowledge, though.
I know all this without having ever taken a physics course. Even in high school, which was so long ago for me as to be irrelevant.
If you spend any length of time driving, and are at least marginally aware of cause and effect, you can figure out how well your car works, and figure out where the limits of your car's performance are, from the experience of driving it repeatedly and frequently. And you don't need a working knowledge of physics in order to figure it out - you just need to be aware of what you're doing.
Whenever I'm in Seattle I'm amazed at how many people spin their tires in the rain. This can only be due to a failure to understand the difference between static and kinetic friction.
"Only" is a very restrictive word. It could also be due to the person behind you being impatient and tailgating you, thus making you feel pressured into going faster than conditions dictate, and causing your wheels to spin; or it could be that you were on your phone and didn't see the light turn green, so you overcompensate (you shouldn't call while you're driving, but that's a different story). Driving also requires reacting to other people.
There is a huge difference between
I always had trouble with physics, I don't know how to change the oil, and I certainly can't explain how the all-wheel-drive system of a Subaru works. For nearly 4 years I lived miles down a dirt road in Vermont that was muddy, icy, snowy, sometimes all three, and I never got my vehicle stuck in any bad weather.
To relate this to baseball, how many major league pitchers can explain the physics of a curveball? I don't know how smart Felix is, but do you think a class in physics would make him a better pitcher?
I think a class in physics
Biomechanics would be even better.
Agreed, but that is a different argument.
Plato!
by Liebkartoffel on Feb 10, 2008 5:24 PM PST up reply actions
Heh,
by Liebkartoffel on Feb 11, 2008 10:32 AM PST up reply actions
Very well written.
In baseball, nothing is done the same way every time. Pitchers, hitters, and fielders make adjustments for every situation (this happens as frequently as every pitch). This means the experiment is run differently every time and the results cannot be accurately distributed on a bell curve.
So, in other words, the statistics in baseball are too random to make any proper statistical analysis. This is why neither side of the issue is going to be correct in their predictions.
Therefore, this is why the games are played. The SABR community does make a good effort, though. I trust their numbers more than just some random predication from some random dude.
Yeah, we're not going to come close to perfect
by Graham MacAree on Feb 9, 2008 2:49 PM PST up reply actions
The SABR community
And according to my statistics, the SABR community has done a 43% better job than conventional wisdom (+/- 8.5% margin of error).
Agreed
But Baker's got a point. At the end of they day, they're just predictions. Adopting an "I'm clearly right and you're wrong...so fuck off" position is extremely hubristic.
Meh
Me saying to the average fan, 'Your argument doesn't hold water because of this, this, this, and this' (assuming good research and all that) really isn't.
What tends to go wrong is that people very familiar with all the work done in the blogosphere sometimes forget that not everyone has already read everything about a certain subject, so they'll just bring out a conclusion without siting the mass of analysis behind it.
I can definitely see how it could be interpreted as someone thinking they're superior and pulling numbers out of thin air, but that's never the intent.
by Graham MacAree on Feb 9, 2008 3:10 PM PST up reply actions
Don't disagree with that at all
Yeah, probably
What gets me is when people take carefully crafted arguments and throw them out of the window with 'pfft. Stat nerd. INTANGIBLES!'
And then I flame them.
by Graham MacAree on Feb 9, 2008 3:22 PM PST up reply actions
Perhaps this is a better explanation
Wait a minute...
by MarinerintheDistrict on Feb 9, 2008 2:53 PM PST reply actions
When you start giving me
by Digger on Feb 9, 2008 2:54 PM PST reply actions
PECOTA gives you the whole damn curve
by Graham MacAree on Feb 9, 2008 2:59 PM PST up reply actions
Also, the latest USSM predictions have STDDEV
by Graham MacAree on Feb 9, 2008 3:00 PM PST up reply actions
Oakland?
Something's broke.
by Jeff Sullivan on Feb 9, 2008 3:02 PM PST up reply actions
Yeah. My guess here
by Graham MacAree on Feb 9, 2008 3:03 PM PST up reply actions
Here's the other point
Just because teams come "out of nowhere" or "crap the bed" doesn't prove there's a flaw in the system, just that there are way too many variables in play for anything approximating a totally accurate prediction.
By the way, at least in a Bedard trade context
The fact that a lot us consider the M's the second-best team in the division isn't going to take anything away from the thrill of watching the season play out. I've enjoyed following longer shots. If you're incapable of separating numbers from your emotions, then you aren't trying hard enough.
Well, clearly
Posted by Druv
6:06 PM, Feb 08, 2008
This will likely go down as the worst trade in Mariner history.
I start to wonder what a lot of these people are drinking, because it deserves its own blog.
Those people are insane
by Jeff Sullivan on Feb 9, 2008 3:13 PM PST up reply actions
This made me laugh:

Totally a middle finger.
DMZ might be mad to know
Now i can refresh this page thousands of times and put pressure on his server
Maybe I just need to stop going to USSM
I am starting to understand why the Mariners are profitable each year. The vast majority of their fans don't understand baseball and are just happy to see the pretty uniforms and the Moose.
msb Says:
[Oh my god. Well], here is an example of your casual Mariner fan:
"Adam Jones has a primadonna attitude who doesn't run the ball out, and doesn't seem to care. They need to platoon Mike Morse, he's a hustler who'll get to the ball."
I guess my point is...who cares? Aren't they a fan of the same team?
I love the show Lost because I read all the theories, check the screen cap blogs, etc. Someone else loves the show just because.
Am I more badass than that person just because I know more about Dharma and can predict the show that much more accurately? All the other dude wants to do is just let it play out and enjoy the ride. Where's the hostility coming from?
i agree that they have just as much right as i do
Dude.
by MarinerintheDistrict on Feb 9, 2008 3:43 PM PST up reply actions
Look man
But at this point, it just seems that you're repeatedly going after them under the premise of... I don't know, what?
I realise that as someone who volunteers over there, I'm biased and all that, but I just don't understand the hostility on your end, even while I acknowledge that there have been some really stupid things said over there recently.
by Graham MacAree on Feb 9, 2008 3:30 PM PST up reply actions
It comes from both sides
by Jeff Sullivan on Feb 9, 2008 3:30 PM PST up reply actions
Don't get me started on the evils of Prius
by Graham MacAree on Feb 9, 2008 3:41 PM PST up reply actions
Dead meme?
by Jeff Sullivan on Feb 9, 2008 3:42 PM PST up reply actions
My favorite thing to do

Can't deny that
Thank God for this place.
Glad someone else mentioned this.
It is all getting kind of silly between them, especially when you consider how high they both were on each other last year. Ever since Geoff has started to get into low-level numbers, there seems to have been tension between the two.
by EnglishMariner on Feb 9, 2008 3:54 PM PST up reply actions
As Jeff mentions in this post
At Baker's Blog it's all white, and at USSM it's all black, unfortunately.
Combine that with the [snotty comment about your spelling]s, or the STFU AND ENJOY THE GAME STAT GEEKS, it makes you ache for a beer diary.
I'm the one who does the []
by Graham MacAree on Feb 9, 2008 4:02 PM PST up reply actions
I stopped reading USSM comments back in
Here's the thing, though...
As an example, my main focus in college (before I dropped out to get the hell away from Olympia, so take this with a grain of salt if you so choose,) was urban studies/urban planning. It's something I have studied pretty intensively both in an academic setting and in my own free time. Now, I wouldn't consider myself an expert by any means, but I know my shit pretty well. And I am of the opinion that increasing traffic capacity in order to ease traffic congestion is the wrong way to go about things. I believe this because I have a lot of research and scientific study to back my position. There are people who disagree with me. Many of them are equally well versed in these things, and while I disagree with their position, I am willing to listen with an open mind if they give decent counter arguments and argue their position intelligently. I am not likely to take seriously the position of anyone who's assertion is that adding freeway lanes will ease traffic because, duh, it's just common sense.
My (rather long winded, yikes) point is that it's totally acceptable to dismiss the opinions of people who don't know nearly as much about something as you, especially when those people are using terrible grammar, insulting the intelligence of others, and refusing to acknowledge widely accepted knowledge with little or no evidence to back their positions up. And with as much crap as DMZ, Dave and the other, more well versed posters at USSM (and other places) it shouldn't come as a surprise that they're getting frustrated and testy. I would be, too.
by Aaron Campeau on Feb 9, 2008 4:19 PM PST up reply actions
No, it's not
Wouldn't it be better to educate them as to your point of view, and attempt to convert them, rather than dismissing them?
by pdb on Feb 9, 2008 4:22 PM PST up reply actions
To a point
If you have one person who doesn't understand a point, it's better to teach than to ignore. But if you have an ungodly number of those people, and if for every one you teach there are five more who don't know anything, it becomes really hard to stay motivated.
by Jeff Sullivan on Feb 9, 2008 4:52 PM PST up reply actions
People must want to learn
There's also that
by Jeff Sullivan on Feb 9, 2008 5:05 PM PST up reply actions
Absolutely
Okay, poor choice of words on my part.
by Aaron Campeau on Feb 9, 2008 5:09 PM PST up reply actions
Re
It's not surprising it irks some of the readership. If you want to put that in the language of statistics, they're committing a gross individualist fallacy by assuming that every commenter is a borderline illiterate moron on the basis of their experience with a few isolated users.
by anunderwaterguy on Feb 9, 2008 3:47 PM PST reply actions
Clearly you read Baker's blog.
USSM is fascinating and all
What Jeff Really Said:
Dead meme yet? heh
by JoeyJoJoJuniorShabadoo on Feb 9, 2008 4:05 PM PST up reply actions
The meme police, they live inside of my head
by pdb on Feb 9, 2008 4:17 PM PST up reply actions
Geoff Baker's blog attacked USSM
I agree.
I honestly do believe that some people develop an inherent hatred for mathematical analysis at a young age. As soon as you ask them to swim in those waters they shut down on you. It is frustrating, but it is how it goes.
When I was a kid
Now maths is my weakest subject by miles. Strange.
by Graham MacAree on Feb 9, 2008 4:11 PM PST up reply actions
I specifically chose work
It is possibly the only gift I have so I like to keep it sharp.
I was always the irritating little snot
"So 2x5 is 10."
"Why?"
"It just is."
"Why?"
"Because that's how multiplication works."
"Why?"
"Listen, 2x5 is 10. Just trust me on this."
"You mean, there's only one answer? How boring"--at which point I'm mentally throttled by everybody else in the room.
by Liebkartoffel on Feb 9, 2008 4:23 PM PST up reply actions
I have a friend
by pdb on Feb 9, 2008 4:25 PM PST up reply actions
Theoretical mathematicians are terrifying
by Graham MacAree on Feb 9, 2008 4:28 PM PST up reply actions
I find it fascinating, in a way
by pdb on Feb 9, 2008 4:29 PM PST up reply actions
Because it's super badass!!
by John Morgan @ Lookout Landing on Feb 9, 2008 4:59 PM PST up reply actions
My brother's a
by Liebkartoffel on Feb 9, 2008 4:30 PM PST up reply actions
I once went over the proof
by Bearskin Rugburn on Feb 10, 2008 1:37 PM PST up reply actions
It's not a hatred
it's a lack of understanding, combined with the occasional presentation tone of "you're not smart enough to understand this so I'll patronize you now" that gets me.
I understand that I'm not as smart in the ways of mathematics as a lot of people, and I'm entirely OK with that; I have other areas of expertise. What I resent is when I'm made to feel like that lack of knowledge of statistics somehow implies that I'm not a fan in the "right" way.
I'd rather have someone show me how to swim in those waters than have someone ask me to swim in them; it's a small distinction, but a critical one.
by pdb on Feb 9, 2008 4:14 PM PST up reply actions
Disagreeing with someone
It might make you wrong, but that's neither here nor there on the fanship scale.
by Graham MacAree on Feb 9, 2008 4:17 PM PST up reply actions
And that's the disconnect
And when I try to get people to look past that, and see the data, and show them posts like this, it works; then those people start to read the inane Baker/USSM blog war, and they're right back to square one.
by pdb on Feb 9, 2008 4:20 PM PST up reply actions
Agreed
February 10th, 2008 at 9:22 pm
10Dave Says:
Do you even realize that the inputs that the simulations Derek is doing aren't based on Diamond Mind's projections? Any problems you have with its engine have nothing to do with this. If you disagree with the expected performances, you're taking objection to the ZiPS projections, not DMB. And you haven't given us any reason to believe that your opinion is more credible than ZiPS.
And that's just one of your many errors (others - assuming `07 run distribution is true talent level, ignoring Guillen/Wilkerson dropoff, ignoring remarkable health of position players last year, not regressing expected performances from Johjima, Ichiro, Burke, and ignoring the weakness of the bench). Not bad for two paragraphs.
I get the argument that they get a lot of crap, so they're defensive. I just don't think that's it. I think it's just that they enjoy being pricks.
I think they're just out of patience.
by Jeff Sullivan on Feb 11, 2008 2:03 AM PST up reply actions
Great post
What's worth reiterating here (I know that it's hinted at but not specifically mentioned) is that this is just as relevent in micro situations, where a decision is made to do something on a play. That attempt might not come off, but if the probability of it coming out good was high enough then it was still the right decision
(It's late, and I'm struggling to come up with a good example - can you tell?).
Meanwhile, who'd have thought that, ten years ago when I studied Stats at school, the initials "pdf" would later be superceded in general vocab quite so strongly... for shame
Sponsor of Jamie Burke's baseball-reference page
Indeed
by anunderwaterguy on Feb 9, 2008 4:24 PM PST up reply actions
I like Bacon
I'm more of a Shakespeare guy.
by pdb on Feb 9, 2008 4:24 PM PST up reply actions
That's why we play the GAMEZ, fellas!!
We still have yet to see Bedard reach +200 innings. I think he can, but it's not a lock. The negative projections are largely due to the fact that there are more things working against this team than for it.
Still, good or bad, the Bedard trade is such an important trade, one that I still lean in favor of. I believe it is still possible for this FO to construct a strong team in a short amount of time (yea, talent evalutation is an issue) while signing Bedard to a reasonable or even overblown extension since this team will be perceived as competitive.
But since we "play the games", how bout giving us a +48 homerun season, King Awesome. :)
I think one of the biggest steps
They don't even predict wins
by Graham MacAree on Feb 9, 2008 4:48 PM PST up reply actions
Yeah but the run estimates they come out
by Edgar for Pres on Feb 9, 2008 4:53 PM PST up reply actions
Well
by Edgar for Pres on Feb 9, 2008 4:53 PM PST up reply actions
I think it's actually luck -after- pythag
by Graham MacAree on Feb 9, 2008 4:54 PM PST up reply actions
I think luck is included in
by Edgar for Pres on Feb 9, 2008 5:00 PM PST up reply actions
What I meant is
by Graham MacAree on Feb 9, 2008 5:01 PM PST up reply actions
Pythag
Put another way, you can come up with an uncertainty value for a number of teams in the same analysis "x% of teams will fall within a particular range of their pythagorean win-loss record", but making explicit statements of uncertainty about a specific team could be problematic due to the high potential for outliers related to roster construction.
by anunderwaterguy on Feb 9, 2008 5:07 PM PST up reply actions
Its totally reasonable to think
by Edgar for Pres on Feb 9, 2008 5:11 PM PST up reply actions
I also think he's assuming
by Edgar for Pres on Feb 9, 2008 5:13 PM PST up reply actions
Example
Pythagorean wins are quite different. If you accept that win-loss records cannot be entirely reduced to the two variables of runs scored and runs allowed (and only the most dogmatic of pythag supporters would argue against this) then the properties of a particular team - whether these are intentional or otherwise - have the potential to alter the accuracy of your measurements. Consequently, I'd imagine it's a little more difficult to come up with an accurate uncertainty figure for pythagorean win projections, than it is with other statistics.
by anunderwaterguy on Feb 9, 2008 6:07 PM PST up reply actions
I'm not a pythag supporter by any means
by Graham MacAree on Feb 9, 2008 6:09 PM PST up reply actions
You have anything better
by Edgar for Pres on Feb 9, 2008 6:31 PM PST up reply actions
Uncertainty from equations is a little
by Edgar for Pres on Feb 9, 2008 6:30 PM PST up reply actions
I'm not anti-pythag
Looking at the standard deviations would allow you to make statements about the general accuracy of pythagorean projections, but at the individual level there's at least the potential for it to break down if a team's specific properties isolate it from more general trends. Anyway, this is a bit of a pedantic point because as you say, the average pythag win% comes out at .500, but it's worth raising nonetheless.
by anunderwaterguy on Feb 9, 2008 6:57 PM PST up reply actions
I know what you mean
by Edgar for Pres on Feb 9, 2008 9:10 PM PST up reply actions
So you're saying if I give you
by Edgar for Pres on Feb 9, 2008 5:09 PM PST up reply actions
Site comment
(Bookmarking is tedious and because I access the web from a variety of locations it isn't that useful.)
Back on topic...
Metaphorically, I'm trying to outline the impact of a freaking ridiculous bullpen on these projections. I'd like to see how data comparisions after this season line up with data comparisons on the 2007 season... how dominant was the bullpen in each year, and how much did we beat Pythag in each year?
I guess I don't look at 2007 as the 90th percentile on its curve. There are a lot of reasons to distrust that - Sexson died, Lopez died, the defense died, and the rotation died. You're telling me only 10% of possible scenarios were better than that (generic you, btw - not anyone in particular)? I think that Pythag is a great tool, but very approximate... and the approximation was particularly bad on the 2007 M's. I'm not a baseball stats expert, so consider this all hot air if you want... but I'd imagine the 2007 M's being somewhere between 45%-65% on whatever the 'accurate' bell curve should have been.
Any constructive thoughts on the theory appreciated.
I'm not an expert by any means.
Just for fun, I went through and looked at the M's Pythag vs. actual W/L record for every year of the franchise's history. The two numbers were exactly the same three times, +-1 nine times, +-2 four times, +-3 four times, +-4 three times, +-5 twice, +-6 twice, +-7 three times, and +-9 once. So the Pythag W/L record was within 4 wins or losses 74% of the time. 74% isn't perfect, but it's close enough that I think it's incorrect to call the system busted based on the biggest gap between actual and Pythag W/L records over 31 seasons.
by Aaron Campeau on Feb 9, 2008 8:04 PM PST up reply actions
I guess...
I wonder what would happen if we took a look at teams with bullpen profiles similar to the 2007 Mariners (300 ERA+ closer and/or whatever the bullpen's ERA+ as a whole was), and charted out their Pythag differences? Is there a trend? Unfortunately, I don't have the time or researching expertise to figure that out, but I'd be curious to see the results if anyone made that comparison.
I don't mind being proven wrong - I guess I just see results that don't make sense intuitively, and want to know why.
That's a fair question.
I'm too inexperienced/lazy to do the research myself, but if someone ever tackled it, I think it would be fun to see the results. Of course, for all I know, this has been covered, and the longer-tenured staheads all think I'm an idiot.
by Aaron Campeau on Feb 9, 2008 8:29 PM PST up reply actions
Okay, this took some work, but here you go
Chart 1: Correlation between difference from Pythagorean record and bullpen strength (measured by total bullpen WPA). There is a relationship, here - stronger bullpens generally lead to outperforming the Pythagorean record a little bit. However, while the correlation is statistically significant, it isn't all that strong. It also doesn't explain the Mariners' huge gap (7.38 bullpen WPA, 9 game difference); by plugging 7.38 in for x, you get a y between 2-3, not 9. So the M's were both good and lucky.
Chart 2: Correlation between bullpen strength in one year and bullpen strength the next. Weak relationship. In other words, even though the Mariners had a great clutch bullpen in 2007, we can't really say much about how they'll do in 2008. Even if the group performance doesn't change that much, the timing of said performance might, which could make a world of difference.
Chart 3: Correlation between Pythagorean difference in one year and Pythagorean difference the next. No clear relationship at all. Beating your Pythagorean record doesn't appear sustainable.
by Jeff Sullivan on Feb 9, 2008 9:17 PM PST up reply actions
Thank you sir,
I still find it incredibly odd that the M's had so many bad performances, and still beat Pythag by so much. Luck, plus a good bullpen, plus two earned run magnets (Weaver + Ramirez), plus something else?
At this point, I would expect some bullpen regression and luck regression, slight-to-modest improvement from Sexson/Lopez, and a large improvement in the rotation to put this team... well, right about 90 wins. This year, however, I think that Pythag catches up to reality. That more than anything else seems like the flukey part of this equation... while I don't trust pythag completely, it is generally a very useful tool.
Jeff, thanks very, very much for the insight. I can't say I'm surprised to see that bullpen excellence can't explain the whole thing away... but neither am I surprised that it has some impact. Cool stuff.
I think, more than the bullpen last year,
by Jeff Sullivan on Feb 9, 2008 9:30 PM PST up reply actions
I agree with that
by Edgar for Pres on Feb 9, 2008 9:42 PM PST up reply actions
Thats very interesting
by Edgar for Pres on Feb 9, 2008 9:26 PM PST up reply actions
I think basically
by Edgar for Pres on Feb 9, 2008 9:39 PM PST up reply actions
You are a good man.
by Aaron Campeau on Feb 9, 2008 10:13 PM PST up reply actions
Help me
10 in a row: 0.5 ^ 10
by Jeff Sullivan on Feb 9, 2008 9:03 PM PST up reply actions
Still a bit dense here.
Is it just a problem of sample size?
Nope
The 100th flip
I know this kind of thing can be difficult to grasp at first. It's why I highly recommend that everybody take an intro stats course, or at least read a probability book or two. This kind of thing is everywhere in life, and if you're able to understand it, you'll get a leg up on everyone else who doesn't.
by Jeff Sullivan on Feb 9, 2008 9:23 PM PST up reply actions
But if nobody understands this
by pdb on Feb 9, 2008 9:25 PM PST up reply actions
A slightly related question
by Mariner John on Feb 10, 2008 12:39 AM PST up reply actions
I don't think it'd be completely independent
by Jeff Sullivan on Feb 10, 2008 1:07 AM PST up reply actions
Here's where I question this...
Certainly all teams have this "noise" and it'd be really hard to analyze the data to find anything statistically significant (because of the sample size issues that would likely be involved). But I'm very comfortable accepting that baseball games are clearly not independent of each other, and it's probably worth trying to analyze this further, even if the results aren't "statistically significant" due to sample sizes...
by PositivePaul on Feb 10, 2008 10:34 AM PST up reply actions
Yeah they aren't independent of each other
Almost no outcomes are independent but many (like baseball games) appear pretty close. We also like to assume that each PA is an independent event even though they aren't really. We do this because if you want to apply almost any statistics, the first assumption you usually make is that the events are independent of the previous/future events. In baseball this is much more true than other sports which is why you have seen so much more sucesss with stats.
by Edgar for Pres on Feb 10, 2008 10:45 AM PST up reply actions
Independent
The coin flip example is used to relate how each game is independent from the next. But in the long run, you have a 50% possibility with each flip. Baseball is a bit more complicated because there are so many variables that go into the game. Predicting wins is difficult and you really do not know how good a team is until after the season finishes.
The Mariners were a .543 team in 2007. Theoretically, this means the Mariners had a .543 chance of winning every game. But when the Mariners were playing that 10th game of a 9-game losing streak, there was no way to really know what kind of odds the Mariners had at winning. Factor in the numerous random variables that go into each game, I do not think you can place any odds on an independent game even knowing the overall winning percentage at the end of the season.
I think I have come to realize that the strength of SABR statistics isn't about calculating the probability of a team winning, it is about finding underappreciated strengths and skills of individual players that will help improve a team's unknown probability of winning. And even then, after collecting every player's statistics and running each team's rosters against each other, I do not think you can determine how well a team will perform because you are using random statistics to find a random result.
Forgive me for thinking out loud. I don't know if I even answered your question.
I wasn't asking from Sabermetrics perspective
Your response was helpful, however.
by Mariner John on Feb 10, 2008 1:42 AM PST up reply actions
I think if your coin
by tworsandtwols on Feb 10, 2008 2:15 AM PST up reply actions
Ok, so
Once you have flipped those 99 coins, the probability that the next one will be heads is 0.5.
The odds of flipping a fair coin 100 times and getting all heads is (0.5^100) or x/2.
Here's the reconciliation:
Consider the coin flipping exercise as navigating through a tree. I'll show you what I mean for the purposes of just a few flips.

That's just the tree for two flips. As you can see, you simply progress along each branch, taking 50/50 (or 0.5, as I've labelled it) chances at each node until you reach the end. Then, to get the probability of a whole chain, you just sum up the probability of all the branches it takes to get to that point (for example, since the HT/TH outcome can be reached two different ways, we just trace one branch back to the origin (0.5x0.5) and then add it to the other (0.5x0.5).
That's a pretty simple example, but you can expand on it very easily by just adding more flips or changing the probabilities assigned to each branch.
Fastforward to a 100-flip tree. You have just flipped the coin 99 times and gotten heads each time. The probability of this chain of events happenening (going back to the start) is x. The probability of the next coin being heads or tails has nothing to do with the chain - the branch probability still stays the same. However, if you do flip that coin, and it comes up heads, the probability of that chain happening was x/2.
Another way to think about it:
Someone tells you 'I have just flipped 99 coins in a row. They all turned up heads. What is the chance that I will get to 100 heads in a row?'
The answer is pretty clearly 50%.
Someone asks 'What is the probability of flipping 100 coins, and them all turning up heads?'
The answer then is x/2.
The difference between question 1 and question 2 is that 1 just has more information in it. We start very close to the top of the tree and only have to work our way through one branch, rather than all of them. There are very limited outcomes at that point (you're either going to have 99 heads or 100 heads), so the probability of each outcome has changed.
Does that help at all?
by Graham MacAree on Feb 10, 2008 3:37 AM PST up reply actions
It does help
I appreciate the trouble you all took.
You can also just work it out
by Edgar for Pres on Feb 9, 2008 9:17 PM PST up reply actions
Hmm, damn you HTML formatting
for 1 toss:
possible outcomes = H or T
chance of 1 heads = 0.5
for 2 tosses:
possible outcomes = H/H or H/T or T/H or T/T
chance of 1 heads = 0.5
chance of 2 heads = 0.25
for 3 tosses:
possible outcomes = H/H/H or H/H/T or H/T/H or H/T/T or T/T/T or T/T/H or T/H/T or T/H/H
chance of 1 heads = 2/8 = 0.25
chance of 2 heads = 3/8 = 0.375
chance of 3 heads = 1/8 = 0.125
by Edgar for Pres on Feb 9, 2008 9:29 PM PST up reply actions
Thanks
"Dice have no memory"
The coin doesn't know it's in the middle of doing something highly improbable, so the next outcome is unaffected by those circumstances.
Potentially Costly
A guy walks up to a roulette wheel and notices that red has come up 7 times in a row, so he plops down a large stack of chips on black. The wheel spins, and a red number comes up.
The guy stomps off, bemoaning his bad luck. After all, what are the odds of 8 reds coming up in a row?
Of course, pointing out to him that the odds were just under 50/50 once 7 reds had hit, might not be safest course of action at that point in time.
All I have to say is that
Even statistician aren't that smart
Using ERA and WHIP to try to prove points. Come on, you can do better than that. You're stats professors. If you look at ERA+ vs Age you'll see an R^2 = 0.0011.
To be fair...
But anyways, I don't think that the professors really showed anything. They did bring up the good point that the Clemens report basically cherry-picked statistics to prove its point, but I kind of feel like the analysis that they did suffers from the same problem.
The real moral of the story is how if you use selection bias, you can prove almost anything. Anytime you see a statistical report, it's important to see what assumptions were made in how the data was collected and analyzed to see if any selection bias is taking place. If there is a lot of selection bias that is not accounted for, then the data was most likely used to prove a point, rather than analyzed to determine what is actually happening.
Yeah I totally agree
by Edgar for Pres on Feb 10, 2008 10:45 PM PST up reply actions
There is more; lots more.
And I can 100% guarantee you
There you go
I just stumbled upon the paper you did with Jensen regarding throwing ability. It looks interesting. I'll try to give it a read at some point today.
I'm much more of a lurker here,
Alright Graham:
Go.
I noticed and replied
by Graham MacAree on Feb 11, 2008 11:32 AM PST up reply actions
<sigh>
Because it's not true Geoff. That's why.
Truth is overrated.
I thought about pointing that out
by Graham MacAree on Feb 11, 2008 11:42 AM PST up reply actions
Slowly Geoff is wearing on me
Of course coin flips are simple. However, if you do not get the coin flip idea then you do not have a foundation of knowledge on which to build. That foundation is necessary to analyze more complex data.
I know
At least he picked up on my point that you should argue the assumptions and machinery behind the projections rather than the output, even though I don't think he noticed that that was one of my more important points.
Ah well. At least it got read.
by Graham MacAree on Feb 11, 2008 12:07 PM PST up reply actions
It's not even just entry-level Stats
Well...
- Build readership
- ?????
- Profit!!!!!
by PositivePaul on Feb 11, 2008 12:38 PM PST up reply actions
By no means do I think he is bad.
That is a little true.
It's already in the dead meme thread
that thread is way too long
I'll filter through it
either that
Well slap me with fishes, then...
by PositivePaul on Feb 11, 2008 4:22 PM PST up reply actions
Apparently I wrote this post
by Jeff Sullivan on Feb 11, 2008 3:09 PM PST up reply actions
I love that people are actually looking for fights
I still like Geoff Baker. Sorry dudes.
by Jeff Sullivan on Feb 11, 2008 3:23 PM PST up reply actions
Hey, he replied to the post
by Graham MacAree on Feb 11, 2008 3:24 PM PST up reply actions
makes sense
/*joe morgan
And with that comment
Watch your attitude.














