PECOTA - not a big Mariners fan

by Teej on Feb 15, 2008 11:39 PM PST

Nate Silver just posted the 2008 projected standings, and it's not pretty. (Subscription required, I believe.) The M's are projected to finish in last place, going 73-89. The culprit is no surprise: The M's are projected to score the fewest runs in baseball, except for -- of course -- the Giants. And the team as a whole is seen slugging under .400 -- again, better than only the Giants.

Here's a link to the Unfiltered post.

Also: I shall fill out my post by pointing out that this is my first diary, and I'm very, very proud of it. Basically, it's the first time I've gotten HTML links to work.

0 recs | Comment 66 comments

Story-email Email Printer Print

Comments

looks like

you don't have a spell-check subsribtion.

Caffeinated Confines

by StevenH on Feb 16, 2008 12:02 AM PST 0 recs

Looks like you don't have shift keys

But nonetheless, it's fixed. Thanks.

by Teej on Feb 16, 2008 12:39 AM PST up 0 recs

Or a C key

by Hit By Pitch Brewing Company on Feb 16, 2008 4:58 AM PST up 0 recs

way to go dude

nothing like a pain-in-the-ass spelling comment with a misspelling in it.

by Mere Tantalisers on Feb 16, 2008 12:48 PM PST up 0 recs

What?

They don't like our cagey War Veterans?

by ThundaPC on Feb 16, 2008 2:43 AM PST 0 recs

I sent Nate Silver an e-mail,

explaining the importance of our awesome DH and "TOR" dominance. We'll see if he gets back to me.

by Teej on Feb 16, 2008 2:50 AM PST up 0 recs

It was wrong last year, too

So were all the projection systems, so I'm not sure why I should take them any more seriously this year.

by GoddardTP on Feb 16, 2008 2:55 AM PST 0 recs

Because they're often pretty accurate?

No one's saying they're the be-all and end-all, but they tend to be pretty solid projections. Everyone howled about PECOTA's extremely pessimistic view of the White Sox last year, yet it hit the win total right on the head.

Not saying PECOTA's perfect -- it's a projection, for Christ's sake -- but just because it's wrong once in a while isn't enough reason to completely ignore it. Look at the aggregate.

by Teej on Feb 16, 2008 3:04 AM PST up 0 recs

Just because PECOTA hits the White Sox's

W-L record right on the head once, it isn't enough reason to completely believe it.

by Wilder83 on Feb 16, 2008 1:16 PM PST up 0 recs

The exact same argument applies

The other way around though

by Graham on Feb 16, 2008 1:20 PM PST up 0 recs

Good, Graham.

You are catching on!

by Wilder83 on Feb 16, 2008 1:34 PM PST up 0 recs

That's why I said consider the aggregate.

PECOTA has a pretty solid record of being accurate. I'm not saying that nailing the White Sox makes it infallible; I chose that example because it was one that was way out of line with most fans' and experts' predictions, just like this Mariners projection is.

by Teej on Feb 16, 2008 1:29 PM PST up 0 recs

I find it hilarious

there are systems that exist trying to predict W-L records in the first place. I am going to keep this short because I do not want to write a book about this today.

Summary: These systems use projected stats of ballplayers that have inherent flaws in them to predict W-L records. You cannot have an accurate statistical system when the input variables are flawed.

by Wilder83 on Feb 16, 2008 1:52 PM PST up 0 recs

They're not going for 100% accuracy

That's impossible. But there's a lot of value in making educated guesses, and PECOTA's more educated than I'll ever be.

by Jeff on Feb 16, 2008 1:58 PM PST up 0 recs

Don't get me wrong.

I agree there is a lot of value in making educated guesses. However, when it comes to predicting W-L records, I think the inherent flaws of players' stats are magnified and it throws the W-L predictions off significantly.

I haven't looked at the stats specifically because I am not a subscriber. Does PECOTA give a variance when it comes to how accurate it thinks it is?

by Wilder83 on Feb 16, 2008 2:42 PM PST up 0 recs

throws the W-L predictions off significantly =

there is significant variation in the system so its not going to be right a lot of the time but it gives you an idea of what to expect and it will probably undershoot about as often as it overshoots.

We say we'll win 85 games next year. A projection system says we win 77 (or less). Thats a big gap and its not just a few runs here or there. Its worrisome is what it is and we should all be taking a long look at how good we actually think this team is gonna be.

by Edgar for Pres on Feb 16, 2008 2:48 PM PST up 0 recs

Here's the thing

I just ran a quick comparison of the accuracy of PECOTA's RS/RA predictions (as described here, anyway) last season to another, cruder prediction system. PECOTA came out on top, but only by 10%. Of the 60 possible predictions (RS & RA for each team), PECOTA was closer on only 33 of them.

The other, cruder system was simply taking the results from the previous season.

by GoddardTP on Feb 17, 2008 3:19 AM PST up 0 recs

For the players or for the teams?

by Graham on Feb 16, 2008 2:49 PM PST up 0 recs

In this case: Teams

by Wilder83 on Feb 16, 2008 3:03 PM PST up 0 recs

I'm not sure then

I've never subscibed to BP. Players definitely do though.

by Graham on Feb 16, 2008 3:07 PM PST up 0 recs

If PECOTA has them for players,

I would think they would have them for teams as well.

It would be interesting to know what those numbers were.

by Wilder83 on Feb 16, 2008 3:16 PM PST up 0 recs

It would be cool

if somebody generated some sort of similarity score for teams and then built a system kinda like PECOTA on it. It wouldn't work great but it would be an interesting project to read about I think.

by Edgar for Pres on Feb 17, 2008 2:39 PM PST up 0 recs

cry

by Graham on Feb 16, 2008 3:12 AM PST up 0 recs

Because, just as you don't judge

a player based on just one year, you don't also judge the projection system based on just one year?

visiting A's fan.

by rfloh on Feb 16, 2008 4:30 AM PST up 0 recs

Not just one

I've got 4 data points, if you'd follow the link. Clearly, these kind of projection systems had some kind of blind spot when it came to the 2007 M's. The 2008 M's are largely the same team, so why would the blind spot go away?

by GoddardTP on Feb 16, 2008 7:28 AM PST up 0 recs

They more or less nailed expected Pythag

by Graham on Feb 16, 2008 7:53 AM PST up 0 recs

That and a buck fifty...

Will get you a cup of coffee. They all completely whiffed on the offense, and are probably doing it again.

by GoddardTP on Feb 16, 2008 8:37 AM PST up 0 recs

Because, sometimes,

much as some humans dislike to admit it, some things are random? Because not everything has an ordered pattern, a cause and effect?

Because, players are human? Just because the players are the same, doesn't mean that they don't change?

Yes, your link had a bunch of projection systems. It was still only 1 year. Would you assess 4 players just based on what happened last year?

visiting A's fan.

by rfloh on Feb 16, 2008 8:44 AM PST up 0 recs

Not all unmeasured variables are random noise

not everything has an ordered pattern, a cause and effect?

Actually, yes, everything has a cause and effect. Ever hear of a guy named Newton? He has some laws of motion he'd like to share with you. Just because you don't know what the variables are that made every single projection system miss on the '07 M's doesn't mean those variables didn't exist. Because they clearly did.

There was only one 2007 Seattle Mariners team. Just one. Solo uno. Soa um. If sample size is n, n=1. This isn't a coin you can flip a bunch of times to determine its pattern of probability. The projections system had exactly one opportunity to get it right, and they missed.

To use your "four players" example, we have to note that there aren't any other players. So, there are four hitters in all of the MLB. There is a single pitcher--the 2007 Seattle Mariners. 100% of the pitchers struck out 100% of the hitters. Season over. Now, those same hitters are coming up to bat against essentially the same pitcher. How do you bet?

I'm not saying that these projection systems are useless, I'm saying they're apparently useless for the purposes of projecting this particular team, in this particular stadium, with these particular players. If you can't accept the fact that there are variables in play that PECOTA doesn't measure, then you've got problems.

by GoddardTP on Feb 16, 2008 9:01 AM PST up 0 recs

They did not clearly miss

If any system had predicted 88 wins for the M's last year, that one would have clearly missed. Projecting 78-84 would have been an accurate result (although they probably all shot low with RS AND RA).

They are projecting Pythagorean win%, no more, no less, and some teams will overperform that, some will underperform.

by Graham on Feb 16, 2008 9:38 AM PST up 0 recs

They did all clearly miss

In particular, they all missed runs scored by about 50, and all in the same direction (low). If that's not a blind spot, I don't know what is.

It's that same projected offense that people are expecting to make the 2008 M's terrible. Do you see why I don't buy it?

by GoddardTP on Feb 16, 2008 9:56 AM PST up 0 recs

Also

"I'm saying they're apparently useless for the purposes of projecting this particular team, in this particular stadium, with these particular players."

Riiiiight. Because the Seattle Mariners are actually a collection of magical beings from an alternate dimension rather than a team of baseball players, just like the rest of the league. What a load of bullshit.

by Graham on Feb 16, 2008 9:41 AM PST up 0 recs

Holy crap!

Riiiiight. Because the Seattle Mariners are actually a collection of magical beings from an alternate dimension rather than a team of baseball players

You just discovered Bavasi's secret logic in player evaluation he's figured out which players are really Extra dimensional super aliens!

by chrisisasavage on Feb 16, 2008 9:55 AM PST up 0 recs

And you all thought

that the Bloomquist Pony and Gritty Bear were imaginary.

Nice Guys Finish Third - Hopelessly lost, but makin' good time.

by pdb on Feb 16, 2008 9:57 AM PST up 0 recs

Ruductio ad absurdum, nice

An example from biology:

Rats, mice & gerbils are all rodents. Though none of them are magical beings from another dimension, they do have certain very small genetic differences. You can make certain predictions about, say, rats and mice that will not hold true for gerbils. Even though they're a bunch of proteins, fats and carbohydrates put together to form a small, herbivorous mammal, just like the rest of the "league".

by GoddardTP on Feb 16, 2008 10:05 AM PST up 0 recs

but...

...baseball players are all humans, last I checked. Which would mean that you CAN in fact make predictions about Mariners that you can make about Padres, right?

Nice Guys Finish Third - Hopelessly lost, but makin' good time.

by pdb on Feb 16, 2008 10:12 AM PST up 0 recs

I should clarify the analogy

The baseball players aren't the rats/mice/gerbils--they're the proteins, fats & carbohydrates that make up the rodents.

And, both with rodents & baseball teams--and, for that matter, humans--there are a lot of predictions that will hold true for one but not another. And some that won't.

by GoddardTP on Feb 16, 2008 10:28 AM PST up 0 recs

buh?

So are humans any more unpredictable than any other subject?

"You know, as that was coming out of my mouth, I knew that it was wrong."

by JI on Feb 16, 2008 11:00 AM PST up 0 recs

how are

"You know, as that was coming out of my mouth, I knew that it was wrong."

by JI on Feb 16, 2008 11:01 AM PST up 0 recs

I never said they were...

But I could, because it's true, because there are a lot more variables in play. Partly because they frown on keeping people in cages in laboratories.

Which is why social sciences have so much less predictive power compared to the hard sciences.

What I said is that in any collection of different complex systems, despite the similarities of those systems, there are going to be variables present in some systems but not others. When predictive equations are used that don't take into account those variables, then those equations will be more accurate when the variables are NOT present than they are when they ARE present.

In baseball, these variables include, off the top of my head, fatigue, time off due to injuries, playing through injuries, adrenaline, the learning of new skills, new good habits, off-field distractions, in-game strategy, cheating, non-age related physical alterations, umpire mistakes, and so forth. Some combination of these, and probably some I haven't thought of, caused the M's offense to beat all their projections rather soundly.

Graham wants to chalk all these up to "luck," apparently because the British version of Occam's razor says "if my formula doesn't take a variable into into account, it must be completely random". Now, some of these variables are random. But not all of them. I'd rather people did more research into why the M's over performed, instead of just attributing it all to luck.

by GoddardTP on Feb 16, 2008 12:10 PM PST up 0 recs

The simple explanation

Is that you are deluding yourself into thinking that PECOTA somehow doesn't apply to the Mariners just because you don't want it to.

by Graham on Feb 16, 2008 12:15 PM PST up 0 recs

The offense overperformed because...

No injuries. Playing time is projected. A lack of injuries in the hitters is attributable to luck.
Jose Vidro didn't suck. This was luck.
Jose Guillen didn't suck. Kansas City's offense is now obviously unprojectable!
PECOTA always gets Ichiro wrong by a significant amount.

by Graham on Feb 16, 2008 12:25 PM PST up 0 recs

Luck?

So, you've admitted to at least one Mariner-specific variable that PECOTA can't account for. Namely, Ichiro. If one can exist, why can't others?

So, Ichiro's not luck, Guillen's not luck, and frankly, I doubt that Vidro was entirely luck, too. And I'm sure the Mariners' doctors and trainers are thrilled to hear that their services are no longer required, because it's all luck, anyway. Oh, and I'm sure Bavasi's FA and trade strategy doesn't take injury into account either. Since it's all luck.

by GoddardTP on Feb 16, 2008 1:01 PM PST up 0 recs

Because Ichiro is a complete anomaly?

There were 3 major overperformers offensively. One should be expected to overperform again. One should not. One is in Kansas city, replaced by Brad Wilkerson.

Yep, I'm sure glad the M's trainers did such an outstanding job with protecting Felix, forcing Sexson to sit instead of playing through an injury, and all that shit.

by Graham on Feb 16, 2008 1:06 PM PST up 0 recs

Well, it worked last year

But seriously, amateur psychoanalysis via the Internet? Lame. Can't someone disagree with you without having impure motives?

by GoddardTP on Feb 16, 2008 12:51 PM PST up 0 recs

Who gives a shit about purity of motive?

You disagree that the projection system has any merit. I think you're spouting complete bollocks. The simple resolution, in my mind, is that you don't like their projections so you're being stubborn about it. If you don't like it, that's too bad. Go find some reasonable arguments (i.e. not 'adrenaline'. Jesus). Ichiro and Putz are good starting points.

by Graham on Feb 16, 2008 1:02 PM PST up 0 recs

Please, find me the sentence

in which I said PECOTA has no merit. I just think it's not an entirely accurate depiction of reality. It still has its uses, but your apparent argument of "PECOTA + Luck + Ichiro! = Reality" is specious.

And if you don't want to think of adrenaline as a variable, feel free to pretend that my lengthy list of other possible variables off the top of my head didn't contain it.

by GoddardTP on Feb 16, 2008 1:34 PM PST up 0 recs

Look

I didn't mean to become embroiled in this fight. I just think that drawing conclusions again the team's unprojectability from one year's worth of data is a really bad way of doing sensible analysis. If there was a trend, sure, but...

by Graham on Feb 16, 2008 2:48 PM PST up 0 recs

Fair enough

The trouble is, there can't possibly be a trend, because we only have one data point for that group of players/coaches/etc. It's just a limitation of the system.

by GoddardTP on Feb 17, 2008 3:11 AM PST up 0 recs

So the Mariners are the only team

this would apply to?

"You know, as that was coming out of my mouth, I knew that it was wrong."

by JI on Feb 16, 2008 12:24 PM PST up 0 recs

Or I can go with Occam's Razor

The Mariners are not impossible to predict. The Mariners were lucky.

by Graham on Feb 16, 2008 11:08 AM PST up 0 recs

I did not say that all unmeasured

variables are random noise. I said SOME things are random. Do you agree?

"He has some laws of motion he'd like to share with you. Just because you don't know what the variables are that made every single projection system miss on the '07 M's doesn't mean those variables didn't exist. Because they clearly did."

Newton said "For every action, there is an equal and opposite reaction." Not, everything has a cause and effect.

I said nothing about the projection systems not missing on 2007. I said it's ONE year.

"To use your "four players" example, we have to note that there aren't any other players. So, there are four hitters in all of the MLB. There is a single pitcher--the 2007 Seattle Mariners. 100% of the pitchers struck out 100% of the hitters. Season over. Now, those same hitters are coming up to bat against essentially the same pitcher. How do you bet?"

I ask how those hitters did in 2006 against that pitcher. I ask how those hitters did in 2005 against that pitcher. Et cetera.

"If you can't accept the fact that there are variables in play that PECOTA doesn't measure, then you've got problems."

Find another strawman to argue with. I'm not going to waste my time.

visiting A's fan.

by rfloh on Feb 16, 2008 9:46 AM PST up 0 recs

You're a strawman?

I thought you were just bad at this.

I ask how those hitters did in 2006 against that pitcher. I ask how those hitters did in 2005 against that pitcher. Et cetera.

You can't--because the M's, as constructed in 2007, didn't exist in 2006.

by GoddardTP on Feb 16, 2008 11:48 AM PST up 0 recs

they were not wrong

You're dealing with projections, not predictions. PECOTA digests a number of data points and spits out an expected performance for the year. It is the expected performance because it is the most likely outcome, not the only possible outcome.

For the most part, PECOTA's predictions are accurate enough, and while it does not forsee and adjust for every statistical anomaly (Ho's W-L record), it is still worth taking seriously.

by Mere Tantalisers on Feb 16, 2008 1:57 PM PST up 0 recs

Honestly,

not to bash PECOTA, but I have no idea how they have Kenji sluggin .395 next year. Also, Yuni should slug better than .390.

I dont know, I just would be very surprised if Yuni, Kenji, and Ichiro all failed to slug .400.

Other than that...jeez those standings are scary

THE FUTURE Richards, Cutler, Felix

by dbroncos31 on Feb 16, 2008 3:18 AM PST 0 recs

That doesn't seem that unreasonable

Johjima is a 32 year old catcher and his SLG was .433 last year. Likewise, Betancort posted a .403 SLG in 2006 and a .413 SLG last year. A .400 SLG isn't exactly a big regression.

Ichiro is another story, and it's probably best just to ignore the PECOTA projection.

by MrIncognito on Feb 17, 2008 3:00 PM PST up 0 recs

Does anyone know

what replacement level that BPro is using for VORP? 20 runs below average?

Looking at their projections for another team, the Mets for example, that seems to be about what they are using: 20 runs below average.

For example, Carlos Beltran is projected to 46 VORP.

If that is the case, some of these Mariner projection seem pretty low.

I don't see how Ichiro! is 17.0 VORP, offensively. Especially since he is now in center, and not a corner. 17 runs above replacement offensively? He's been about 15 runs above a league AVERAGE hitter offensively throughout his career.

Johjima only 10 runs above a replacement CATCHER offensively? He's been a league average hitter the last couple years.

visiting A's fan.

by rfloh on Feb 16, 2008 4:47 AM PST 0 recs

Ichiro 22 runs above average throughout his

career.

visiting A's fan.

by rfloh on Feb 16, 2008 5:13 AM PST up 0 recs

PETCOA

Is racist against Japanese players. Duh ;)

by Limerickx on Feb 16, 2008 9:16 AM PST up 0 recs

Nate Silver

wrote an article about Ichiro a couple years back and why PECOTA doesn't like him. If you want to read it its right here, but if you don't the basic gist of why Ichiro's projection is alway off is because nobody has his skillset and the players that have similar skills suck a grand majority of the time.

by bergini on Feb 16, 2008 4:29 PM PST up 0 recs

I'm aware of that

I'm aware that balls in play is usually a good thing, for pitchers.

The article still does not explain why 3 years later, Silver is still using a model that does not work for Ichiro to project Ichiro.

visiting A's fan.

by rfloh on Feb 16, 2008 9:15 PM PST up 0 recs

Because it works for 95% of the players?

You just have to throw Ichiro out as an anomaly.

by Matthew on Feb 17, 2008 7:27 PM PST up 0 recs

One of these times

I think the projections will get it right because I really don't see Ichiro aging gracefully. I'm afraid he could go from an All-star to replacement level offensively in a year or two. I love him but at some point he's gonna get hurt and lose footspeed or his batspeed will slow down. I just don't see Ichiro playing at the same level in 3-5 yrs with his skillset. He's proved everybody wrong already though so proving me wrong should be a piece of cake.

by Edgar for Pres on Feb 18, 2008 8:03 PM PST up 0 recs

Well, yeah.

But if you predict him to do worse every year, someday you're bound to be right. Unless he retires coming off the best year of his career, in which case I'll tattoo his face on my scrotum to pay homage to his masterfulness.

...and now I'm here

by Librocrat on Feb 19, 2008 12:23 AM PST up 0 recs

Thing is,

studies, at least by Bill James, have shown that "toolsy" athletic players age more gracefully than those with "old man" skills.

visiting A's fan.

by rfloh on Feb 19, 2008 11:03 AM PST up 0 recs

I apologise for responding late

I'm not saying that he shouldn't use the model for the other players.

I'm saying that he shouldn't use it for Ichiro. It clearly does not model Ichiro anywhere close to realistically. Hell, just use a simple Marcel for Ichiro, if he does not want to spend the time coming up with a new model for Ichiro.

visiting A's fan.

by rfloh on Feb 19, 2008 11:02 AM PST up 0 recs