Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
Around SBN: Trent Richardson Interviews Fellow Brown Brandon Weeden

When Good Writers Get Lazy

The Prospectus Triple Play made its way back to the Mariners today. Go ahead and click; it's a free article.

After making the obligatory mention of new resident stathead Mat Olkin, Jim Baker proceeds into a brief discussion of 2005 PECOTA projections for certain Mariners. Which would ordinarily be perfectly fine, but Baker treats the forecasts almost as indisputable fact, when the reality is that there are a few significant flaws in his reasoning. For example:

(1) Yes, Sexson is pegged for a 23.6 VORP. In about 425 plate appearances. Healthy Sexson collected 2,025 plate appearances between 2001 - 2003 - 675 a year. Projecting that VORP over a full season gives a figure of 37.5, which would put Sexson's production at 2004 Kevin Millar.

But then, I don't really like VORP forecasts anyway - trying to predict couting stats is a fool's errand. PECOTA sees Sexson posting a .254/.347/.480 line in Safeco, which adjusts to a .261/.357/.504 line in a neutral environment. If that doesn't seem like a perfect match to Sexson, circa 2002, then I don't know what does. Still not really worth the contract, but it's a comfortably above-average performance. Baker's selling him short.

(2) All right, so PECOTA doesn't like Adrian Beltre's chance of building on his 2004 season very much. A .279/.337/.486 line in Safeco isn't out of the question. That said, Beltre's Similarity Score has got to be frighteningly low - that is, there isn't an established historical precedent for 25 year olds suddenly exploding with an MVP-type season after underachieving (partially) due to injury. With this in mind, attempting to project Beltre's 2005 season by comparing him to similar players is a futile endeavour. With so few comparisons, the forecast range will be enormous, and averaging them out into a weighted mean is just wasted time. PECOTA's neat, but it can't account for statistical outliers - just like with Jamie Moyer and Randy Johnson, all we can really do with Adrian Beltre is guess and watch.

(3) As if that weren't enough, Baker goes on to mention the Ichiro forecast. You know who PECOTA considered to be the most similar player to Ichiro a year ago? Matty Alou, a career .307/.345/.381 hitter with 156 steals in 1667 games. Others on the list of comparisons include Lance Johnson, Tony Womack, and Luis Polonia. Nate Silver might as well just make a random guess for Ichiro's performance every year; it would probably be more accurate than whatever numbers his computer spits out.

(4) Baker's final point is that Joel Pineiro will lead the pitching staff with a 22.1 VORP, the third-lowest in baseball among team leaders. Looks pretty bad, right? I mean, Ryan Franklin was miserable last year, but even he managed a 22.7 VORP.

So what's the deal? Well, as a conservative forecast system, PECOTA tends to underestimate plate appearances and innings pitched. This year, it projects Pineiro to throw 138.1 innings, Moyer to throw 146, Madritsch to throw 106.2, Franklin to throw 147.2, and Meche to throw 124. The biggest component of VORP is playing time - this is the nature of any counting stat. When you undershoot the probable innings pitched of everyone in a starting rotation by 50-60 IP each (although, again, Franklin could lose his spot), they're obviously going to wind up with unimpressive counting stats.

The thing is, Baker knows all this, but neglected to mention it. Maybe he didn't have enough space in the article. Maybe he assumed that everyone would understand the caveats inherent in his reasoning. Maybe he couldn't think of anything to write about, so he went with something easy. Whatever the case, it came out as a throwaway column. I've come to expect better from the guys over there, and this kind of thing disappoints me.

Comment 10 comments  |  0 recs  | 

Do you like this story?

Comments

Display:

Baker's PECOTA articles
That was probably the worst piece I have ever seen at BP.  

Baker just uses PECOTA like a meat grinder to crank out a column.  Even Buster Olney puts more effort into his work than I think Baker put into this piece of trash.  To get this junk up on the BP website is a double failure, because not only did Baker write it, it got past someone who is (or ought to be) reviewing and approving what gets put up.

BP plays itself as a site that provides thoughtful insight and analysis.  I hope someone over there is embarrassed by having their name attached to dreck such as this.  I never thought I would see the day when something at BP would make Phil Rogers seem intelligent and reasoned in comparison.

by Steve Nelson on Feb 4, 2005 4:20 PM PST reply actions  

Re: PECOTA not reliable across-the-board
PECOTA is a suggestion. Nothing more, nothing less. Among the available projection systems, it's probably the best we've got, but it's not enough to make or break a case.

by Jeff Sullivan on Feb 4, 2005 5:18 PM PST reply actions  

PECOTA
Jeff,

I know that PECOTA has proven to be a pretty good projection system, but this is sorta inherent in the way it is designed.  

The way that PECOTAs degree of error is evaluated is by looking at all projections versus the real stats afterwards.  Thus, the guys that overachieve and the guys that underachieve will tend to cancel each other out.  I know that it is a lot more complicated than this, but the conservative nature of the projection will result in the system projecting a VORP midway between bad and good.  

For instance, if Sexson is fully healthy, he will put up a 50-60 VORP over the course of a season (155+ games).  If he is not OK, he might not play at all, or only play to a level of 10-15 in limited time.  The system hedges it bets, so to speak, by estimating that Sexson will play in about 100 games and put a VORP of 24.  

Since the system is designed to work well over large samples, it will get the best results with these conservative estimates.  It will estimate high VORP values for players who have been consistent producers and consistently healthy.  Thus, guys like Sexson and Beltre are going to get low values.  

This article is a perfect example of a stat guy becoming so pleased with his analytical tool that he starts to believe what it tells him, instead of using that data critically to understand something.  It is pretty sad.  

by Jerry on Feb 4, 2005 9:12 PM PST reply actions  

Phwhewh...
I got "busted" (lectured, moreso than "Corco'd") over at USS Mariner for hijacking a thread trying to get help on my internal debate as to whether or not I should shell out the dough for a subscription to BP (or a similar web site) or spend a little less buying BP 2005 and Bill James' 2005 book.  With further counseling from my former roommate (who contributed to James' book, yet still was able to give me balanced information that helped me make my decision), I decided to go buy the books.  

After reading Baker's bit, boy am I glad I made THAT decision!  Of course, Amazon shipped me the BP 2004, a blunder on their end since the 2005 I ordered isn't out quite yet.  So, I don't yet have access to PECOTA stuff.  Looking at this article, I'm less inclined to go with PECOTA this season.  Sure, it's one of the best forecasting tools out there, but I think it needs to do a MUCH better job at forecasting the outliers before it's really classified as a solid system.  

I mean, heck, I fundamentally agree with Tossed Salad in that from what I've seen, it can really only accurately predict truly predictable players.  Anyone who pays attention to baseball should be able to do that, no matter how much attention to detail you pay to the stats.  

The fact that, according to the article, Bret Boone is predicted to lead the offense in VORP is just flat out wrong!  I think he'll most certainly bounce back, now that he doesn't have to carry this team on his shoulders, but it's absurd to think he'll lead the team in VORP.  Plus, Ichiro has been around the league for 4 years.  You'd think he'd be a little more forecastable with his experience level.  

Still, again, it does appear that the writing over at BP has severely declined, even in the limited stuff I've been able to read for free.  If I ever get Amazon straightened out and get BP 2005, I'm sure I'll find it useful.  Their web site content, though, is not all that enticing.

Go M's! http://marinersmorsels.blogspot.com

by PositivePaul on Feb 4, 2005 11:50 PM PST reply actions  

The thing to consider
with PETCOTA, they are forecasts and should be treated as such. Much like weathermen who make 10 day forecasts, they are predicting what they believe will occur. Sometimes they are right, other times they are wrong. You have to take the good and the bad.

by Trent on Feb 5, 2005 1:55 PM PST reply actions  

The thing to consider
Trent - I totally agree about taking the good and the bad.  But Baker's piece was just ugly - Eli Wallach ugly.

by Steve Nelson on Feb 5, 2005 2:33 PM PST up reply actions  

Eli Wallach
haha. I would have gone with Otis Nixon ugly, but Eli Wallach is much better. There is no denying Baker's piece exemplifies what happens when you forget you have a deadline, I was just commenting on some of the comments.

by Trent on Feb 5, 2005 3:24 PM PST up reply actions  

My bad!
Sorry for the misconjugation!  I certainly wouldn't want to imply a change in tense, especially to the past tense :-)

Error M's blogger, if you're scoring at home...

Go M's! http://marinersmorsels.blogspot.com

by PositivePaul on Feb 5, 2005 10:46 PM PST reply actions  

Comments For This Post Are Closed


User Tools

By reading a game thread of your own volition you agree to accept all liability for any and all damage done to your delicate sensibilities.

FanPosts

Community blog posts and discussion.

Recent FanPosts

Small
Starlin Castro's fit with Seattle
Kawasaki80_small
Lists! So many lists!
M_s_hat_copy_small
OT -- May 22nd In Memoriam
Ichiro_small
Why do managers and media members hate walks?
Wbc_029_small
Friday Morning Music Thread
Small
Dustin Ackley BP swing vs game swing
Beastquakerwallpaper_small
More on the Struggles of Smoak
Randy2_for_sbn_small
Albert Pujols 2012: Three Retrospectives
Small
On Batting Orders
Niehaus_small
More on Dustin Ackley and the strikezone

+ New FanPost All FanPosts >

Yahoo_full_count

Sexy People

Wbc_029_small Jeff Sullivan

Small Matthew

Claw_small JY