Theo sounds like Jack in a confiding mood

Data collectors, stringers? I’m at a loss. Anyway. Their standards and training, how they limit bias on the input end and create some level of consistency between teams/games/leagues.

This is kind of extreme, but what happens if the people watching the plays are fans? For instance, Raul Ibanez had a specific, well known reputation for defense in Seattle with a very vocal, insistent fan base promoting that image. He signs in Philly and it’s a popular signing, maybe more for his bat like Gold Glove awards seem to go to the better hitters at a position. So if the stringers are a part of the local fan base, will that influence how they score a play?

I don’t even know how they collect the data, so this is all conjecture. But I do like the idea of the Dewan +/- system, from what I’ve read every play is centrally reviewed. So if their is bias then perhaps that mitigates the effect, or at least makes it consistent.

I don’t think this will work without explaining exactly how UZR works, and I’m probably going to screw this up, but this is a very basic explanation from my understanding of it:

Basically, UZR breaks the playing field into 64 different zones and assigns a run value to a hit in each zone. For example if a hit to a particular part of the field often results in extra bases, it is assigned a higher run value than a part of the field that often results in singles, etc..

For each year, the system records the number of hits in each zone, the average run value of hits in each zone, and the number of outs recorded in that zone for each fielding position. (Some zones overlap between multiple positions, so outs are recorded by individual position).

For each player and each position, they then record the number of hits vs. outs recorded in each zone while that player was on the field. Some zones are on the fringes of a positions ability to get to, and plays made or not made in these zones generally tell you a lot about a particular player’s range, and also have higher run values.

The reason UZR data generally takes 3 years to stabilize, from my understanding, comes from the fact that individual players sometimes don’t experience a whole lot of balls hit to those fringey zones that really help determine his range over the course of a single year. There’s some luck involved in which zones they have a chance to make a play at in a given year and exactly where in that zone the ball was hit, and that can mean you get data that’s not accurate enough to draw conclusions from.

There’s probably more involved with it than that, but that would be my guess where the small sample sizes come into play from a single year of data. Once you get to three years, you have a much bigger number of plays made in each zone to work with for each individual player, and you can draw firmer conclusions about their ability.

Fielding doesn’t have the TTO components that batting does, and they serve to stabilize batting stats considerably. Thinking of fielding as, say, FABIP calls to mind the way a player’s BABIP will fluctuate more between seasons than overall batting stats will. Regardless, batting outcomes on balls in play still strike me as more random than fielding measurement, which looks to take account of where in the zone the ball is and how it gets there.

along with the view of a couple of MLB scouts in 2006 who thought Betancourt was basically Ozzie Smith, but maybe better.
The problem is that UZR never viewed him as an exceptionally rangy SS. At the time, we could shrug it off as a sample size problem. But with the benefit of hindsight, I start to wonder. Again, the guy I saw in Tacoma seemed like exactly what you describe: perfection at shortstop. Combined with the mediocrities in LF at the time (including Shin-Soo Choo), I swear Betancourt played SS and LF simultaneously. But while we could always shrug off his poor UZRs to sample size weirdness, his career arc gives one pause. He’s never posted a positive range rating…. ever. We all agree that the metric is describing reality now – what do we do with his meh ratings in 2005/6? I’m perfectly comfortable with the answer ‘ignore them’ because that preserves whatever validity I’d attach to my own meager scouting reports on the guy from 2005. If any conflicting data is ‘completely untrue’ then I’m more than OK with that.