Throw Zone Ratings Down the Well So the WAR Can Be Free

| | Comments (11)

Statistics are hard, y'all. When baseball nerds decide that mere averages are not enough to judge a player's worth and start using algebra, it really throws the rest of us for a loop. Heck, I minored in mathematics in college and some of the newfangled sabermetric calculations make my brain melt.

But Wins Above Replacement, or WAR, is important, even if you (or I) don't quite understand where the numbers come from. To put it simply, WAR tries to figure out how many wins a player adds to his team through his hitting and fielding output. It makes adjustments for positional difficulty and the cost of replacing that player with a regular schmo.

Bill Baer at the Phillies blog Crashburn Alley, now hosted by ESPN as part of Rob Neyer's Sweet Spot network, takes an academic approach to explaining WAR, saying that while the equation itself is not perfect, the theory behind it is sound. He mentions Philies beat writer Dennis Deitch, who criticizes the WAR equation for not making any sense when it tells him that Mark Reynolds is more valuable than Ryan Howard.

The thing is, they're both right, and neither one is wrong. The equation for WAR is ideal for evaluating a player's worth, in that it deigns to tell us how a player performs at the plate, how a player performs in the field, and making two necessary adjustments (positional and replacement-level) to even the ratings. The WAR equation wants to make sure that a good fielding shortstop gets more credit than a subpar leftfielder. It also gives credit for more games played and innings played, and, in the case of Jeff Francoeur, takes credit away for playing too many dang innings.

But Deitch is correct when he claims WAR doesn't really pass the smell test. Ryan Howard is having such a massively better offensive year than Mark Reynolds that the difference in their respective defensive prowesses should not be swinging the WAR equation in favor of Reynolds. Yes, third base is harder to play than first base, so the positional adjustment should be there. But the problem with WAR is that there is no universally-accepted way to measure actual defense.

It's not the equation that is faulty, it's one of the variables. And that faulty variable is the defense.

Right now, the saber-y website Fangraphs uses something called Ultimate Zone Rating, or UZR, to calculate the fielding runs in the WAR equation. Meanwhile, the statty website Baseball-Reference uses Total Zone to figure out the fielding runs in their WAR equation. Which is why Mark Reynolds can simultaneously have 2.4 WAR and 1.2 WAR...there are two different ways of calculating his glovework!

Both UZR and Total Zone are intelligent, forward-thinking, and useful ways of evaluating individual defense but if they don't agree over a sample size of 140+ games, how can we trust the WAR equation to properly evaluate a single season for a player? The problem is that the zone ratings themselves use faulty source material: their numbers come from this map, used for any baseball field:


That looks nothing like the baseball parks I've visited this year. More like the ones I visited in the 1980s, the cookie cutter fields in multipurpose stadiums that have gone the way of the dodo everywhere except St. Petersburg, I suppose. Plus the zones are all different shapes and sizes; how can it be effective when there's one the size of Prince Fielder and one the size of Chuck Knoblauch?

What these nerds need is to abandon zone ratings entirely and move towards a more concise way of evaluating fielders. And they may get it, soon, thanks to the same folks who brought PITCHf/x to the blogosphere. The company Sportvision has recently introduced FIELDf/x cameras to AT&T Park that can capture 15 images per second of the entire field and measure defensive events. This data can be extrapolated and manipulated to objectively determine how fielders approach a fly ball, or turn a double play, or just about anything.

Rob Neyer attended the recent PITCHf/x summit and had this to say:

Greg Rybarczyk -- the genius behind Hit Tracker -- introduced the idea for a new defensive metric: True Defensive Range (TDR).

Why on earth would anyone want another new defensive metric?

As Greg pointed out, all of our current "new" defensive metrics are "zone-based"; that is, they begin by separating the field into distinct zones, noting in which zone a play has been made, and then apportioning credit (or not) when a fielder makes a play (or doesn't) in that zone. I'm simplifying, of course, but essentially every reputable system now in use shares two significant defects: the "zones" don't fit neatly into today's highly variable outfield dimensions, and the systems don't have any way to account for the fielder's starting position.

Cameras are the future. You can watch Greg's presentation at the Sportvision site, but if you don't have a half hour to spare, here's what you should know: having multiple, smart cameras focused on the outfield during every major league game, we can objectively determine which fielders are making an awesome play, and which players are just making an average, can-o'-corn type play. FIELDf/x is the future.

Similarly, the cameras focused on the infield can tell us the same thing about the Derek Jeters and Chase Utleys of the world. Maybe some day, we'll even be able to objectively evaluate foul ball catches made by fans, with adjustments for fans holding babies or beer.

Unfortunately, the data is not universal yet. In fact, the smart folks who presented at the summit had access to just a few innings worth of defensive plays to work their mathematical magic on. But soon, the nerds who make sure baseball statistics keep progressing forward will get their mitts on this data, and they'll be able to give the WAR equation a sense of standardization.

PREVIOUS: Far Within Us #4: Today's Afternoon Games: NL Wild Card Edition   |   NEXT: Great Moments in Creampuffery: Paul Konerko


Even with the cameras, are things like defensive positioning taken into account? Would a Jim Edmonds dive into the center field wall count more than when Curtis Granderson makes the same catch because he is actually positioned correctly?

Yes. That's the main advantage of FIELDf/x over the zone ratings.

With cameras, it can all be vectored up like a motherfucker. Peter Bourjos ran XX meters on a ball with YY seconds of hangtime to make a catch on ball hit with ZZ trajectory. Jim Edmonds fell over and the ball hit him in the coconut.

I think the whole "replacement level" thing throws people. It isn't an absolute figure and it changes depending on the position. Replacement level for first baseman is set by a quad-A slugger (Shelly Duncan, for example) while replacement level shortstop would have a much different offensive profile.


Who was that guy that hashtagged "defense matters", anyway? I forgot...

A whole week of pretty much no updates and YOU THROW MATH AT US ON A WEEKEND? What is wrong with you savages....

Apparently though, whenever I went to Brewers games this year, Edmonds' positioning was quite sound. Maybe I bring out the best in ol' Jimmy. Perhaps I'm the reason the Cardinals won the World Series over the Tigers. My presence made Jim an intelligent fielder...

In other news, Hasbro just approved my new toy idea of "Hungry Hungry Fielders." The object of the game is to collect as many cheeseburgers as you can.

"Hungry Hungry (Prince) Fielders" for those of you not in the know about cheeseburger Fielder.

Nice read, nerd.
I made a comment last night about Pablo Sandoval's WAR:
2009: 5.3 (quality)
2010: 1.0 (bench)
I'm assuming I can compare his numbers from year to year since he didn't change teams. He only changed belt sizes.

#defensematters is Matthew Leach's baby.

One note, which doesn't negate the real message of your post (i.e. that fielding metrics are imperfect): UZR (and maybe TotalZone, too, I don't know) actually is park-adjusted.

MGL explains how the park-adjustments work at his UZR Primer. It's actually pretty intuitive.

Also, LOOK at the baby! He's INCREDULOUS!

Leave a comment