When Bill James updated his Historical Baseball Abstract with Winshares in 2001, he felt comfortable about the offensive components but still was uneasy about defense. It’s very difficult to measure defensive skill and defensive value, and to make matters worse, skill and value are not necessarily related.
In researching the odd statistical variance between Bill Buckner and Steve Garvey, he hit upon a key element of defense which makes it difficult to quantify: discretion. Specifically he noticed that Buckner, who carried a weak defensive reputation racked up a ton of assists while Garvey, who owned four Gold Gloves, did not.
In the real world, this was a trivial distinction; all it really indicates is the preference of each player in making a certain play. Baseball players are taught from a young age that, when a ground ball is hit to the first baseman, it is the pitcher’s responsibility to cover first base. Buckner, in part because he had constant pain in his legs, was fanatic about insisting that pitchers do this. I can still see him in my mind’s eye, standing five feet from first base, fielding a slow-hit grounder with the glove on his right hand, pointing vigorously to the bag with his left hand, saying “Your play. Get over there. Cover the bag.” … If a pitcher failed to cover first, Buckner would immediately go to the mound and tell him about it.
Garvey, on the other hand, was paranoid about making unnecessary throws, and strongly preferred to make the play himself if he could. As Garvey saw it, why risk the throw when you can make the play yourself? In part, he saw it this way, no doubt, because he couldn’t throw; he was a fine first baseman, but he had no arm.
…Thus if you use assists by a first baseman to represent “range,” you will reach the conclusion that Buckner was a much better defensive first baseman than Garvey. … The problem with this is, it’s just not true. Buckner was not an outstanding first baseman, and Garvey was not a poor defensive first baseman. A hundred and twenty extra assists per season doesn’t really have any value to the team in this case, because it doesn’t refelct anything other than a choice.
For most of my high-school career, I played in left field next to a speedy center fielder and behind pitchers that often over powered their competition. I played more shallow and towards the line than a typical left fielder would play. I wasn’t fast, but I made good reads and got good jumps. The centerfielder had a better arm than I did, so if we converged on a ball with men on base, I let him make the catch. His skill-set in center let me get to foul balls and turn bloop singles into outs. I may not have made as many plays as I would have as an individual playing a more conventional depth, but as a team, we probably made more plays.
We’ve come a long way since 2001 with the defensive statistics. FanGraphs uses UZR and there’s Dewan’s Fielding Bible’s +/- system plus tons of other stuff is evolving all the time. Not to mention proprietary information hoarded by some, if not all, of the clubs. These systems are dogged and incredibly detailed. The most widely employed use the best information available to divide the field into buckets and to describe the kinds of balls hit into those buckets, and then to assign credits and debits for the plays that are made or not made. It’s a little dizzying, but if you want to read about all the hard work this entails, and how well-thought out the systems are, check here, here and here.
But baseball defense is a team defense, dynamic and shape shifting. It resists our attempts to assign individual debits and credits. The manager scatters his players over the field in whatever manner he thinks will prevent the most runs from being scored. During the course of play, he redeploys the fielders from batter to batter and situation to situation. The coaching staff’s ability to properly position the players – and the organization’s ability to scout opposing hitters effectively – directly impacts a player’s defensive statistics.
Beyond positioning, there are several plays in any game which may be made by more than one fielder. The fielder who catches the ball gets all the credit, the person who stands next to him gets none. That makes sense when you want to record the event for posterity, but it tells us absolutely nothing about the players’ skills nor his contributions to the team.
The best way to start defensive analysis is at the macro level. How did the team do at preventing runs? What was their defensive efficiency? And then assign credit amongst the players for the total accomplishment with the understanding that defense is a team-wide effort.
Right now, to decide the value of each player’s defensive contribution, we take a player’s runs saved or allowed and compare it to the average player at the same position in his league that year. This is not nearly as definitive a tally as it is for hitters. Take any hitter and compare him to the Major League average and you’re comparing him to 269 other lineup spots, and 170,000 other plate apearances. Compare Derek Jeter to the average American League shortstop, and you’re comparing him to only 13 other data points.
It’s no wonder a player can leap from ten runs below average to ten runs above average in a single season. With absolutely no change in skill or performance on the player’s part, or even with the expected decline in skill that accompanies age, an injury to an elite fielder or a change in positioning philosophy can swing the results dramatically.
Next we take those runs above or below average as if they were exaclty as reliable as our offensive runs calculated and mash them together to get an all-encompassing WAR. If a player is found to be nine runs below average on defense, he loses about a full unit of WAR. But does being nine calculated runs below average – whatever average that year happens to be given the radically shifting annual landscape – actually cost your team anything at all? I’m not confident the results are so precise.
I have no doubt that we’ll keep getting better at this. I have no doubt that there is useful information in what we already have. But I also have no doubt that current defensive statistics should not be used to decide someone’s MVP vote nor free agent contract amount. They should be evidence in favor or against a player, but in proper proportion to their inherent lack of reliability.
When Bill James discovered the flaw in the relative ratings of Buckner and Garvey, he solved the problem in the statistics by looking backward at the question. By looking to traditional scouting and peer review to understand the statistics, he was able to eliminate the noise. Isn’t it possible that something similar is happening in the Bronx? Because right now Curtis Granderson’s defensive statistics are dragging down his MVP campaign to the point where his ability to hold down center field for a playoff team is actually costing him ground to corner outfielders and first basemen.
For two years in a row, the Yankees have played excellent outfield defense. They have two speedy guys capable of holding down center, so one of them shifts to left. And then in right field, the least demanding position in Yankee Stadium’s outfield, they have a guy who seems about average. In 2011, the way the Yankees have positioned their outfielders combined with the distribution of fly balls hit at them, along with their own good days and bad days, spits out that the left fielder is having one of the greatest defensive seasons on record while the center fielder is having the worst season of any center fielder in the league.
I’m sure Gardner has caught a lot of balls in left center, but did those catches actually add any value to the Yankees? Or, isn’t it much more likely that Granderson could have caught most of them too? If Gardner was next in line behind Willie Mays and Andruw Jones, wouldn’t a scout have noticed in the minors? If Granderson was all of the sudden the worst center fielder in the league, wouldn’t that show up somewhere else in his speed game?
What is most likely happening is that Gardner is an excellent, though not historically great, defensive outfielder, an excellent center fielder slumming in left. And Granderson is a good center fielder who just doesn’t get to show it as much. But the Yankees don’t care because as a team, they make a lot of plays.
There are going to be more Eureka moments like James had with the first baseman. In the meantime, hopefully MVP votes and free agent contracts don’t get any loopier than usual.
It's worth noting that Gardner's defensive stats in CF were ALSO off the charts, and that many people have been advocating a Gardner in center, Granderson in left strategy for 2 years now.
[1] I agree that Gardner is great, and is the best center fielder on the Yankees. But I don't believe he's saving the number of runs he's getting credit for nor that Granderson is costing the Yankees the numbers of runs he's getting debited for.
I think if you look at them together and w/ swisher, you see a successful outfield that happens to catch balls in a way that the defensive stats are not sophisticated/nuanced enough to track accurately.
You've hit the nail on the head. The reason I am not a big fan of defensive metrics is because it treats outcomes as a zero sum game. When Gardner ranges into CF or Cano goes out for a popup, the Yankees get the out, but Granderson is penalized. That doesn't make any sense. For whatever reason, I've noticed that Granderson defers to fielders entering "his zone" more frequently than most CFers, something that negatively impacts a zone-rating based calculation. Also, Tyler Kepner recently reported that the Yanks shift Grandy toward RF because of Gardner's range, another decision that makes Granderson look "bad".
Defensive metrics and derivatives like WAR are very valuable, but they do require discretion, something of which I think too many lose sight.
[1] Granderson is a truly elite outfielder, but it's worth noting that when he played CF, he didn't have a version of himself vulturing balls.
That's really interesting.
So, Jon, do you think that using the basic ideas the sabermetrics uses, but applying them to, say, an outfield rather than a particular player, would give pretty reliable results? I have some doubts about that. But at least you'd iron out some of the problems with the current systems.
What made baseball so amenable to serious statistical analysis was in part the spectacularly individualistic nature of the offensive portion of the game. You can find a measure of a player's offense -- say, wOBA -- and then you can be confident that replacing that player with someone else whose wOBA you know will change the team output by an expected amount. There's nothing comparable in basketball, say. And you can't answer the question, "How valuable is Tom Brady?" You can only answer, "How valuable is Tom Brady on the Patriots?" and "How valuable would he be on the Bengals?"
Maybe baseball defense is more like football or basketball, and not like baseball offense. Each player contributes to the team, but the whole is not simply the sum of the parts. And that's why sabermetricians have been thwarted.
Just speculating.
[4] What if you started with the question - are there balls in play to the outfield that are usually outs for other teams, but that the yankees don't catch? Then find out where those balls are landing, and then figure out, to the best or our ability, given the yankees defensive alignment, who was responsible for that area of the field?
On FanGraphs, Swisher is worth +10 runs to the yanks, Gardner +19, and Granderson -9. i'd like to know where these balls in play are landing for hits and extra bases given that array.
[4] Another excellent observation. The inter-connectivity of defense goes well beyond how fielders interact. For example, if the entire defense shifts based on pitching patterns, but the pitcher misses location, the result could be an otherwise fairly routine ball that lands safely for a hit. On the whole, you'd expect that even out, but maybe not as much as assumed. Teams with great pitchers should be able to hit their spots more, so presumably, the defense should be more efficient (if being positioned properly).
In many ways the relationship between pitching and defense is a chicken-and-egg proposition, just like the link between RBs and an offensive line.
[3] You mentioned something about that zero sum nature in a previous thread and that lined up exactly with some of my thoughts and I figured, well if William agrees, maybe I'm not crazy and should write it down!
[5] RIght.
Well, one issue is that it's doubtful the alignment would be the same every day. So you might see Granderson shading to right-center, but then if you saw extra hits landing in short right-center you would have to be sure they hadn't re-shifted the outfield for the particular batter.
Here's a possibility. If there were a record for every single PA of where the given OFer was positioned, you could plot every ball that landed in the OF and every ball that player caught, and you'd get a kind of cloud shape around him representing his range. The cloud shape would not be a region of the field, but would be defined relative to the fielder as he moved around.
You'd still have to factor in overlapping fielders -- somewhere you'd have to account for the balls that the other fielders caught. And obviously you'd have to separate line drives, soft flies, and so forth.
Hmm. It would be an interesting piece of information, but I still wouldn't use it the way I'd use offensive stats.
[7] Hate to break it to you, but if I agree, there's a much better chance you are crazy.
[6] Yes, absolutely. You definitely can't expect it to even out in the long run, unless you were looking at a player who switched teams a whole lot.
My thought is that dividing the types of hits into line drives, soft flies, and so forth, will help to iron those things out. But the division will be somewhat subjective and a bit unreliable, so it only helps to a degree.
Also, I hadn't thought about the non-zero-sum problem, but that's also huge. Take an extreme case: you have a the fastest OF in the history of the world. Now add the second and third fastest to his left and right. Their contributions must be less than the sum of their individual abilities, because you're getting Willie Mays stealing hits that Duke Snider would have snagged anyway, and so on.
There's a bit of this in offense, too, insufficiently appreciated, but the effect is much smaller.
[8] I think the only truly reliable system would be a technological, not statistical one. Perhaps using PitchFX like systems, the speed, angle, and direction of the ball could be compared to the fielders starting position and plotted against outcomes. For example, one play might produce output like: CF catches ball (114mph, 45 by 160 degrees) traveling 24 feet from point of origin. A formula could then convert that data into a ranking based on league-wide comparisons.
[11] So we might get better metrics with more technology, true.
There's an inherent limitation. Once you break things down into such fine-grained categories, the sample sizes for any particular type of ball are miniscule. You would have to make assumptions (or maybe just try out different models) for how the difficulty of the play depended on the various attributes of the ball, so you could combine all of the different hits coherently. Daunting.
[12] And probably not worth the time and money and needed. In the end, it just might be that the most accurate, efficient, and cost effective way to evaluate defense is via observation.
[11] Field F/X
Only problem is that the data is likely to remain accessible only to MLB teams and maybe a couple of the largest data crunching organizations.
Also, the dataset generated is freaking huge, something around a terabyte from every game, IIRC.
Great post. I hope this gets out into the world, so to speak.
Thanks a lot for writing this. I feel like I gained a full unit of WAR just from reading it.
This whole debate, believe it or not, reminds me a lot of the debate teachers have about how to teach kids to read. When I entered kindergarten thirty-six years ago, reading was taught exclusively through phonics. If kids could sound out words, they would become readers. By the time I became a teacher, people realized that while some kids learned fine that way (we'll call them the Garveys) others (the Buckners) did not. So everything was turned upside down. Phonics were thrown out, and the Whole Reading movement took over. The idea was that if kids were exposed to lots of good text through read alouds and independent reading, they would magically learn to read, as if by osmosis. Lots of kids actually learned this way -- my bet is that some of your own kids have magically learned to read while sitting in your lap in front of Dr. Seuss. But guess what? While the Buckners liked Whole Reading, all the Garveys fell by the wayside. It took a long, long time for people to realize that what we actually need is BOTH approaches at the same time.
And so it seems to be with defensive metrics. I think the moneyballation of player analysis has been a great thing, especially for the hitters, but I still believe that there are some aspects of defense that have to be measured with the eyes. Think back two years before the installation of this current outfield. Yankee outfielders were old and plodding. I can't tell you how many times I watched a Yankee hitter launch a rocket into the gap only to have it run down by a speedy opposing outfielder, and each time I'd be pissed off for two reasons: one, an out was made; two, I knew the Yankee outfield wouldn't have been able to make the play. I didn't need defensive metrics back then to tell me the Yankee outfield was a defensive liability, just like I don't need them now to tell me the opposite.
I'm sure this information is out there (and I know this idea appears numerous times earlier in this thread), but it seems like it would make more sense to use defensive stats for an entire outfield. You can debate the relative importance of Granderson vs. Gardner, but what matters most, especially in terms of wins and losses, is the Yankee outfield is light years ahead of where it was two or three years ago.
Oh, and for all the voters who are reading, please give Brett Gardner the Gold Glove. Please.
I look at it as just another way to prevent a deserving Yankee from consideration in winning an award.
Remember 2009 and how 3 players on the same team cancel each other out? Not in Beantown it doesn't.
Granderson's year is completely exposing the fielding flaws in WAR. I watch every game, and I think I might recognize a bad CF if I saw one, in errors, or at least misplayed balls.
"In UZR, when a ball is caught and turned into an out by one fielder, no other fielder gets docked any runs. This helps to minimize the effects of 'ball-hogging.'"
"Of course, with this method, a ball hogger will get slightly more credit than he deserves, but as long as his ball-hogging is done on easy fly balls, he isn’t going to get much credit anyway. For example, if a certain type of ball and location is caught 90% of the time, whoever catches it is only going to receive .1 (1 minus .9) times .83, or .083 runs in credit."
To be a -9, Granderson is not getting to some balls that some CF's get to, and they are landing for hits.
One major problem with UZR in WAR is simply sample size. Granderson could be seeing a disproportionately high number of tough plays that are just outside of his range. That kind of thing isn't going to even out over the course of one season.
Another issue is positioning. UZR makes no account for where the fielder starts. If you're a coach, the effectiveness of your outfield positioning strategy will be determined not by a player's individual UZR, but by the combined UZR of the outfield. Of course if Granderson is being positioned differently, you would think it would allow him to make plays other centerfielders cannot to balance out the plays he isn't making that other centerfielders do make (over a large enough sample size, of course).
The biggest problem of all with UZR is simply the measurement error. You need a human being to make a judgment as to the nature of the ball being hit. There are various categories of fly balls, and of course the ball needs to be assigned to the appropriate zone on the field.
The measurement error and positioning issue (though, in theory those should be a wash) is really what field f/x will solve.
[18] Thanks for that information Steve. I would like to study which balls are falling in against this outfield defense that are being caught by other teams.