From Raffo: I really like this fanshot by Erik O and the ensuing thread, but because it's a fanshot I can't bump it to the cover. So, I'm forcing it at the top of the river.
Erik O: So this is way, way after the fact, but I thought it was interesting. Back when the Grizzlies hired Hollinger, David Berri was interviewed by the Grizzlies website, 3 Shades of Blue about what he thinks about Hollinger. In the advanced stats community, it's no secret Berri thinks PER is kind of a joke, so he was a bold (and great) choice for the interview.
Berri begins by saying that Hollinger's metric is based off of all kinds of arbitrary weights he uses that he thinks make sense, which makes it flawed. I've complained about PER in the same way, that it uses certain weights that are completely subjective, but then the formula is used objectively. But then if you scroll down into the comments, Neil Paine (part of the basketball-reference.com team) calls out Berri and his Wins Produced formula for also having arbitrary weights, and makes subjective choices, which it absolutely does. For example, Berri says the value of a point is 0.033 wins, and the cost of a possession is 0.034 wins. Aren't these arbitrary weights as well?
Berri goes on to ding PER for not having any correlation with wins, point out that it isn't good for guessing team records--of course Hollinger has admitted as much before, but I guess Berri didn't see that. In the comments section again, Neil Paine explains why Wins Produced (and Win Shares too, interstingly made by Justin Kubatko of basketball-reference.com, which tells us Neil Paine is at least being unbiased) are so highly correlateed with wins, and why it doesn't mean they are better metrics necessarily. In a nutshell, Neil says what Steve has said tons of times, that these metrics are so highly correlated with actual wins because they have a strong basis in offensive/defensive ratings (which are derived from points scored/allowed, which is always going to be very highly correlated with wins). Steve has repeatedly said that you can get high correlation just looking at team points scored and points allowed (or even easier, Margin of Victory). Surprise, surprise, the teams with a high Margin of Victory have the best records, with the obvious exception being the Lakers this year. Wins Produced also randomly has exceptions too, which is why it's only 95% correlated. As Steve, again, has pointed out to me (and John R, Michael White, SilverClip etc. in the discussions I'm remembering back to), while there is a high correlation between the "wins" formulas and actual wins, there's very little predictability power behind those formulas. That is, you can't use them to predict how a team will do very well. Isn't that the whole point? Why correlate with wins if you can't even use it to predict wins?
Neil Paine's ultimate conclusion is that you shouldn't rely on any of the advanced metrics solely, because they are imperfect for predicting what's going to happen. No one, not a single metric, had any idea that Andray Blatche would be good this year. I've always thought he was just a crappy version of Marreese Speights, but this year he's playing like a borderline all star. What's funny is that for all their arguing, in the interview Berri also acknowledges that you can't just use stats to run a team, even if it’s wins produced. You have to look at the coach’s system, the type of players you have and what roles they play, the chemistry between players, and lots of other non-stats factors. This is the first time I’ve ever seen Berri say this out loud, and it was kind of nice to see he’s not a total nutjob. Hollinger never really said this, which has always bugged me, but I think Berri's right that Hollinger won't run the Grizzlies' scouting just based on PER.
Anyway, super nerdy, but still interesting.