Baseball-style of analysis hits NHL

[LadyStanley]

http://slam.canoe.ca/Slam/Hockey/News/2006/10/05/1957407-cp.html

Click to read more...

 

[Ted Hoffman]

As much as people like to tout the Red Sox winning the 2004 World Series as evidence that the system can work, it must also be pointed out that Boston spent something like $127 million that year, 2nd only behind the Yankees ... and the Sox had to rally from a 3-0 deficit to get out of the ALCS. So what was the bigger driver in winning the title - was it Bill James' idea, the fact that they spent $127 million, or just pure luck that Mariano Rivera blew saves in back-to-back games with the pennant within grasp?

That said, statistical analysis may provide a significant advantage in a capped environment where getting the most bang for your buck is even more important since spending is limited. This has been going on in the NHL on a small scale with a few teams for a couple years, and it's a question of finding something that gives a team an advantage that no one else knows about.

Whoever comes up with the key to finding that advantage is going to be a very sought-after (and very well-compensated) person. IMO it'll still never replace watching a player at a handful of games to make sure the results are going to be correct, but it can narrow the field of players a team wants to consider up front.

 

[Dr Love]

Whoever comes up with the key to finding that advantage is going to be a very sought-after (and very well-compensated) person. IMO it'll still never replace watching a player at a handful of games to make sure the results are going to be correct, but it can narrow the field of players a team wants to consider up front.

This is a reply to this line of thinking, not to you directly.

The people who run baseball teams with stats weighing in heavily do not ignore scouting. In the "scouts vs stats" debate, the only side that thinks the other side is one dimensional is the scouts side. I'm not sure if you're saying that looking at stats will never replace watching with your own eyes or that both should be taken hand in hand, I think it's the latter, in which case I totally agree with you. IMO, you can't have one without the other.

Vitriolic debates on the merits of traditional scouting versus performance scouting (as quantitative analysis is often called) ensued, boiling over after the release of Michael Lewis' "Moneyball" in 2003. The controversial book examined how the Athletics bucked convention to subvert their economic shortcomings and win. It rubbed many the wrong way for implying that the traditional methods used by most teams were flawed.

Actually it stated that it was flawed. And most people either don't know or ignore that there is a whole chapter on scouting, or that a player's "makeup" is extremely important (at least, in drafting) to them ("put a Milo on him").


[SIZE=-0]
[/SIZE]

 

[Ted Hoffman]

I'm not sure if you're saying that looking at stats will never replace watching with your own eyes or that both should be taken hand in hand, I think it's the latter, in which case I totally agree with you. IMO, you can't have one without the other.

Yes, I am saying the latter. And I'll agree with another point you raised - there are some that think there can be a statistical analysis that will be the end-all, be-all answer ... and if no one has come up with something to accurately predict lottery numbers, you can bet there's not going to be a model that's 100% accurate.

But if you can narrow down a group of 100 players to even 20, you're saving coaches, GM's, and scouts a whole lot of time and effort. IMO that's the best end result one can hope for in all of this.

Actually it stated that it was flawed.

I've had discussions on this with a few people - certainly with goalies it's difficult if not impossible to predict through statistics alone who is going to be great. (See: Roy, Patrick - who had miserable stats in the Q.) It might be possible to do something with the skaters, but the number of variables to consider is just incredible when you start writing them down, not to mention the volume of data one would need to conclusively rule it in or out. And that's just the start of it.

 

[Tokyo Bucks]

Too bad most of the research is being done behind closed doors by teams trying to get an edge on eachother. Sabermetrics started off very openly with people tossing various ideas out to eachother in public forums (or in letters to Bill James).

See: Roy, Patrick - who had miserable stats in the Q

Factors such as the team he was on and what the scoring tendency of the Q were in those years have to be taken into account. Even when normalized though, they'd probably still look pretty bad.

Anyways, scouts value qualitative analysis more than stats in many cases precisely because we're using the wrong stats to evaluate players. Rob Brown in the article is a good example, and in baseball the traditional reliance on AVG, HR, RBI, W, ERA, SV means that many players are not valued properly (OBP and SLG portray a much more accurate picture for hitters, there are more esoteric stats that have better accuracy). So, there's a good chance that G-A-P-+/- and W-L-GAA-Sv% can be sometimes misleading. Once better statistics are found to better reflect actual player performance and contribution to the team, then scouts' qualitative evaluation and the accompanying stats might start falling into line more often.

It's true that hockey's a fluid sport, but so is basketball and they're way ahead of hockey in terms of statistical analysis. Just the thing you need to find undervalued players (there's a reason why despite the low payroll - effectively a cap - Oakland keeps on getting into the playoffs every year on strong 2nd halves, the playoffs in baseball are too much of a crapshoot dependent largely on starting pitching and luck, hockey's just as reliant on hot goalies but less luck dependent than baseball, when comparing best of 7 series for both sports).

 

[AdmiralPred]

*CUT*
It's true that hockey's a fluid sport, but so is basketball and they're way ahead of hockey in terms of statistical analysis. Just the thing you need to find undervalued players (there's a reason why despite the low payroll - effectively a cap - Oakland keeps on getting into the playoffs every year on strong 2nd halves, the playoffs in baseball are too much of a crapshoot dependent largely on starting pitching and luck, hockey's just as reliant on hot goalies but less luck dependent than baseball, when comparing best of 7 series for both sports).

I think the 'fluidness' of hockey vs. basketball are very different. I have been one to argue that basketball would be easier to apply sabremetrics to than football despite the 'fluid' action. Basketball can still be broken down by possession and even further by who has the ball and what they did while they had it - based off of newer play-by-play charts. Plus there are 5 players to track with a common goal - even though the have 5 separate responsibilities.

Using conventional stats and play-by-play sheets in hockey, it would be nearly impossible to apply the Bill James approach to valuing players. You'd have factors like digging the puck out of the corners, which players were in the corner at the time and who eventually 'won' the battle in the corner. Scrums in front of the net are another example.

 

[Ted Hoffman]

Here's a fundamental problem:

In baseball, you can find out how a batter performs when the pitch count is 2-2, if the bases are loaded, if his team was trailing by fewer than 3 runs in the 7th, 8th, and 9th innings, or when a left-handed pitcher throws a pitch that crosses the inside part of the lower 3rd of the strike zone.

Someone tell me what Roberto Luongo's save percentage was when his team was shorthanded, in the 3rd period of games, or when guys shoot stick side high. You can't - because any data on those stats either doesn't exist (especially in the last example), or has to be painstakingly culled from game summaries because it's not immediately available anywhere.

Among other reasons, that is why statistical analysis in hockey is so much more difficult than in baseball or even basketball or football.

 

[Bear of Bad News]

As someone who's been a big fan of Alan's work over the past several years, I just wanted to say that it's good to see him finally get some recognition for his hard work.

If anyone can take the ball and run with this under the public eye, it's him.

 

[19nazzy]

So, there's a good chance that G-A-P-+/- and W-L-GAA-Sv% can be sometimes misleading.

I think that GAA can be very misleading. Someone needs to find a way to find stats that really show a goalie's true play. Most of their stats are really team stats.

 

[Injektilo]

I think alot of this stuff has probably been going on for a while, but the public hasn't seen much mention of it until recently, if ever.


If you wanna check out some interesting stats analysis, read the HF Oilers board Library, specificially posts by Igor and Mudcrutch. They've stopped posting here for the most part, but they run blogs which are probably even better.

 

[GSC2k2*]

Here's a fundamental problem:

In baseball, you can find out how a batter performs when the pitch count is 2-2, if the bases are loaded, if his team was trailing by fewer than 3 runs in the 7th, 8th, and 9th innings, or when a left-handed pitcher throws a pitch that crosses the inside part of the lower 3rd of the strike zone.

Someone tell me what Roberto Luongo's save percentage was when his team was shorthanded, in the 3rd period of games, or when guys shoot stick side high. You can't - because any data on those stats either doesn't exist (especially in the last example), or has to be painstakingly culled from game summaries because it's not immediately available anywhere.

Among other reasons, that is why statistical analysis in hockey is so much more difficult than in baseball or even basketball or football.

Before the development of sabermetrics in MLB, you could make the exact same observation about baseball. It seems foreign now, but once upon a time peope did not know all the things that they all track in baseball. Games were not tracked pitch-by-pitch.

With appropriate effort, anything can be tracked in hockey.

 

[GSC2k2*]

Using conventional stats and play-by-play sheets in hockey, it would be nearly impossible to apply the Bill James approach to valuing players. You'd have factors like digging the puck out of the corners, which players were in the corner at the time and who eventually 'won' the battle in the corner. Scrums in front of the net are another example.

One of the ultimate benefits of sabermetrics is that one actually can determine what is useful to track and what is not.

For example, common baseball wisdom held that the sacrifice bunt was a good play. It has been proven to be relatively useless.

The factors that you list above come with the embedded assumption that those things help teams win games. I would be interested in finding out which traditional hockey notions would go out the window once the game is rigourously analyzed. I for one would be willing to consider the possibility that all of the emphasis of fans on a player's "physical play" or rugged qualities (as typified by "scrums" in front of the net) is overvalued in winning hockey games.

 

[Injektilo]

As someone who's been a big fan of Alan's work over the past several years, I just wanted to say that it's good to see him finally get some recognition for his hard work.

If anyone can take the ball and run with this under the public eye, it's him.

The article in teh first post doesn't actually mention Alan Ryder (who I assume you mean) but this is probably the article you're talking about :

http://www.globesports.com/servlet/story/RTGAM.20061003.wryder03/GSStory/GlobeSports/home

 

[Injektilo]

One of the ultimate benefits of sabermetrics is that one actually can determine what is useful to track and what is not.

For example, common baseball wisdom held that the sacrifice bunt was a good play. It has been proven to be relatively useless.

The factors that you list above come with the embedded assumption that those things help teams win games. I would be interested in finding out which traditional hockey notions would go out the window once the game is rigourously analyzed. I for one would be willing to consider the possibility that all of the emphasis of fans on a player's "physical play" or rugged qualities (as typified by "scrums" in front of the net) is overvalued in winning hockey games.

I don't think it's really possible to analyize the stuff like "battles won in the corner" and whatnot, and like you, i'm not sure it's all that useful.

I doubt anyone will ever be able to tell how good a game a player has had simply by looking at the stats for that one game, it's more over an entire season that you get a big enough sample size to analyize how much a certain player contributes to his team winning and losing games.

 

[Bear of Bad News]

The article in teh first post doesn't actually mention Alan Ryder (who I assume you mean) but this is probably the article you're talking about :

http://www.globesports.com/servlet/story/RTGAM.20061003.wryder03/GSStory/GlobeSports/home

That's the one!

I didn't check the SLAM! link - I assumed it was a repackaging of the same article.

Regardless, Alan does great work.

 

[Ted Hoffman]

Before the development of sabermetrics in MLB, you could make the exact same observation about baseball. It seems foreign now, but once upon a time peope did not know all the things that they all track in baseball. Games were not tracked pitch-by-pitch.

Here's the difference though: one could go through the game data and find most of that stuff out. Even going back to the 60s, we can see what Sandy Koufax did in the 7th inning with runners on base and a 1-run lead because the official scorebooks were preserved. We could go back to the 20s and before and find this stuff out on whoever we want.

The game data on the NHL is so incomplete in that regards, it's difficult if not impossible to find something as "simple" as how many PP shots a team had in a game up until the last 7 years or so. The potential is there to track shots, saves, and so forth, but whether or not it's being done (my bet: if it is, the NHL isn't officially part of it) is another story. IIRC, the NHL can't even say that it has complete game sheets from about 1955 on back.

 

[william_adams]

The potential is there to track shots, saves, and so forth, but whether or not it's being done (my bet: if it is, the NHL isn't officially part of it) is another story. IIRC, the NHL can't even say that it has complete game sheets from about 1955 on back.

You're likely right that it is not being done by the league, but I would be shocked if there aren't some teams out there that DO track some of this stuff and DO use it, but keep it to themselves for obvious reasons...

Not sure where he got the data, but despotic came up with some fascinating stats about penalties in this thread http://hfboards.com/showthread.php?t=282910 (he also posted in at least some individual teams pages for individual players) and these should definitely used to evaluate the cost of penalties...

 

[mudcrutch79]

The game data on the NHL is so incomplete in that regards, it's difficult if not impossible to find something as "simple" as how many PP shots a team had in a game up until the last 7 years or so. The potential is there to track shots, saves, and so forth, but whether or not it's being done (my bet: if it is, the NHL isn't officially part of it) is another story. IIRC, the NHL can't even say that it has complete game sheets from about 1955 on back.

The good stats stuff online isn't being done by the NHL, it's being done by obsessive compulsives. You can go to vhockey.blogspot.com and find out what players on another team a given Oiler was playing on a certain night. My site has a number of stats weighting shots based on where they're taken and adjusting for goaltender save percentage based on that.

Funny that you mentioned goalies as being hard to evaluate. First of all, serious baseball analysts don't really pay much attention to the situational aspect of hitting - it's been impossible to show that the situation matters. The same should be true with goaltending, although it hasn't been tested. Personally, I ignore wins and GAA; I just care about save percentage. With better data collection, you'd be able to come up with very good information as to the difficulty of the shots faced. Goalies are easy - if I was working for an NHL team and someone threw some money at me, I could easily come up with something that would give you fantastic information breaking them down. It's the skaters, and particularly defencemen who are difficult.

 

[Dr Love]

First of all, serious baseball analysts don't really pay much attention to the situational aspect of hitting - it's been impossible to show that the situation matters.

Yes, they do. There is Win Probability for batters and Leverage Index for pitchers.

 

[mudcrutch79]

Yes, they do. There is Win Probability for batters and Leverage Index for pitchers.

The implication of the post that I was responding to is that the performance would change in a given situation in terms of score, and that the change would be relevant. I disagree, outside of the obvious that comparing ES to PP or anything like that is stupid.

LI is valuable because it offers an explanation as to the value of a run in certain situation. WPA shows the same and the impact that the players have had on the game. They aren't measures of ability, only of the importance of the event that took place as a result of the ability.