Careful what you say, Ted Nolan is every bit the coach that is capable of pulling these guys to the 5th to 10th over all draft position.
Corsi is an advanced stat useful for evaluating this position. Corsi is shots + missed shots + blocked shots. CF% is (Team Corsi)/(Team Corsi + Opponent Corsi), and is useful in determining whether the team or the opponent had more opportunities to score in the game. A CF% of 50% means that teams had equal opportunities to score. A CF% less than 50% means the opponent had more opportunities to score, while a CF% of greater than 50% means the team had a greater opportunity to score.
Corsi is a useful stat to look at because Corsi correlates with goals, which correlate with wins (points). While it can be argued that wins and points are all that matter, tracking wins and points are not a reliable method for evaluating team performance. A team can play well and lose, or play poorly and win, and because there is a relatively small sample size of events (games, 82 in a season), the noise factors can not be filtered out. Goals and goal differential are another possible metric, but again teams can score many goal on a few shots and win, or score few goals on many shots and lose. so goal differential is also not really a great stat to determine how well a team is playing. Corsi has been identified as a stat that best evaluates a teams performance. Assuming similar goalie save %'s, a team with a higher CF% is expected to score more often and thus win more often than a team with a lower CF% over time, even if the relatively more rare and smaller sample sized stats of actual goals scored and wins/points do not correlate in the short term.
So what can this tell us about Rolston vs. Nolan?
Well, in all 5 on 5 situations regardless of score, the Sabres in 20 games under Rolston had a CF% of 42.31% (StDev of 7.22%, 1664 total corsi events). This means that if a game had 100 total corsi events 5 on 5, the sabres would have on average attempted 42 shots to their opponents' 58 shots.
In 62 games under Nolan, the Sabres had a CF% of 43.34% (StDev of 7.66%, 5328 total corsi events). This means that if a game had 100 total corsi events 5 on 5, the sabres would have on average attempted 43 shots to their opponents' 57 shots.
What this tells us is that the "Nolan effect", if it did indeed exist at all last year, accounted for all of 1 extra shot attempt per 100 total attempted shots in the game (both teams). The significance of this increase is... not much. The Sabres were 29th in CF% 5 on 5 last season with a 43.1% for the year. Only Toronto was worse with a 42.8%. The oilers, in 28th, were at 44.3%. It's possible you could argue that the change from Rolston to Nolan improved our CF% 5 on 5 by one position from dead last to just ahead of Toronto, but that is all.
It's also possible that there really was no difference at all between the team's perfomance under Rolston and Nolan - that the small change in Corsi is simply a matter of noise, or other factors that have nothing to do with the coach. Below are some visual representations of the data I'm looking at.
Here is a plot of the game by game 5 on 5 corsi. It's very hard to argue from this data that Nolan's team really performed any better than Rolston's. Perhaps there is a bump compared to Rolston from games 20-40, but there also looks like there is a dip compared to Rolston from games 40-60. And I wouldn't say there is any difference between games 1-20 under Rolston and 60-82 under Nolan. But a lot external factors went into the corsi that are not taken account for.
Here is a plot for the statistitians. This is the normal distribution for Rolston's CF% compared to Nolan's CF%. Now, I didn't exactly run any statistical tests on the data, but just by looking at the plot I would say any statistical test would find that there is no statistical evidence that Nolan's CF% was any different than Rolston's CF%.
The last plot I have is my attempt and adjusting for 1 external factor: quality of opponent. Remember how earlier in this post I explained how teams with higher CF% were expected to be better than teams with lower CF% over time? Well, that means I can use a teams total CF% at the end of the season as a stand-in for quality of opponent. This plot, similar to the first plot, is a plot of all 82 games CF%. However, instead of simply being plotted sequentially, I've plotted the Sabres game by game CF% vs. their opponents' season ending CF%. Data points on the right side of the chart were therefore against better teams, and data points on the left side of the chart were against easier teams. In other words, you would expect the Sabre's CF% to be worse against teams with higher CF%, and better against teams with worse CF%. This chart confirms that expectation.
It also shows that Rolston's opponents on average had a higher CF% than Nolan's opponents, as the blue dots are on average more to the right side of the plot than the red dots. Where Rolston's and Nolan's data points overlap against opponents with similar CF% (48%-54% on the X-axis), there isn't any noticeable difference in the teams performance (game CF%, y-axis). Therefore, from this chart it is easy to see that any improvement in the Sabres CF% under Nolan can pretty much be entirely attributed to the fact that on average, the Sabres played teams with a worse CF% under Nolan than under Rolston. In other words, Rolston had the tougher schedule, and there was really no improvement at all in on-ice performance that can be directly contributed to Nolan being the better coach.
I also took a look at all the same data but for 5 on 5 close. 5 on 5 close is defined as being tied or ahead or behind by 1 in the 1st and 2nd period, and tied only in the 3rd period. I looked at this data to be fair to Nolan. Based on TOI, Rolston's team was only 5 on 5 CLOSE in 58% of all 5 on 5 situations, while Nolan's team was 5 on 5 CLOSE in 67% of all 5 on 5 situations. Well, when games were not close last year that typically meant the sabres were behind, and teams that are behind typically see a boost in CF% as they press and opponents sit back to protect. the 9% difference in time spent CLOSE could mean that Rolston's CF% 5 on 5 all situations got a bit more of a boost than Nolan's CF% 5 on 5 all situations due to spending relatively more time behind. As shown above, this doesn't really say anything about the coach, but more to the fact that the Sabre's had a tougher schedule under Rolston and would be expected to be behind more often.
In 5 on 5 close situations, Rolston had a 39.02% compared to Nolan's 41.64%. Both coaches CF% 5 on 5 close was lower than their 5 on 5 all situations score, meaning both coaches must have seen an improvement in 5 on 5 all CF% due to getting more opportunities after falling behind. However, Rolston's CF% 5 on 5 close saw more of a drop than Nolan's as expected. In terms of shot attempts, assuming 60 corsi events in 5 on 5 close situations (remember 5 on 5 close is about 60% as many attempts as 5 on 5 all for both coaches), Rolston would expect 23.4 shot attempts to their opponents 36.6 shot attempts, while Nolan would expect 25.0 shot attempts to their opponents 35 shot attempts. In other words, the 'Nolan effect', if it existed, would only account for an extra 1.5 shots in 5 on 5 close situations in a game.
Like in all situations, this is not exactly a monstrous effect. The sabres finished dead last in the league in CF% 5 on 5 close with a season ending rating of 41.1%. The Leafs finished next to last with a rating of 42.1%, so any improvement, even if directly attributed to Nolan, does not even pull us out of last place in 5 on 5 close situations.
Here are the same 3 plots as above, except for 5 on 5 CLOSE situations. I don't have to say anything about them because I would just be repeating myself:
There is just nothing in the data that shows there is a Nolan effect. There is no statistical evidence that shows a Nolan coached team, all else being equal, will perform better just because Nolan. The improvement that most saw in the team after Nolan took over is just a coincidence that Nolan took over at a time when the schedule became easier.
Look I'm not trying to defend Rolston or bash Nolan. I hated Rolston as a coach and prefer Nolan. There are many things the eye test shows that stats don't that are enough to prove that making the switch from Rolston to Nolan was a vast improvement for the team. But in the context of this thread, and the opinions of many that there is a 'Nolan effect' that can account for an improvement in points that alone could drag us out of the bottom spot in the league, well there is just nothing that shows that this 'Nolan effect' exists at all. The Sabres were as bad a team under Nolan as under Rolston. Sure, Nolan brought some amazing intangibles to the team. Hard work, effort, improvements in Myers, Ennis, and others, etc. But that is just what they seem to have been:
intangible. As in, nice to see, but no effect on the bottom line.
Could the team be better next year? Sure! Perhaps the positive effects of having a competent coach take more time to set in, and eventually we will see an actual improvement in the team that can be attributed to coaching. Perhaps individual players had a poor year last year and will improve. Younger players may improve. New players (free agents, draft picks) may improve the teams overall corsi. Last year the team had a league worst shooting percentage of 5.7%. If that was an outlier, then even if nothing else changes but shooting percentage increases to a still bottom 3rd of the league 7%, then the sabres will score more goals and likely win more games. But also, the sabres had a pretty average team save percentage of 92.2%. With Miller gone, we should expect that to be lower, and if we drop to Islanders goal tending levels (91%) or worse (FLA was last with 89.9%), then we're going to be giving up a lot more goals and losing even more games. (all the stats in this paragraph were in 5 on 5 close situations)
But really, this whole idea that the Sabres will be better because Nolan really just needs to go away. The team may somehow be near .500 or somewhere other than the cellar in January, but it's likely not going to be attributed to Nolan getting more out of the team than other coaches would be able to.
*all stats in this post came from www.extraskater.com
