TBN: How the Sabres use analytics: Q&A with Jason Nightingale, assistant director of scouting

[Jim Bob]

It's cool that they actually let him talk!

Click to read more...

 

[joshjull]

 

[Jim Bob]

A: When I was hired seven years ago that was one of the mandates, to build an infrastructure here. Obviously it’s something where you need to have more than one person on it. We’ve been really lucky to have some really great engineering schools in the area and we’ve had up to five engineering students that have worked with us to build our database tools, some of our analytic tools, some of our reporting tools and I’ve also had an assistant (Mae Sementilli) that’s going back to get her Ph.D., but she’s worked here for the past five years. Very talented engineer. I think we’re looking at staffing right now, but what I would say that our priority is on technical training and it’s because of the volume of data. And the ability to make sure we’re not dealing with noise. We’re capturing strong signals relative to player evaluation and we’re passing that along to the coaches and management.

People would say that the Sabres weren't investing in analytics because they only had Nightingale listed on the website.

But, it would appear that they had a larger team than that.

https://www.linkedin.com/in/mae-sementilli-67061a126/

Hockey Data Analytics Graduate Assistant

Company Name
Buffalo Sabres
Dates EmployedJun 2014 – Present

Employment Duration6 yrs 2 mos
- Creating and managing databases using Matlab, Python, MySQL
- Data mining in Matlab and Python
- Developing and managing applications written in Python

 

[joshjull]

 

[joshjull]

Q: Do you expect the organization to employ more data engineers and analysts now that you’re shifting to this scouting model?
A: When I was hired seven years ago that was one of the mandates, to build an infrastructure here. Obviously it’s something where you need to have more than one person on it. We’ve been really lucky to have some really great engineering schools in the area and we’ve had up to five engineering students that have worked with us to build our database tools, some of our analytic tools, some of our reporting tools and I’ve also had an assistant (Mae Sementilli) that’s going back to get her Ph.D., but she’s worked here for the past five years. Very talented engineer. I think we’re looking at staffing right now, but what I would say that our priority is on technical training and it’s because of the volume of data. And the ability to make sure we’re not dealing with noise. We’re capturing strong signals relative to player evaluation and we’re passing that along to the coaches and management

 

[Gabrielor]

Love this stuff.

I'm just super antsy to see moves made where it's applied. Until then, the beautiful words will still just be words to me.

 

[struckbyaparkedcar]

Nightingale having a seat at the table is going to keep my interest for the offseason even if they don’t make the Gillis/Dudley/etc veteran executive hire that would placate the greater fanbase.

 

[Doug Prishpreed]

So besides JN, the only people working on data modeling where UB undergrad students? Probably for $15/hr. I was a UB engineering student myself once upon a time, I can't imagine being all that helpful at that age.

 

[Fezzy126]

So besides JN, the only people working on data modeling where UB undergrad students? Probably for $15/hr. I was a UB engineering student myself once upon a time, I can't imagine being all that helpful at that age.

Database creation and analysis is often by interns or fresh college grads at many companies. There's nothing special about the work, but it is extremely monotonous and often low-paying. The youth in those positions also can open up a certain degree of creativity or new visions that people with more experience lack.

The key point here is how that data analysis is integrated within the organization, and as @struckbyaparkedcar mentioned, with JN having a seat at the big boy table maybe that changes their approach at several levels of the organization.

 

[Fezzy126]

When you read this and really start to appreciate the thought process here... and then immediately think about how they've deployed Risto

 

[struckbyaparkedcar]

When you read this and really start to appreciate the thought process here... and then immediately think about how they've deployed Risto

Feels like it’s not exactly a coincidence that this is the first time we’re hearing from Nightingale.

 

[Doug Prishpreed]

Feels like it’s not exactly a coincidence that this is the first time we’re hearing from Nightingale.

He said he was given a larger role in 2017...nothing has improved since then - on the pro side, at least. Maybe they've only been applying his work to the amateur scouting?

 

[Doug Prishpreed]

Not everyone who has a role in analytics is actually good at it. It's great news that they're giving a larger role to analytics staff, but is he actually the right person? I'm skeptical. I've seen zero proof that he's good at this.

He doesn't just have to be competent -- he has to be better than the people performing the same roles on all of the other teams.

 

[SackTastic]

Nothing there about how they have used analytics, it's all about how they plan to use analytics.

 

[Doug Prishpreed]

Nothing there about how they have used analytics, it's all about how they plan to use analytics.

Huh? He talks plenty about how they have used analytics in the past. The main reason they focus on the future more here is because there's a big change in player tracking and data coming next season, so there would've been changes in that regard even if Botts was still here.

 

[SackTastic]

Huh? He talks plenty about how they have used analytics in the past. The main reason they focus on the future more here is because there's a big change in player tracking and data coming next season, so there would've been changes in that regard even if Botts was still here.

Not really.

Over the next six years, Nightingale built an infrastructure that provided Sabres management and coaches with data that could drive decision-making.

A: I think the amount of data – someone used the analogy of watching a game on black-and-white versus color. For example, typically what we had in the past is just shot attempts. Many of the initial data points were driven by shots.

Those are the only references to how data has been used in the past, and there is nothing there.

 

[struckbyaparkedcar]

He said he was given a larger role in 2017...nothing has improved since then - on the pro side, at least. Maybe they've only been applying his work to the amateur scouting?

My feel is that he might have been able to scale his department, but his output only factored into decision making when it confirmed front office/coaching bias.

 

[Buffaloed]

How the Sabres use analytics: Q&A with Jason Nightingale, assistant director of scouting
This is new for the Sabres
Q: How are you taking advantage of the additional time to prepare for the NHL draft?
A: This a unique time. Obviously, there are a few games going on. If you think about the amateur draft, we have more access to video and higher-quality video than we’ve ever had. We have access to data, to things that will tell us about player tendencies, than we’ve ever had and we do have time now to really implement a data-driven player evaluation process, specifically over the last month I’ve really enjoyed working with Jeremiah and our scouting staff. They’ve been working hard in revisiting the 2020 draft and trying to incorporate more information, to be open-minded when it comes to evaluating players and I think also to develop a scout calendar and a process that can be a hybrid of both the data and the experts’ opinions, the scouts.

Botterill, Housley, and Krueger have been anything but open-minded when it comes to analytics. I think Adams is trying to devise a structure where the departments work closely together. You could tell Housley was frustrated at one point when he said something to the effect, "it's not my fault, I just use what Jason gives me" and I can't believe Krueger had any say in Dalton Smith.

 

[Doug Prishpreed]

Not really.





Those are the only references to how data has been used in the past, and there is nothing there.

"A: When I was hired seven years ago that was one of the mandates, to build an infrastructure here. Obviously it’s something where you need to have more than one person on it. We’ve been really lucky to have some really great engineering schools in the area and we’ve had up to five engineering students that have worked with us to build our database tools, some of our analytic tools, some of our reporting tools and I’ve also had an assistant (Mae Sementilli) that’s going back to get her Ph.D., but she’s worked here for the past five years. Very talented engineer. I think we’re looking at staffing right now, but what I would say that our priority is on technical training and it’s because of the volume of data. And the ability to make sure we’re not dealing with noise. We’re capturing strong signals relative to player evaluation and we’re passing that along to the coaches and management."

 

[Buffaloed]

"A: When I was hired seven years ago that was one of the mandates, to build an infrastructure here. Obviously it’s something where you need to have more than one person on it. We’ve been really lucky to have some really great engineering schools in the area and we’ve had up to five engineering students that have worked with us to build our database tools, some of our analytic tools, some of our reporting tools and I’ve also had an assistant (Mae Sementilli) that’s going back to get her Ph.D., but she’s worked here for the past five years. Very talented engineer. I think we’re looking at staffing right now, but what I would say that our priority is on technical training and it’s because of the volume of data. And the ability to make sure we’re not dealing with noise. We’re capturing strong signals relative to player evaluation and we’re passing that along to the coaches and management."

He's talking about how they built it, not if or how it was used by the coach or GM. It's the old "you can lead a horse to water" bit.

 

[joshjull]

When you read this and really start to appreciate the thought process here... and then immediately think about how they've deployed Risto

It goes back to how to properly contextualize any player’s performance. Using any stand alone numbers as if they tell much of a story has always been flawed. (I'm referring to us as fans breaking things down not the team. Who I would hope uses the broader context when breaking things down).

 

[jc17]

Regarding player tracking and the information gathered there, some argue that could be what got the coyotes into trouble. That the focus became more on the micro, than macro and lost value because there is a lot more noise. That said, the following quote is reassuring. As obvious as it sounds, its so important to not get lost in data for the sake of data but using what has predictive value.

To me, it’s imperative on the analyst to earn trust with the hockey staff and to be able to put forward useful information that is predictive of success and repeatable.

 

[toomuchsauce]

 

[struckbyaparkedcar]

View attachment 356636

“Data and data accessories.”

 

[struckbyaparkedcar]

Regarding player tracking and the information gathered there, some argue that could be what got the coyotes into trouble. That the focus became more on the micro, than macro and lost value because there is a lot more noise. That said, the following quote is reassuring. As obvious as it sounds, its so important to not get lost in data for the sake of data but using what has predictive value.

Which of their acquisitions were Montour style micro-stats adds? Most of their mistakes had more to do with misreading aging curves for players that had good underlying numbers, and Chayka’s initial mandate to build around fraudulent assets in OEL/Strome.

It’s frustrating how Arizona has become this case study on both analytics and micro vs macro stats, when their issues have far more to do with an incompatible owner/executive relationship that would torpedo most approaches.