I think size isn't a problem as far as an individual player is concerned but can be a problem if you're building for a real world NHL roster.
Let's say you're building a real defense corps. You'd take an Adam Fox over a Colton Parakyo every day of the week. And if could take Quinn Hughes second, you'd take him over over an Erik Cernak. But do you keep stocking with Torey Krug, Samuel Girard, Jared Spurgeon, etc. moving down the roster? Or do you start circling back to Cernak to try and diversify the group to play in different situations?
I think the Leafs basically have that dilemma. Instead of having a couple of 6'4" horses who can play in the top 4 who can give you big minutes, two way play, long wingspan defense, they suddenly need to pad out the bottom of the roster with Lyubushkin and Holl who can give them some of those attributes.
Well we are not exactly stacking up on 5'9" guys here though, nor are we talking about guys who are offensive defensemen. Koster is really the only guy we have who is that small.
Ideally, you have a bunch of guys who know how to do many different things. We have guys who are only average-sized, but they know how to defend in tight, they throw hits, they defend using their bodies and their feet... Effectively, they can do a lot of different things. Sandin and Niemela, especially, still need some more meat on them but they are also still young. If they can both get to 6'0", 200lbs, I don't think we have to be overly concerned with them being able to handle guys out front, even if they are 6'4"+. I am fairly sure the whole "leverage" analysis thing was thrown around here a few years ago, and how that matters a lot more than pure size and strength... Which is also why a guy like Rasanen was getting destroyed in battles despite being 6'7", 230 lbs going up against half a foot shorter and 30lbs+ lighter than him.
I am more concerned if we had a bunch of one type of defensemen on our team... But I have seen 6'0" defensive defensemen who could shut down pretty much anyone and I have seen 6'4" large offensive defensemen who couldn't defend for their life.