Preliminary Top 100 Guidelines

[reckoning]

But, if you want to have a top 100 of all time HOH list, then you have to include all players.

Exactly. I thought that was the whole point.

 

[VanIslander]

well, if it is an ALL-TIME draft, how about explicit minimum quotas for particular eras and places

as, i mentioned before, a top-100 list without Bowie and Bobrov just isn't an ALL-time draft

perhaps certain assumptions/biases could be explicitly stated

 

[Transplanted Caper]

Actually Kelly is in my Top-25. Messier will probably make Top-40. I think if a guy can "lose" some points for a few flaws - Frank Mahovlich comes to mind - it should apply to a guy like Messier... And Messier's lows are lower than Mahovlich ones, IMO. Maho was just moody... but he never sent back a team, by his sole presence, at the bottom of the standings.

Mahovlich's moodiness was in large part due to Imlach. I suppose it's not overly relevant as moodiness is moodiness, but I don't think it was ever really the case that Frank was a bad team guy. He got routinely ripped by the Toronto fans and by his coach and dealt with depression.

 

[Transplanted Caper]

As per my post in the original thread, I intend to throw a list together. Have started it off and it really is quite interesting. I find myself jotting down some names compiling a list of 20-25 guys to start and then all of a sudden I realize that I forgot to put Player X in there. Really interesting to compare and contrast styles and to compare longevity with pure skill.

 

[Transplanted Caper]

Where the goalies wind up on this list is going to be interesting. With so many of the elites being really bunched together, it'll be interesting too see if this is reflected in the lists (i.e a string of goalies within 10-12 spots on the list)

 

[noodleboy*]

I've recently moved to a new city and can only access the internet from the library, but I'd like to take part. I don't have the ability to make a list of the top 120, but would I still be able to join the discussion and the later voting rounds?

 

[Hockey Outsider]

Our rationale for the "NHL only" rule was to be upfront about the fact that many people here are less knowledgeable about great European and pre-NHL players. Anyway, this rule is up for debate, so let's look at it another way. Is there anybody who thinks that they would be unable to contribute if we dropped the NHL only restriction?

I've recently moved to a new city and can only access the internet from the library, but I'd like to take part. I don't have the ability to make a list of the top 120, but would I still be able to join the discussion and the later voting rounds?

Ideally everybody should participate as much as possible but there are more important things in life. Participate whenever you have the chance.

 

[Weztex]

Our rationale for the "NHL only" rule was to be upfront about the fact that many people here are less knowledgeable about great European and pre-NHL players. Anyway, this rule is up for debate, so let's look at it another way. Is there anybody who thinks that they would be unable to contribute if we dropped the NHL only restriction?

I personally consider my knowledge of soviet/pre-NHL players at about 5% of my NHL knowledge. I've been building up a NHL-HOF monitor for over a year now and could contibute on a similar level than most of the poster here at the project in his actual form. However, since most people seem to be in favor of including all spheres of hockey, I may just lay back and turn myself into a outside witness since I don't think I would be able to bring a valuable insight.

However, I am not agaisnt expanding the process to every level of hockey. I would sure love to participate but in the end I think the posters should choose what they want to be part of.

 

[Weztex]

In fact, there's probably a way to make both side participate. A way to merge ''NHL lists'' with ''global lists'' into one top 100 using position average instead of a point system.

I'll sleep on this and try to come up with a system that could work.

 

[FissionFire]

In fact, there's probably a way to make both side participate. A way to merge ''NHL lists'' with ''global lists'' into one top 100 using position average instead of a point system.

I'll sleep on this and try to come up with a system that could work.

Which is essentially what the system I posted would be doing. Letting everyone participate in an NHL-only list and then once that was done take selected posters who wanted to create an All-Time list to make one including non-NHL players. Merging lists based on average placing is just horrible. For one, players will be slotted 1 or more places downward on an All-Time list than they would on an equivalent All-NHL list. It almost sounds like you want to use a weighted average but that's not really a great way to do it. Plus come Round 2 for the final voting it's impossible to include people who have little to know knowledge of these players in the process. How can someone accurately rank, say, Lindsay vs. Fetisov vs. Kharlamov vs. Gainey? How could someone who isn't confident enough in their ability to rank them on a Top 120 list be confident enough to cast a final ballot to place them? Just doesn't really work if you ask me.

As for voting, I'm really leaning towards only allowing people who participate in Round 1 to vote in Round 2. Skipping the first part then voting later just seems lazy. If you can't put in the time and effort to rank these players (which is certainly not a simple or easy process) I have a hard time considering that vote a well-informed one in Round 2. Part of the point of Round 1 is to be able to weed out poor lists or bad potential participants who simply don't belong. If anyone could vote you may end up with either more current players vote totals skewed too high or fanboys of certain teams coming along trying to pad a vote total. It's just too hard to screen out idiot voters in the 2nd phase. I suppose exceptions could be made but you'd need at least a few highly respected references to vouch for you before consideration. I hate to disenfranchise anyone but to me the integrity of the project is the most important goal. I still need to talk to Hockey Outsider some before anything is penciled down and get his feelings on it.

Remember, all these rules are up for discussion. The final verdict rests with the community as a whole.

 

[Weztex]

Which is essentially what the system I posted would be doing. Letting everyone participate in an NHL-only list and then once that was done take selected posters who wanted to create an All-Time list to make one including non-NHL players. Merging lists based on average placing is just horrible. For one, players will be slotted 1 or more places downward on an All-Time list than they would on an equivalent All-NHL list. It almost sounds like you want to use a weighted average but that's not really a great way to do it. Plus come Round 2 for the final voting it's impossible to include people who have little to know knowledge of these players in the process. How can someone accurately rank, say, Lindsay vs. Fetisov vs. Kharlamov vs. Gainey? How could someone who isn't confident enough in their ability to rank them on a Top 120 list be confident enough to cast a final ballot to place them? Just doesn't really work if you ask me.

The ''system you posted'' is essentially the same I posted two weeks ago. I am aware of his fonctionalities. So far this is what would work best. Still, I'm trying to find alternatives to get the most people involved since that seems to be the main concern. I know all the distortions a basic average system could do but those can be erase by a correction system. Now if you absolutely want people who participate at round 1 to get into round 2 there's nothing I can do about.

The final verdict rests with the community as a whole.

Then why don't we just make a poll to decide which way this project is gonna take. You and HO are making a tremendous job but I'm not to happy by the fact that, at this very early point of the process, the opinion of hundreds of posters are judge by only 2 of them.

 

[FissionFire]

The ''system you posted'' is essentially the same I posted two weeks ago. I am aware of his fonctionalities. So far this is what would work best. Still, I'm trying to find alternatives to get the most people involved since that seems to be the main concern. I know all the distortions a basic average system could do but those can be erase by a correction system. Now if you absolutely want people who participate at round 1 to get into round 2 there's nothing I can do about.

It depends. How could you know that voters were legit or even had a clue about the players they were voting on? The first part is really the weeding out process where the idiots who would have Crosby in their top 100 get dumped out. What's the point in rejecting an absurd list if you don't ban that person from the final voting process? The same level of incompetence will just carry over and taint the results. The only really way to ensure that the participants are knowledgable is off that initial list. I'm open to ideas on any other to ensure the participants are qualified if you have one. This is simply the best I've been able to come up with. Could anything be worse than having someone like bilros voting?

Then why don't we just make a poll to decide which way this project is gonna take. You and HO are making a tremendous job but I'm not to happy by the fact that, at this very early point of the process, the opinion of hundreds of posters are judge by only 2 of them.

Because a poll on HF is pretty much proven to be unreliable. Lord knows who would vote. What happens in 60% of the people vote for one option, then when the projects starts only 5% of the participate? Was that really reflective of the HOH community or of the lurkers who just wanted to say something? The entire point of this thread is for people who want to and plan on participating to voice their opinions and views. If someone doesn't comment on the process, of course they won't be heard.

HO and myself are also not "judging" anything. The prelim guidelines were what the majority seemed to prefer based on the original thread. We filled in some of the blanks and put it out there for comment. So far, there's been more negative feedback about not including all players than positive. Is there a middle ground - that's what we are trying to find. Don't feel that I'm fighting against you. I'm simply pointing out potential pitfalls or shortcomings. It would be pretty pointless to spend so much time and thought making a list only to have it second-guessed almost immediately because some detail or loophole wasn't accounted for.

If it's any consolation, l've voluntarily removed myself from any voting process in any phase to ensure that no suspected bias will occur on my behalf. There's also very little chance any list would get rejected barring something crazy like Ovechkin or Crosby making it. Even then, as stated, the person would have a chance to justify the pick. I've got a pretty good idea which names should be on every list and about a 20 placing range or so where they could realistically fall based on voter preferences, etc. Someone sending a list with Bobby Orr at #12 would probably be asked to explain that choice. If the person has a reasonable answer that shows he/she spent time researching the choice and has a rationale for it then it's fine. If he says "Bruins Suck" I doubt anyone here would want that list included, no matter how reasonable the rest of it may seem.

 

[FissionFire]

Sorry for the double-post, but my brain just started working again.....

Why only make 1 list? We could set up an All-NHL list and an All-Time list and people would be able to submit their lists for either one (or even both). This way everyone gets to participate, with the NHL-strong posters taking part in one list centered around their strength and knowledge and the more worldy posters able to rank every hockey player ever with peers at the same level of knowledge. This should protect the integrity of the All-Time list by ensuring the knowledge of non-NHL players amongst the voters, while at the same time allowing those who do not yet have the level of expertise to create a list of their own based on their expertise. Both lists would be created concurrently with the same rules and deadlines. It might be interesting to see how the different groups view players.

 

[Weztex]

If it's any consolation, l've voluntarily removed myself from any voting process in any phase to ensure that no suspected bias will occur on my behalf.

Nah you guys are doing a great job. I wouldn't want to see any of you abandon this great project. It just seems that everything is moving slow since very few people come here and state their opinions. We're basically at the same point than 5 days ago. I would just love to see more people activily take part of trowing ideas for puting up solid bases. Nothing wrong with you and HO's work. It just seems things will lift off in April or May at the pace thing are going. My lack of patience has been talking lately. Sorry if I seemed harsh.

 

[reckoning]

Could anything be worse than having someone like bilros voting?

Good point.

 

[Hockey Outsider]

As for voting, I'm really leaning towards only allowing people who participate in Round 1 to vote in Round 2. Skipping the first part then voting later just seems lazy. If you can't put in the time and effort to rank these players (which is certainly not a simple or easy process) I have a hard time considering that vote a well-informed one in Round 2. Part of the point of Round 1 is to be able to weed out poor lists or bad potential participants who simply don't belong. If anyone could vote you may end up with either more current players vote totals skewed too high or fanboys of certain teams coming along trying to pad a vote total. It's just too hard to screen out idiot voters in the 2nd phase. I suppose exceptions could be made but you'd need at least a few highly respected references to vouch for you before consideration. I hate to disenfranchise anyone but to me the integrity of the project is the most important goal. I still need to talk to Hockey Outsider some before anything is penciled down and get his feelings on it.

You're probably right about this. Screening would be easiest upon the submission of the initial list.

When "quality control" is done for the lists, I think we should look for systematic biases. Nobody should have their list thrown out for including a questionable player or having a consensus top-30 player ranked too low. A list probably would get thrown out if somebody is clearly lacking in knowledge about a certain aspect of NHL history (i.e no pre-1930s players).

I think we'd need some volunteers for this, and to keep this process as open and transparent as possible, it shouldn't be FF or myself.

I'm still thinking about NHL vs. no restrictions on eligibility. Settling it by vote is probably the best way to see where the consensus lies, but it should be conducted by PM so that only posters likely to participate have a say.

 

[Czech Your Math]

Would be glad to help with counting votes, if they can be put into an Excel file.

I'd like to see more discussion first of the players themselves, particularly the top 50 or so. Possibly threads separated by position (G,D,F) and/or possibly era, with arguments and data presented for each player on the appropriate thread. That would give everyone a resource to make a more informed decision.

There are many ways votes can be tabulated, including:

- simple point system
- point system, with extra points for being on list
- order players by which was ranked higher on more lists
- take the median "X" number of votes and use those... X would probably be from 1/4 to 1/2 of the total votes, so if there are 40 voters, the middle 10 or 20 votes.

 

[FissionFire]

I'd like to see more discussion first of the players themselves, particularly the top 50 or so. Possibly threads separated by position (G,D,F) and/or possibly era, with arguments and data presented for each player on the appropriate thread. That would give everyone a resource to make a more informed decision.

I think discussing merits of certain players prior to submitting the initial list is a great idea. My only request is that people don't start putting up lists until the 2nd round of voting. One of my concerns (and maybe I'm just being overly cautious) with people posting full or partial lists before the submission deadline is that it might inspire a "copycat" effect where a poster who isn't very confident in an era or on the fence about some rankings will just see how others did it and rank accordingly. I'd also worry that if someone say a player being ranked seemingly close to another, they might bump a player up or down a couple spots to try and give him the better ranking. At the end of the entire process I'm considering posting the rankings based on votes for each participant. That would help ensure people don't make crazy votes since there would be full disclosure. Sort of the anti-Coach's Poll for college football.

There are many ways votes can be tabulated, including:

- simple point system
- point system, with extra points for being on list
- order players by which was ranked higher on more lists
- take the median "X" number of votes and use those... X would probably be from 1/4 to 1/2 of the total votes, so if there are 40 voters, the middle 10 or 20 votes.

All good ideas, but the it seems a simple points method where everyone submits a list of 120 players ranked in order is the method that we'll use. This should remove the need for any "appearance bonus" for players at the bottom of the list. Since the bottom 20 players will essentially not even be counted on the final it's a way to build-in a buffer to the process without running the risk of having bonus points pushing them higher up in the rankings. So far nobody has really seemed to express any complaints with this method or pointed out any major flaws. If you see one by all means please let us know. Nothing would be worse than spending all this time and effort on the process only to overlook a seemingly obvious flaw which taints the entire process.

 

[Czech Your Math]

I think discussing merits of certain players prior to submitting the initial list is a great idea. My only request is that people don't start putting up lists until the 2nd round of voting. One of my concerns (and maybe I'm just being overly cautious) with people posting full or partial lists before the submission deadline is that it might inspire a "copycat" effect where a poster who isn't very confident in an era or on the fence about some rankings will just see how others did it and rank accordingly. I'd also worry that if someone say a player being ranked seemingly close to another, they might bump a player up or down a couple spots to try and give him the better ranking. At the end of the entire process I'm considering posting the rankings based on votes for each participant. That would help ensure people don't make crazy votes since there would be full disclosure. Sort of the anti-Coach's Poll for college football.

I agree that lists shouldn't be posted of actual votes, until after everyone has voted.

Why not use voting, polls, and similar measures that have already taken place to form a group of goalies, defensemen and forwards for discussion:

- Hockey News list of top 100 (and top 60 post-expansion)
- Polls on HFboards: I know Canucksfan had top 25-30 forwards of all time/since WWII, I would guess there are other polls for goalies and defensemen.
- All Time Draft: could look at last X All Time Drafts on HFboards

From those, can have a consensus for discussion. I would like to see threads like this before actual voting:

- Top 20 or 30 players thread- Reserved for players with a legitimate argument for being one of the top 20 or 30 players of all time.

- General discussion by position: Goalies, Defensemen, Forwards.

- Pre-WWII thread:- Players that few have ever seen play, and many don't know a lot about.

People can post player stats, studies, profiles, and reasoned arguments on the appropriate site. This could be done over the course of a couple weeks to a month.

Voters then submit their lists. The lists are put into Excel, a copy is made with the voters' names removed and then sent to some other people for tabulation (by various means listed previously). There is a thread where results are posted publicly (stil no voters' names) to verify correct tabulation. Within a week, the verified results are posted for discussion, probably grouped 1-20, 21-40, or similarly. Optionally, voters' individual lists are also posted publicly with names attached.

What do you think of something along those lines?

 

[FissionFire]

I agree that lists shouldn't be posted of actual votes, until after everyone has voted.

Why not use voting, polls, and similar measures that have already taken place to form a group of goalies, defensemen and forwards for discussion:

- Hockey News list of top 100 (and top 60 post-expansion)
- Polls on HFboards: I know Canucksfan had top 25-30 forwards of all time/since WWII, I would guess there are other polls for goalies and defensemen.
- All Time Draft: could look at last X All Time Drafts on HFboards

From those, can have a consensus for discussion. I would like to see threads like this before actual voting:

- Top 20 or 30 players thread- Reserved for players with a legitimate argument for being one of the top 20 or 30 players of all time.

- General discussion by position: Goalies, Defensemen, Forwards.

- Pre-WWII thread:- Players that few have ever seen play, and many don't know a lot about.

People can post player stats, studies, profiles, and reasoned arguments on the appropriate site. This could be done over the course of a couple weeks to a month.

Voters then submit their lists. The lists are put into Excel, a copy is made with the voters' names removed and then sent to some other people for tabulation (by various means listed previously). There is a thread where results are posted publicly (stil no voters' names) to verify correct tabulation. Within a week, the verified results are posted for discussion, probably grouped 1-20, 21-40, or similarly. Optionally, voters' individual lists are also posted publicly with names attached.

What do you think of something along those lines?

Isn't that essentially what Round 1 is in the original post? How exactly does it differ from that? All the pre-submission discussion would have to be initiated by participants. I don't think H.O or myself starting up discussion threads would be appropriate. Feel free to start a thread on some or all of the topics you mentioned. Arguing the merits of players is in the realm of the voters, not the organizers IMO. I also think it would be extremely time consuming to create some type of pre-lists like you mentioned for discussion before voting. I also think it would be inappropriate because if could potentially influence some people to not rank a player where they may have thought he belonged simply because he is/is not listed on a "top 30" prelist or "best of" prelist. The research is really left up to each voter and the discussion is something that should be voter-driven.

 

[Czech Your Math]

Isn't that essentially what Round 1 is in the original post? How exactly does it differ from that?

My understanding is that people were to submit lists soon, the results would be tabulated, then discussion by 10 players at a time to "fine tune" the results.

I can see why the organizers would not want to start discussion threads. I could start a couple threads, but not sure how much time would be allowed for discussion before voting takes place.

 

[Dark Shadows]

So when do we need our lists ready? Has a date been set?

Im waiting for a thread to confirm sending PM's with lists, etc

 

[Transplanted Caper]

I think it's probably best to go with the outline selected in the original post. We can;t find a perfect medium here and we'll just keep going around and around with different scenarios.