NHL.com + Sochi 2014 Data URLs in JSON and JSONP

Discussion in 'By The Numbers' started by Step7750, Feb 1, 2014.

  1. CAF1837

    CAF1837 Registered User

    Joined:
    Mar 7, 2006
    Messages:
    383
    Likes Received:
    0
    Trophy Points:
    64
    Home Page:
    Thanks. The game-by-game stats are only available on the actual gamesheets (as far as I know) which are pdf files. Anyways, I'll find a way. Thanks again!
     
  2. patiw

    patiw Registered User

    Joined:
    Nov 23, 2014
    Messages:
    19
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Wladyslawowo, POLAND
    Sad to say, but this code dosen't work for me :( There is a hint:
    Warning: preg_replace(): Compilation failed: nothing to repeat at offset 4 in /Applications/XAMPP/xamppfiles/htdocs/xampp/daily.php on line 15
    Line 15 is:
    PHP:
    $rawscores preg_replace("/.+?(**.+}).+/","$1",$rawscores);
     
  3. Pominville Knows

    Pominville Knows Gameplanhockey(DOT)com, no pay-to-play!

    Joined:
    Sep 28, 2012
    Messages:
    4,121
    Likes Received:
    153
    Trophy Points:
    82
    Gender:
    Male
    Occupation:
    Faceplant
    Location:
    Down Under
    Home Page:
    I'm looking to put selected players only current seasonal stats into a table. How do i do this the most easiest?

    Thanks!
     
  4. patiw

    patiw Registered User

    Joined:
    Nov 23, 2014
    Messages:
    19
    Likes Received:
    0
    Trophy Points:
    0
    Location:
    Wladyslawowo, POLAND
    Can you explain "selected players only current seasonal stats" ?
     
  5. Pominville Knows

    Pominville Knows Gameplanhockey(DOT)com, no pay-to-play!

    Joined:
    Sep 28, 2012
    Messages:
    4,121
    Likes Received:
    153
    Trophy Points:
    82
    Gender:
    Male
    Occupation:
    Faceplant
    Location:
    Down Under
    Home Page:
    Okey, i want a scoring race that updates automatically with only select players, for example:

    1. Alexander Ovechkin
    2. Brent Burns
    3. Taylor Chorney

    Needs only to be updated when loading the page, not like Java or whatever withour reloading. Thanks for tips!
     
    Last edited: Apr 23, 2015
  6. King Karl

    King Karl Swagalicious

    Joined:
    Mar 18, 2014
    Messages:
    551
    Likes Received:
    0
    Trophy Points:
    34
    Location:
    Halifax, NS
    I guess you could pick out their player id's from the table and store them in an array, and then when you search through your database to pull up the stats (Or if you just want to pull it directly from the JSON tables), you could only select ones that match an id in the array.
     
  7. RTM

    RTM Registered User

    Joined:
    Mar 19, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    For the record I found a way to get the IIHF game statistics. IIHF updated their website and statistics are now also available in HTML here http://www.iihfworlds2015.com/en/games/2015-05-01/CAN-vs-LAT/#statistics-tab.

    I dug around a bit and found that the site uses this source for the web site: http://stats.iihf.com/hydra/2015-wm/widget_en_2015_wm_game_summary_new_13.html.

    But how nice would it be to find the actual REST API that returns that json or is that exactly the endpoint I'm looking for? I doubt it is as the requests can sometimes take really long time so one would think there is a rest API somewhere responding much quicker.
     
  8. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    For those of you who know Python, there is a package called Beautiful Soup which makes it easy as hell to scrape data.

    Code:
    from bs4 import BeautifulSoup
    import urllib2
    
    url = "http://www.nhl.com/ice/standings.htm?type=lea#&navid=nav-stn-league"
    page = urllib2.urlopen(url)
    soup = BeautifulSoup(page, "html.parser")
    
    rows = soup.find("table", width=660).find("tbody").find_all("tr")
    
    for row in rows:
    	cells = row.find_all("td")
    	for i in range(0, 17):
    		print cells[i].get_text()
    Just with the above code, the entire table in the URL above is printed to the screen, stripped of all HTML tags.

    Here's the output:

    Code:
    1
    *p - NY Rangers
    M
    82
    53
    22
    7
    113
    49
    252
    192
    +60
    25-11-5
    28-11-2
    4-5
    7-3-0
    Won 1
    2
    *y - Montréal
    A
    82
    50
    22
    10
    110
    43
    221
    189
    +32
    26-9-6
    24-13-4
    7-5
    5-2-3
    Won 3
    3
    *z - Anaheim
    P
    82
    51
    24
    7
    109
    43
    236
    226
    +10
    26-12-3
    25-12-4
    8-5
    6-4-0
    Won 1
    4
    *y - St. Louis
    C
    82
    51
    24
    7
    109
    42
    248
    201
    +47
    27-12-2
    24-12-5
    9-4
    6-3-1
    Won 2
    5
    *x - Tampa Bay
    A
    82
    50
    24
    8
    108
    47
    262
    211
    +51
    32-8-1
    18-16-7
    3-5
    6-3-1
    Won 3
    6
    *x - Nashville
    C
    82
    47
    25
    10
    104
    41
    232
    208
    +24
    28-9-4
    19-16-6
    6-6
    4-4-2
    Lost 3
    7
    *x - Chicago
    C
    82
    48
    28
    6
    102
    39
    229
    189
    +40
    24-12-5
    24-16-1
    9-3
    4-6-0
    Lost 4
    8
    *x - Vancouver
    P
    82
    48
    29
    5
    101
    42
    242
    222
    +20
    24-15-2
    24-14-3
    6-2
    6-3-1
    Won 3
    9
    *x - Washington
    M
    82
    45
    26
    11
    101
    40
    242
    203
    +39
    23-13-5
    22-13-6
    5-4
    6-3-1
    Lost 1
    10
    *x - NY Islanders
    M
    82
    47
    28
    7
    101
    40
    252
    230
    +22
    25-14-2
    22-14-5
    7-6
    4-3-3
    OT 1
    11
    *x - Minnesota
    C
    82
    46
    28
    8
    100
    42
    231
    201
    +30
    22-13-6
    24-15-2
    4-3
    6-3-1
    Lost 1
    12
    *x - Detroit
    A
    82
    43
    25
    14
    100
    39
    235
    221
    +14
    22-10-9
    21-15-5
    4-10
    4-4-2
    Won 1
    13
    *x - Ottawa
    A
    82
    43
    26
    13
    99
    37
    238
    215
    +23
    23-13-5
    20-13-8
    6-7
    6-2-2
    Won 3
    14
    *x - Winnipeg
    C
    82
    43
    26
    13
    99
    36
    230
    210
    +20
    23-13-5
    20-13-8
    7-6
    6-3-1
    Won 1
    15
    *x - Pittsburgh
    M
    82
    43
    27
    12
    98
    39
    221
    210
    +11
    23-14-4
    20-13-8
    4-6
    3-5-2
    Won 1
    16
    *x - Calgary
    P
    82
    45
    30
    7
    97
    41
    241
    216
    +25
    23-13-5
    22-17-2
    4-3
    6-3-1
    Lost 1
    17
    *Boston
    A
    82
    41
    27
    14
    96
    37
    213
    211
    +2
    24-10-7
    17-17-7
    4-10
    5-3-2
    OT 1
    18
    *Los Angeles
    P
    82
    40
    27
    15
    95
    38
    220
    205
    +15
    25-9-7
    15-18-8
    2-8
    5-4-1
    Won 1
    19
    *Dallas
    C
    82
    41
    31
    10
    92
    37
    261
    260
    +1
    17-16-8
    24-15-2
    4-3
    7-3-0
    Won 4
    20
    *Florida
    A
    82
    38
    29
    15
    91
    30
    206
    223
    -17
    21-13-7
    17-16-8
    8-10
    5-4-1
    Won 2
    21
    *Colorado
    C
    82
    39
    31
    12
    90
    29
    219
    227
    -8
    23-15-3
    16-16-9
    10-4
    6-4-0
    Won 3
    22
    *San Jose
    P
    82
    40
    33
    9
    89
    36
    228
    232
    -4
    19-17-5
    21-16-4
    4-6
    5-4-1
    Lost 1
    23
    *Columbus
    M
    82
    42
    35
    5
    89
    33
    236
    250
    -14
    19-20-2
    23-15-3
    9-2
    9-0-1
    Won 3
    24
    *Philadelphia
    M
    82
    33
    31
    18
    84
    30
    215
    234
    -19
    23-11-7
    10-20-11
    3-11
    4-3-3
    Lost 2
    25
    *New Jersey
    M
    82
    32
    36
    14
    78
    27
    181
    216
    -35
    19-14-8
    13-22-6
    5-7
    1-6-3
    Lost 1
    26
    *Carolina
    M
    82
    30
    41
    11
    71
    25
    188
    226
    -38
    18-16-7
    12-25-4
    5-7
    4-5-1
    Lost 1
    27
    *Toronto
    A
    82
    30
    44
    8
    68
    25
    211
    262
    -51
    22-17-2
    8-27-6
    5-5
    3-5-2
    OT 1
    28
    *Edmonton
    P
    82
    24
    44
    14
    62
    19
    198
    283
    -85
    15-23-3
    9-21-11
    5-7
    4-5-1
    OT 1
    29
    *Arizona
    P
    82
    24
    50
    8
    56
    19
    170
    272
    -102
    11-25-5
    13-25-3
    5-5
    3-7-0
    Lost 3
    30
    *Buffalo
    A
    82
    23
    51
    8
    54
    15
    161
    274
    -113
    14-22-5
    9-29-3
    8-5
    3-6-1
    Lost 2
     
  9. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
  10. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    Well, it looks like the NHL updated some stuff and made data mining substantially easier.

    All skater summary data (every season, from 1917 to 2015):

    http://www.nhl.com/stats/rest/individual/skaters/game/skatersummary

    All team summary data (every season):

    http://www.nhl.com/stats/rest/grouped/teams/season/teamsummary

    Player mugs:

    http://cdn.nhle.com/photos/mugs-silo-stats/<playerid>.png

    Change <playerid> to the appropriate player id


    You can filter the data using Cayenne expressions like so:

    http://www.nhl.com/stats/rest/individual/skaters/game/skatersummary?cayenneExp=gameDate>="1987-10-08T06:00:00.000Z" and gameDate<="2015-06-18T06:00:00.000Z" and gameTypeId=2

    That will get all regular season player summaries from 1987 to 2015


    Maybe you just want players with a first name of Bill:

    http://www.nhl.com/stats/rest/grouped/skaters/season/skatersummary?cayenneExp=playerFirstName == "Bill"
     
  11. DBizzle99

    DBizzle99 Registered User

    Joined:
    Oct 28, 2015
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    0
    Occupation:
    programmer
    Location:
    Celina, TX
    Awesome thread, thanks for all the research and insight!

    One question, can these NHL api's be used to pull boxscore json legally, without violating their terms of use?

    I've got a fun little side project I've been working on for years... It started as a chrome plugin to transform the display of my son's hockey games on pointstreak.com, and over the years I've tweaked it and created a website (betterboxscore.com) to do the same for NHL boxscores (even wrote a free Android app!). But the way I read their terms of use, screen scrapping is prohibited and I've been entering scoring data manually the morning after. I'd love to be able to write a util that takes advantage of these api's to pull data automatically, potentially even near-realtime, but the LAST thing I want to do is run afoul of the NHL legal department.

    Fyi, this is all for free right now, that makes it non-commercial, right? I'm not opposed to monetizing it in some way, though, if the opportunity presents itself (there's space on the website for google ads, for example, but doesn't show anything right now). Would this make any difference in the legality question?
     
  12. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    I've been scraping NHL.com every five minutes for the past two years and they either haven't noticed or haven't cared. I've also been using team logos. I doubt they care if tiny websites scrape their site, so unless your site becomes as huge as something like war-on-ice, I wouldn't worry about it.
     
  13. PuckNuts

    PuckNuts LEOHL is the best Old-Timers Hockey in Canada...

    Joined:
    Apr 15, 2007
    Messages:
    11
    Likes Received:
    3
    Trophy Points:
    61
    Gender:
    Male
    Location:
    London, Ontario
    Home Page:
    Thanks

    Awesome links, thanks to all for the very informative posts.

    My results for daily scores.

    NHL Scores

    Regards
    PuckNuts
     
    Last edited: Nov 28, 2015
  14. JetsFlamesFan*

    JetsFlamesFan* Registered User

    Joined:
    Jan 21, 2014
    Messages:
    63
    Likes Received:
    0
    Trophy Points:
    0
    I found this thread very useful. I was trying to download and scrape gamesheets before I discovered this thread. Now I've been working on just obtaining the data and displaying it in some readable format. I used a combination of php and javascript to print out all the data from the JSONP file.

    This part just load the data and gets rid of the extra loadscoreboard( ) surrounding all the data, because javascript doesn't seem to understand jsonp format.
    Code:
    <?php
    	$jsonFile = file_get_contents("http://live.nhle.com/GameData/GCScoreboard/2015-11-04.jsonp");
    	$jsonArray = explode("(", $jsonFile);
    	$jsonFile = $jsonArray[1];
    	$jsonArray = explode(")", $jsonFile);
    	$jsonFile = $jsonArray[0];
    ?>
    
    Then I used the following php/javascript code to just output all the actual game data (between the body tags):

    Code:
    <script type="text/javascript">
    	var scoreBoard = JSON.parse('<?php echo $jsonFile; ?>');
    			
    	for (key in scoreBoard.games) {
    		for (property in scoreBoard.games[key]) {
    			document.write(property + ": "+ scoreBoard.games[key][property] + "<br>");
    		}
    		document.write("<hr>");
    	}
    </script>
    
    This just outputs all the data, so you can figure out what all the different properties are. Then you can just mess around with it and pull the data you want. As an example, I edited the for in loop to display the team name, score and boxscore:

    Code:
    	for (key in scoreBoard.games) {
    		document.write(scoreBoard.games[key]["atcommon"] + "&nbsp;&nbsp;&nbsp;&nbsp;" + scoreBoard.games[key]["ats"] + "<br>");
    		document.write(scoreBoard.games[key]["htcommon"] + "&nbsp;&nbsp;&nbsp;&nbsp;" + scoreBoard.games[key]["hts"] + "<br>");
    		document.write(scoreBoard.games[key]["bs"] + "<hr>");
    	}
    
     
  15. Kulstad

    Kulstad Registered User

    Joined:
    Nov 14, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    This is a fantastic thread, and I've been able to make tremendous progress in an app I'm creating because of this. Thank you all for your efforts.

    I do have a few questions/requests...

    • is there a URL that contains any or all of the following (I would be expecting these IDs to be a numeric value, but wouldn't be surprised if they were text):

      TeamId
      DivisionID
      ConferenceId

    • Is there a URL that contains the full season schedule? In my mind, I see it something like:
      GameID...GameDate...ATID...HTID​
      where ATID is the away team ID and HTID is the home team ID
     
  16. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    I can't find any JSON files for one, but you could create a program to scrape NHL.com and create your own JSON file for the app to call.
     
  17. Kulstad

    Kulstad Registered User

    Joined:
    Nov 14, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    Thank you for the effort. I've got Admin tool portion of my app gathering the game day JSON for each day, and loading it into the DB already, but I guess I'm just looking for an easy/lazy way out of doing the kind of initial setup I need
     
  18. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    If you're creating a commercial app out of data from NHL.com, you have to abide by their TOS which states:

    2. Prohibited Content and Activities

    Engage in unauthorized spidering, scraping, or harvesting of content or information, or use any other unauthorized automated means to compile information;

    Source: http://www.nhl.com/ice/page.htm?id=26389

    If I were you, I'd make sure everything is fine with their legal department.
     
  19. meetc

    meetc Registered User

    Joined:
    Jun 30, 2014
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    4
    Does anyone know if/where I could find a data source for live scores of the IIHF Wold juniors when it starts?
     
  20. Kulstad

    Kulstad Registered User

    Joined:
    Nov 14, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    Does anyone have experience with MLBAM-created websites? MLBAM bought all the NHL digital rights in the summer and they are launching a brand new NHL.com website during the All-Star break and then rolling out new club websites and mobile apps in the off-season. This has the very real and serious potential to break any apps that use the links provided in this thread.

    http://www.sportsnet.ca/hockey/nhl/a-closer-look-at-nhls-partnership-with-mlbam/
     
    Last edited: Dec 26, 2015
  21. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    That is some exciting stuff, hope they do a good job with presenting the analytics. If you're using any data source, you should be able to easily adapt and find whatever need be on your own; you must understand that it may be unreliable. Obviously, it'll probably break most things in this thread, but that shouldn't be too much of an issue if you understand common protocols and such (which is how this thread was formed).
     
  22. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
  23. The Saw Is the Law

    The Saw Is the Law Registered User

    Joined:
    Jul 31, 2010
    Messages:
    2,139
    Likes Received:
    25
    Trophy Points:
    66
    Where could I find/scrape TOI stats? I mean total Time On Ice, not avg toi/game
     
  24. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
  25. The Saw Is the Law

    The Saw Is the Law Registered User

    Joined:
    Jul 31, 2010
    Messages:
    2,139
    Likes Received:
    25
    Trophy Points:
    66

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
monitoring_string = "358c248ada348a047a4b9bb27a146148"