NHL.com + Sochi 2014 Data URLs in JSON and JSONP

Discussion in 'By The Numbers' started by Step7750, Feb 1, 2014.

  1. King Karl

    King Karl Swagalicious

    Joined:
    Mar 18, 2014
    Messages:
    551
    Likes Received:
    0
    Trophy Points:
    34
    Location:
    Halifax, NS
    Hi all, just wondering if anyone know the link to Capgeek's json/xml table. It's either that or manually inserting 800+ players salary into a database.
     
  2. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    They don't release one. Think about creating a program to scrape the data from their website, even though it's against their rules.

    http://capgeek.com/data-usage/
     
  3. King Karl

    King Karl Swagalicious

    Joined:
    Mar 18, 2014
    Messages:
    551
    Likes Received:
    0
    Trophy Points:
    34
    Location:
    Halifax, NS
    Crap. Time constraints probably won't allow for it.
     
  4. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    What are you exactly creating?
     
  5. King Karl

    King Karl Swagalicious

    Joined:
    Mar 18, 2014
    Messages:
    551
    Likes Received:
    0
    Trophy Points:
    34
    Location:
    Halifax, NS
    I'm creating an nhl statistics page. I'm a second year CS student, and I figured what's better than NHL stats for a databases project. I've finished the page (Apart from a few little bugs), so feel free to take a look. https://web.cs.dal.ca/~egener/index.php
     
    Last edited: Dec 1, 2014
  6. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta

    This should have what you want from CapGeek:

    This should (theoretically) have the salaries for every active player in the NHL:

    Code:
    http://www.capgeek.com/armchair-gm/scripts/controller.php?mode=in_season&type=acquire_players&_=1417476555608
    Just parse the JSON and update the db.

    --------------------------------------------------------------------

    To get players for individual teams:


    Code:
    http://www.capgeek.com/armchair-gm/scripts/controller.php?mode=in_season&type=team&id=9&date=2014-12-01
    Change the variable "id" based upon the team you want data for.

    This has the team ids:
    http://www.capgeek.com/armchair-gm/scripts/controller.php?mode=in_season&type=teams

    Code:
     { "teams": [{"id":6,"name":"Anaheim Ducks","abbreviation":"ANA"},{"id":25,"name":"Arizona Coyotes","abbreviation":"ARI"},{"id":5,"name":"Boston Bruins","abbreviation":"BOS"},{"id":8,"name":"Buffalo Sabres","abbreviation":"BUF"},{"id":9,"name":"Calgary Flames","abbreviation":"CGY"},{"id":10,"name":"Carolina Hurricanes","abbreviation":"CAR"},{"id":11,"name":"Chicago Blackhawks","abbreviation":"CHI"},{"id":12,"name":"Colorado Avalanche","abbreviation":"COL"},{"id":13,"name":"Columbus Blue Jackets","abbreviation":"CBJ"},{"id":14,"name":"Dallas Stars","abbreviation":"DAL"},{"id":15,"name":"Detroit Red Wings","abbreviation":"DET"},{"id":1,"name":"Edmonton Oilers","abbreviation":"EDM"},{"id":16,"name":"Florida Panthers","abbreviation":"FLA"},{"id":17,"name":"Los Angeles Kings","abbreviation":"LAK"},{"id":18,"name":"Minnesota Wild","abbreviation":"MIN"},{"id":2,"name":"Montreal Canadiens","abbreviation":"MTL"},{"id":19,"name":"Nashville Predators","abbreviation":"NSH"},{"id":20,"name":"New Jersey Devils","abbreviation":"NJD"},{"id":21,"name":"New York Islanders","abbreviation":"NYI"},{"id":22,"name":"New York Rangers","abbreviation":"NYR"},{"id":23,"name":"Ottawa Senators","abbreviation":"OTT"},{"id":24,"name":"Philadelphia Flyers","abbreviation":"PHI"},{"id":26,"name":"Pittsburgh Penguins","abbreviation":"PIT"},{"id":3,"name":"San Jose Sharks","abbreviation":"SJS"},{"id":27,"name":"St. Louis Blues","abbreviation":"STL"},{"id":28,"name":"Tampa Bay Lightning","abbreviation":"TBL"},{"id":4,"name":"Toronto Maple Leafs","abbreviation":"TOR"},{"id":29,"name":"Vancouver Canucks","abbreviation":"VAN"},{"id":30,"name":"Washington Capitals","abbreviation":"WAS"},{"id":7,"name":"Winnipeg Jets","abbreviation":"WPG"}] }

    Remember that even though this is not web scrapping, you should still credit CapGeek for ethical concerns.

    Also, this is not meant to be a public data source.
     
  7. CAF1837

    CAF1837 Registered User

    Joined:
    Mar 7, 2006
    Messages:
    383
    Likes Received:
    0
    Trophy Points:
    64
    Home Page:
    Hey everyone,

    I'm trying to find the NHL roster reports (see link below) in JSON or some other easily scrape-able format. More importantly, I'm trying to get a list of all players listed as Captains or Alternates for each game.

    http://www.nhl.com/scores/htmlreports/20142015/RO020760.HTM

    I'm not coming across it anywhere on the gamecenter JSON's posted earlier in this thread.
    Anyone got a clue on this?
    Thanks in advance!
     
  8. ugghhh

    ugghhh Registered User

    Joined:
    Apr 17, 2009
    Messages:
    1,983
    Likes Received:
    1
    Trophy Points:
    96
    Accessing stats in real time for a web app

    Hi,

    Does anyone have experience / know how I would be able to get game events in real time to update a website?

    For instance, I'm looking to get real time info on players on ice, shots on goals, scoring chances, goals, penalties.

    Thanks in advance for any help
     
  9. Bear of Bad News

    Bear of Bad News HFBoards Escape Goat

    Joined:
    Sep 27, 2005
    Messages:
    6,644
    Likes Received:
    2,669
    Trophy Points:
    156
    Location:
    Windsor
    Awards:
    This thread is the place where you're most likely to get what you need.
     
  10. RTM

    RTM Registered User

    Joined:
    Mar 19, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    Great info and awesome sources for NHL stats! Any ideas for getting stats from the upcoming (or past) IIHF World Championships?
     
  11. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    They'll probably have an app for it, I'll look at the network requests if I have time...
     
  12. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    I just stumbled upon an incredibly handy tool you guys would love.

    http://www.import.io.

    As an example, enter: http://www.nhl.com/ice/standings.htm?type=lea#&navid=nav-stn-league

    It returns a perfect table of the standings.

    If you download the desktop version, you're given an API key so you could use their Python script to get a .json file as output. There are things you could use the program for, but the Python API is the only one I'll need so far.

    Edit - It is free, by the way. Don't let that Pricing link on top scare you since it's mainly for people/companies with a bigger need.
     
    Last edited: Mar 21, 2015
  13. RTM

    RTM Registered User

    Joined:
    Mar 19, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    For example the last years stats can be found here: http://stats.iihf.com/Hydra/387/index.html.

    But all the actual data can only be found in PDF files, which I found hard to read automatically. There's a few open source php classes to read the contents of a PDF file, but for some reason the game summaries can't be read.

    Hence the need for a API or a service.
     
  14. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    I've found their internal API (for 2015 Worlds), it is in JSON and completely utilizes POST requests.

    They most likely don't want this to be made a public developer API for aspects of abuse; don't hammer their servers.

    Here you go...


    Stats for every Game in the tournament, including scores:

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=games&tournamentId=414

    Return Excerpt (from one of the games):

    Code:
    {
            "gameNumber": 1,
            "tournamentPhase": 9,
            "gamePhase": "PR",
            "group": "A",
            "venue": "PRA",
            "venueName": "Prague / O2 Arena",
            "homeTeam": "CAN",
            "guestTeam": "LAT",
            "hot": 0,
            "liveCommentaryAvailable": 0,
            "guessAvailable": 1,
            "notificationsCount": 148,
            "progressPerc": 0,
            "progressCode": "Scheduled",
            "homeTeamScore": 0,
            "guestTeamScore": 0,
            "guestTeamGuess": 216,
            "homeTeamGuess": 415,
            "checkInsCount": 2,
            "playoff": 0,
            "homeBestPlayer": 0,
            "guestBestPlayer": 0,
            "tournamentID": 414,
            "highlightsURL": "",
            "uniqueID": "1t414",
            "notification": 0,
            "notifGoal": 0,
            "notifPeriod": 0,
            "notifGame": 0,
            "notifLocal": 0,
            "userGuess": 0,
            "userCheckIn": 0,
            "venueLat": 50.1048,
            "venueLong": 14.4938,
            "venueFoursquareId": "4adcdaa3f964a5207e4e21e3",
            "homeCramo": 0,
            "guestCramo": 0,
            "showTime": 1,
            "dateTime": "1430489700"
        }
    

    Stats for a Specific Game:

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=games&gameNumber=1&tournamentId=414

    Parameters to change:
    gameNumber: Number of the game that you want stats for (found in request 1 or 2)

    Returns:

    Code:
    {
            "gameNumber": 1,
            "tournamentPhase": 9,
            "gamePhase": "PR",
            "group": "A",
            "venue": "PRA",
            "venueName": "Prague / O2 Arena",
            "homeTeam": "CAN",
            "guestTeam": "LAT",
            "hot": 0,
            "liveCommentaryAvailable": 0,
            "guessAvailable": 1,
            "notificationsCount": 148,
            "progressPerc": 0,
            "progressCode": "Scheduled",
            "homeTeamScore": 0,
            "guestTeamScore": 0,
            "guestTeamGuess": 216,
            "homeTeamGuess": 415,
            "checkInsCount": 2,
            "playoff": 0,
            "homeBestPlayer": 0,
            "guestBestPlayer": 0,
            "tournamentID": 414,
            "highlightsURL": "",
            "uniqueID": "1t414",
            "notification": 0,
            "notifGoal": 0,
            "notifPeriod": 0,
            "notifGame": 0,
            "notifLocal": 0,
            "userGuess": 0,
            "userCheckIn": 0,
            "venueLat": 50.1048,
            "venueLong": 14.4938,
            "venueFoursquareId": "4adcdaa3f964a5207e4e21e3",
            "homeCramo": 0,
            "guestCramo": 0,
            "showTime": 1,
            "dateTime": "1430489700"
        }
    

    Lineup for a Specific Game:

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=gameLineup&gameNumber=1&tournamentId=414

    Parameters to change:
    gameNumber: Number of the game that you want stats for (found in request 1 or 2)

    *This URL will NOT return anything until roughly 60 minutes before the game*

    Return:

    Code:
    Unknown
    

    Game Results (in depth?) for a Specific Game

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=gameResults&gameNumber=1&tournamentId=414

    Parameters to change:
    gameNumber: Number of the game that you want stats for (found in request 1 or 2)

    *The contents of this URL is not known*

    Return:

    Code:
    Unknown
    

    Game Situations for a Specific Game:

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=gameSituations&gameNumber=1&tournamentId=414

    Parameters to change:
    gameNumber: Number of the game that you want stats for (found in request 1 or 2)

    *The contents of this URL is not known*

    Return:

    Code:
    Unknown
    

    Team Standings

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=teamStandings&tournamentId=414

    *Won't return anything until the tournament starts*

    Return:

    Code:
    Unknown
    

    Team List

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=teamList&tournamentId=414

    Return Excerpt:

    Code:
        {
            "noc": "AUT",
            "homeJersey": "http://static.egluservices.com/iihf/jerseys/t1415/dress_aut_home.png?5f",
            "awayJersey": "http://static.egluservices.com/iihf/jerseys/t1415/dress_aut_away.png?5f",
            "tournamentID": 414,
            "notification": 0,
            "notifGoal": 0,
            "notifPeriod": 0,
            "notifGame": 0,
            "notifLocal": 0,
            "uniqueId": "AUTt414",
            "cheersCount": 6,
            "userCheer": 0
        }
    

    Team Members for a Specific Team

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: tournamentId=414&lastUpdate=0&os=ios&requestName=teamMembers&noc=CAN

    Parameters to change:
    noc: Team name abbreviation

    *The contents of this URL is not known*

    Return:

    Code:
    Unknown
    

    Final Rankings

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: lastUpdate=0&os=ios&requestName=finalRanking&tournamentId=414

    *The contents of this URL is not known*

    Return:

    Code:
    Unknown
    

    Tournament Statistics

    URL: http://iihf.egluservices.com/tournaments1415/query.php

    Data: type=pk&lastUpdate=0&os=ios&requestName=statistics&tournamentId=414

    Parameters to change:
    type: Change it to pk, gk, pen, se, or pp

    *The contents of this URL is not known*

    Return:

    Code:
    Unknown
    
     
  15. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    It's a good idea, but parsing the HTML yourself will reduce overhead, minimize API request costs, and allow more flexibility in data extraction. I tested out some JSON, the service seems to not parse it.
     
  16. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    That's true. For my small needs, I don't really need to worry about the overhead, so I just care about the easiness and this is definitely easy. It also gives me good practice with JSON since I never used it before until yesterday and now I feel pretty good about it.

    How do you find these JSON files you posted above? I tried to look at the requests on nhl.com for the standings and I couldn't find anything, so I'm wondering if you know how to find something like that.
     
  17. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    I look at dynamic applications and components of sites and proceed to analyze their traffic and code. A lot of websites are delivered statically, hence the difficulty for you to find the dynamic APIs. Try to look for parts of the site that have dynamic components and other versions of that data (mobile etc...).

    Thanks for reminding me about the standings, NHL doesn't require the use of a device ID in the request anymore:

    The following is in JSON (GET) from NHL's servers (most likely the regular season only)

    http://f.nhl.com/nhl/leagueapp/20142015/standings/standingsO1T1.json


    Even then, I still don't have a use for the millions of stats and videos I've mined on NHL's services. Maybe I should make a public API, I don't really know. I'm pretty sure I've developed extraction/parsing methods for every format the NHL has had for data since 1999.
     
  18. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    Thanks! That would be cool if you were to do that, even though I use only two pages from nhl.com. Maybe others would want to use more.

    And just out of curiosity, where did you find that URL from? As in, where on which page requested it?
     
  19. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    It wasn't on an HTML page, it was on their mobile app (https://itunes.apple.com/ca/app/nhl/id465092669?mt=8). I'm pretty sure they use certificate validation, so you should be mindful of that if you try logging requests with MITM attacks.
     
  20. Kane One

    Kane One Registered User

    Joined:
    Feb 6, 2010
    Messages:
    35,664
    Likes Received:
    563
    Trophy Points:
    140
    Location:
    Brooklyn, New NY
    Thank you!
     
  21. RTM

    RTM Registered User

    Joined:
    Mar 19, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    Kudos to you man! Very convincing.

    Can I find then entire request list of the rest API somewhere? I queried the API with last years tournamentId 387 but I still can't find what I'm looking for - meaning player stats of a single game.
     
  22. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    Make sure every request has this header: Content-Type: application/x-www-form-urlencoded; otherwise it will fail.

    This is an internal API, there is no full request list or documentation. With the header change, the API works fine for retrieving tournament 387.

    This was my response for the game list from 387 (excerpt):

    For game statistics (goals, penalties etc...), try out this:
    URL: http://iihf.egluservices.com/tournaments1415/query.php
    Data: lastUpdate=0&os=ios&requestName=gameSituations&gameNumber=1&tournamentId=387


    If you want a tournament list, go here:

    URL: http://iihf.egluservices.com/tournaments1415/query.php
    Data: lastUpdate=0&os=ios&requestName=tournamentList


    Data for type=se&lastUpdate=0&os=ios&requestName=statistics&tournamentId=387 works fine also.
     
    Last edited: Mar 23, 2015
  23. RTM

    RTM Registered User

    Joined:
    Mar 19, 2015
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    0
    The API itself works, even with GET-method - that's not the issue.

    But what I'm trying to do is fetch the entire player stats of a single game (passes, goals, pp goals, sh goals, penalties, +/-, shots) that can be found from the game summary ("Game Statistics" in this file http://stats.iihf.com/Hydra/387/IHM387B01_74_3_0.pdf).

    On NHL.com the same info can be found from boxscore sheet (see "Stats" here: http://www.nhl.com/gamecenter/en/boxscore?id=2014021084).

    I have a small fantasy league for a few friends and so far I've had to update the stats by hand which is a bit frustrating...
     
  24. CAF1837

    CAF1837 Registered User

    Joined:
    Mar 7, 2006
    Messages:
    383
    Likes Received:
    0
    Trophy Points:
    64
    Home Page:
    Has anyone ever found (or looked) for JSON data for Sweden's second division (Hockey Allsvenskan)?

    I'm trying to put together player game-by-game stats.
    Their site (http://www.hockeyallsvenskan.se/) looks fairly static, and the only other resource I know of is http://stats.swehockey.se
    I'm not sure they even have a player database with playerID's though...

    Thoughts?
     
  25. Step7750

    Step7750 Registered User

    Joined:
    Jan 24, 2014
    Messages:
    55
    Likes Received:
    1
    Trophy Points:
    0
    Location:
    Calgary, Alberta
    Just scrape the HTML, not much you can do
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice
monitoring_string = "358c248ada348a047a4b9bb27a146148"