Two ways of web scraping

Screen scraping: What we have done by extracting the data from the source code of the website.

Web APIs (Application Programming Interface): Website offers a set of http requests that you can use use to access data. We will not cover this in this class.

However, there are wrapper packages for some APIs. In short, you can connect to certain APIs (i.e. if the package exists) without having to know about too much about working with APIs.

Example

library(spotifyr)

Get Access to the Spotify API

https://developer.spotify.com/dashboard/login

You will get a Client ID and Client Secret

Authentication

Do not use this specific method on a public computer.

usethis::edit_r_environ()

In your .Renviron write the following and save your file. The XXXXXXXXX comes from your own Spotify developer account.

SPOTIFY_CLIENT_ID = XXXXXXXXXXXXXXXXXXXXXXXXXXX
SPOTIFY_CLIENT_SECRET = XXXXXXXXXXXXXXXXXXXXXXXX

https://open.spotify.com/artist/6vWDO969PvNqNYHIOW5v0m

get_artist("6vWDO969PvNqNYHIOW5v0m")

## $external_urls
## $external_urls$spotify
## [1] "https://open.spotify.com/artist/6vWDO969PvNqNYHIOW5v0m"
## 
## 
## $followers
## $followers$href
## NULL
## 
## $followers$total
## [1] 24835016
## 
## 
## $genres
## [1] "dance pop" "pop"       "r&b"      
## 
## $href
## [1] "https://api.spotify.com/v1/artists/6vWDO969PvNqNYHIOW5v0m"
## 
## $id
## [1] "6vWDO969PvNqNYHIOW5v0m"
## 
## $images
##   height                                                              url width
## 1    640 https://i.scdn.co/image/ad8b0e5a18a5a443a2678768bd73f59833941abc   640
## 2    320 https://i.scdn.co/image/802895be7bc5339087ba36194b0b7307c467df96   320
## 3    160 https://i.scdn.co/image/a932ba0a31bd2807fe76c77b64c680bec2f3d14a   160
## 
## $name
## [1] "Beyoncé"
## 
## $popularity
## [1] 89
## 
## $type
## [1] "artist"
## 
## $uri
## [1] "spotify:artist:6vWDO969PvNqNYHIOW5v0m"

get_artist("6vWDO969PvNqNYHIOW5v0m") %>% 
  str()

## List of 10
##  $ external_urls:List of 1
##   ..$ spotify: chr "https://open.spotify.com/artist/6vWDO969PvNqNYHIOW5v0m"
##  $ followers    :List of 2
##   ..$ href : NULL
##   ..$ total: int 24835016
##  $ genres       : chr [1:3] "dance pop" "pop" "r&b"
##  $ href         : chr "https://api.spotify.com/v1/artists/6vWDO969PvNqNYHIOW5v0m"
##  $ id           : chr "6vWDO969PvNqNYHIOW5v0m"
##  $ images       :'data.frame':    3 obs. of  3 variables:
##   ..$ height: int [1:3] 640 320 160
##   ..$ url   : chr [1:3] "https://i.scdn.co/image/ad8b0e5a18a5a443a2678768bd73f59833941abc" "https://i.scdn.co/image/802895be7bc5339087ba36194b0b7307c467df96" "https://i.scdn.co/image/a932ba0a31bd2807fe76c77b64c680bec2f3d14a"
##   ..$ width : int [1:3] 640 320 160
##  $ name         : chr "Beyoncé"
##  $ popularity   : int 89
##  $ type         : chr "artist"
##  $ uri          : chr "spotify:artist:6vWDO969PvNqNYHIOW5v0m"