How to Play Encrypted HTTP Live Streams Offline with AVFoundation for iOS using Swift 4
Date published: May 16, 2019
8 min read
1. Introduction to AVFoundation
AVFoundation is a framework that provides APIs for various media activities including capturing, editing, exporting and the playback of videos. This article will mainly focus on the playback of videos and how AVFoundation works with the HLS format.
AVFoundation Playback supports a wide selection of media formats (e.g. .mpeg, .avi, .aac, etc.) that can be played from the local storage or played over the network on a server. When the media file is on a server, it gets played using the progressive download playback. Once the download starts, even if the network quality changes, it will continue with the same media file. To add a dynamic change, formats like the HTTP live stream (HLS) appeared, which can adapt to a network quality change.
The main classes that we will cover in this article are the following:
-
AVPlayer: The core class of playing videos on iOS.
-
AVPlayerLayer: This is a CALayer subclass that can display the playback of a given AVPlayer instance.
-
AVAsset: This is a representation of a media asset. An asset object contains information such as duration and creation date.
-
AVPlayerItem: This represents the current state of a playable video. This is what you need to provide to an AVPlayer to get things going.
2. HTTP Live Streaming (HLS)
HLS is a protocol that allows you to send on-demand video, live video or audio streams to your devices. This technology is developed by Apple and it is optimised to deliver the best possible quality.
How exactly does HLS work? In short, the base URL usually refers to a master playlist that contains a list of URLs for variant media playlists that can vary in bit rate, resolution or quality level. This enables automatic switching between streams as network conditions change.
Media playlists are represented as text files saved in the M3U format (.m3u8) and contain URLs to a series of small files called media segments and other information needed for playback.
Normally, a media segment is represented as a short MPEG-2 transport stream file (.ts) that contains video/audio content with a duration of 5 to 10 seconds per file.
HLS offers a method to protect the media content through content encryption. The most commonly used method for HLS encryption is AES-128 encryption.
⦁ Here we have an example of a master playlist with three available variants with different resolutions:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1755600,CODECS="avc1.42001f,mp4a.40.2",RESOLUTION=640x360 media/hls_360.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2855600,CODECS="avc1.4d001f,mp4a.40.2",RESOLUTION=960x540 media/hls_540.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=5605600,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1280x720 media/hls_720.m3u8
⦁ This is an example of one of the variants from the masterlist above:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-PLAYLIST-TYPE:VOD #EXTINF:9.9001, http://www.example.com/segment0.ts #EXTINF:9.9001, http://www.example.com/wifi/segment1.ts #EXTINF:9.9001, http://www.example.com/segment2.ts #EXT-X-ENDLIST
3. Play HTTP Live Streams
To play a media file, we need to create an AVPlayer instance pointing at the video URL.
Then to play/pause, we use these obvious commands:
func initializePlayer(videoUrl: URL) {
let videoAsset = AVURLAsset(url: videoUrl)
let playerItem = AVPlayerItem(asset: videoAsset)
player = AVPlayer(playerItem: playerItem)
playerLayer = AVPlayerLayer(player: player)
}
1. Create an AVUrlAsset from the video URL.
2. Create an AVPlayerItem with the AVURLAsset so that the player can have playback control.
3. Create the player from the player item.
4. Finally, add the player to the AVPlayerLayer to display the playback.
Then to play/pause, we use these obvious commands:
func play() {
player.play()
}
func pause() {
player.pause()
}
For encrypted HLS, the playlist contains the URL of the decryption key as shown below:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:10 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-KEY:METHOD=AES-128,URI="https://www.example.com/hls.key",IV=0xecd0d06aef664d8226c33816e78efa44
The video playback will work with the same method as the unencrypted HLS and the player will automatically access the key URL and download it.
4. Download encrypted HTTP Live Streams
To download an HLS asset, we will use an instance of the AVAssetDownloadURLSession, which is a class that supports the creation and execution of asset download tasks.
Because it’s an encrypted HLS, besides downloading the media file, we also need to download and save the encryption key locally. For this task, we will trigger the asset resource loader to access the .m3u8 files manually:
func initialiseDownloadSession() {
let configuration = URLSessionConfiguration.background(withIdentifier: downloadIdentifier)
downloadURLSession = AVAssetDownloadURLSession(
configuration: configuration,
assetDownloadDelegate: self,
delegateQueue: OperationQueue.main
)
}
We pass an AVURLAsset with the video URL to AVAssetDownloadURLSession, but before this, we need to replace the URL schema of the video URL with a fake one (“nothttps”).
This method forces the download task to call the AVAssetResourceLoader delegate because it cannot handle the fake URL:
func downloadTask(videoUrl: URL) {
var urlComponents = URLComponents(
url: videoUrl,
resolvingAgainstBaseURL: false
)!
urlComponents.scheme = "nothttps"
do {
let asset = try AVURLAsset(url: urlComponents.asURL())
asset.resourceLoader.setDelegate(self, queue: DispatchQueue(label: "com.example.AssetResourceLoaderDelegateQueue"))
downloadURLSession
.makeAssetDownloadTask(
asset: asset,
assetTitle: "My Video",
assetArtworkData: nil,
options: nil
)?.resume()
} catch { print("Erorr while parsing the URL.") }
}
In the resource loader, we check if the request has the URL we just modified. Then we change the schema back to the correct one and download the file with a URLSession.
Next, we return true to indicate that the delegate will load the requested resource:
func resourceLoader(
_ resourceLoader: AVAssetResourceLoader,
shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest
) throws -> Bool {
guard let url = loadingRequest.request.url else { return false }
if url.scheme == "nothttps" {
var urlComponents = URLComponents(
url: url,
resolvingAgainstBaseURL: false
)
urlComponents!.scheme = "https"
let newUrl = try urlComponents!.asURL()
downloadHlsFile(videoUrl: newUrl, loadingRequest: loadingRequest)
return true
}
return false
}
The resource loader will be called as many times as the AVPlayer fails to interpret a resource. We are aiming to get inside a playlist variant, which is where the encryption key is located.
So, in the first request, we will get the master playlist and replace all the URL schemas for all the playlist variants with fake ones. Next, we will return the modified data in the loading request and then finish loading the request.
Now that we have replaced all the schemas with fake ones, when AVFoundation selects the best media stream, the shouldWaitForLoadingOfRequestedResource method from the delegate will be triggered again with one of the URLs from the master playlist.
In the second request, we will finally get access to the encryption key URL. Then we will extract the URL from the file and save it temporarily. We will then replace the key URL in the file with a fake one („notkeyhttps”) and respond with the modified data to the loading request.
func downloadHlsFile(videoUrl: URL, loadingRequest: AVAssetResourceLoadingRequest) {
var request = URLRequest(url: videoUrl)
request.httpMethod = "GET"
let session = URLSession(configuration: URLSessionConfiguration.default)
let task = session.dataTask(with: request) { data, response, _ in
guard let data = data else { return }
let strData = String(data: data, encoding: .utf8)!
let modifiedData: Data!
if strData.contains("EXT-X-KEY") {
let start = strData.range(of: "URI=\"")!.upperBound
let end = strData[start...].range(of: "\"")!.lowerBound
let keyUrl = strData[start..<end]
downloadHlsKey(keyUrl: keyUrl)
let replacedKey = strData.replacingOccurrences(
of: keyUrl,
with: "notkeyhttps://example.com/hlsKey"
)
modifiedData = replacedKey.data(using: .utf8)
} else {
let replacedSchema = strData.replacingOccurrences(of: "https", with: "nothttps")
modifiedData = replacedSchema.data(using: .utf8)
}
loadingRequest.contentInformationRequest?.contentType = response?.mimeType
loadingRequest.contentInformationRequest?.isByteRangeAccessSupported = true
loadingRequest.contentInformationRequest?.contentLength = response!.expectedContentLength
loadingRequest.dataRequest?.respond(with: modifiedData)
loadingRequest.finishLoading()
}
task.resume()
}
Now that we have modified the key URL schema, the delegate method shouldWaitForLoadingOfRequestedResource will trigger for a third time. We will detect when this happens and modify the data with the persistent key, which is the key that we saved locally.
Notice that for the previous requests, we used the MIME content type for the loading request; this time we will use AVStreamingKeyDeliveryPersistentContentKeyType as the content type. This will tell AVFoundation that the content contains the encryption key.
Notice that for the previous requests, we used the MIME content type for the loading request; this time we will use AVStreamingKeyDeliveryPersistentContentKeyType as the content type. This will tell AVFoundation that the content contains the encryption key.
So, the final version of shouldWaitForLoadingRequestedResource will look like this:
func resourceLoader(
_ resourceLoader: AVAssetResourceLoader,
shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest
) throws -> Bool {
guard let url = loadingRequest.request.url else { return false }
if url.scheme == "nothttps" {
var urlComponents = URLComponents(
url: url,
resolvingAgainstBaseURL: false
)
urlComponents!.scheme = "https"
let newUrl = try urlComponents!.asURL()
downloadHlsFile(videoUrl: newUrl, loadingRequest: loadingRequest)
return true
}
return false
} else if url.scheme == "notkeyhttps" {
let hlsKey = Keychain.getData(key: .hlsKey)
loadingRequest.contentInformationRequest?.contentType = AVStreamingKeyDeliveryPersistentContentKeyType
loadingRequest.contentInformationRequest?.isByteRangeAccessSupported = true
loadingRequest.contentInformationRequest?.contentLength = hlsKey.count
loadingRequest.dataRequest?.respond(with: hlsKey)
loadingRequest.finishLoading()
return true
}
While the AVFoundation is downloading each segment of the media file, we can monitor the progress using the AVAssetDownloadDelegate method. With this method, you can update the download progress in your app:
func urlSession(_ session: URLSession, assetDownloadTask: AVAssetDownloadTask, didLoad timeRange: CMTimeRange, totalTimeRangesLoaded loadedTimeRanges: [NSValue], timeRangeExpectedToLoad: CMTimeRange) {
var percentageComplete = 0.0
for value in loadedTimeRanges {
let loadedTimeRange = value.timeRangeValue
percentageComplete += loadedTimeRange.duration.seconds / timeRangeExpectedToLoad.duration.seconds
}
percentageComplete *= 100
DispatchQueue.main.async {
self.progressView.updateProgress(percentageComplete.toCGFloat)
}
}
When the download is finished, the following method will be called:
func urlSession(_ session: URLSession, assetDownloadTask: AVAssetDownloadTask, didFinishDownloadingTo location: URL) {
OfflineHandler.save(key: .location, data: location.relativePath)
let storageManager = AVAssetDownloadStorageManager.shared()
let newPolicy = AVMutableAssetDownloadStorageManagementPolicy()
newPolicy.expirationDate = Date().addYears(1)!
newPolicy.priority = .important
let baseURL = URL(fileURLWithPath: NSHomeDirectory())
let assetURL = baseURL.appendingPathComponent(location.relativePath)
storageManager.setStorageManagementPolicy(newPolicy, for: assetURL)
}
In this method, we get the location where the media file was downloaded. You should save this location because we will use it when the device is offline. Also, notice that the storage manager from iOS will delete files that it doesn’t consider to be very important from time to time.
In this case, we will set a policy for the new downloaded file with a duration and a priority level.
In this case, we will set a policy for the new downloaded file with a duration and a priority level.
5. Play encrypted HTTP Live Streams Offline
func initializePlayer() {
let assetPath = OfflineHandler.get(.location) as! String
let baseURL = URL(fileURLWithPath: NSHomeDirectory())
let assetURL = baseURL.appendingPathComponent(assetPath)
let videoAsset = AVURLAsset(url: assetURL)
videoAsset.resourceLoader.setDelegate(self, queue: DispatchQueue(label: "com.example.AssetResourceLoaderDelegateQueue"))
let playerItem = AVPlayerItem(asset: videoAsset)
player = AVPlayer(playerItem: playerItem)
playerLayer = AVPlayerLayer(player: player)
}
To play the media asset offline, we will need to get the location of the file that we downloaded, make a URL from it, and pass it to an AVURLAsset.
The data in the playlist file still has the fake URL schema for the encryption key, so the resource loader will still need to be called to interpret the key and when that happens, we will provide the key we saved locally.
func resourceLoader(
_ resourceLoader: AVAssetResourceLoader,
shouldWaitForLoadingOfRequestedResource loadingRequest: AVAssetResourceLoadingRequest
) throws -> Bool {
guard let url = loadingRequest.request.url else { return false }
if url.scheme == "notkeyhttps" {
let hlsKey = Keychain.getData(key: .hlsKey)
loadingRequest.contentInformationRequest?.contentType = AVStreamingKeyDeliveryPersistentContentKeyType
loadingRequest.contentInformationRequest?.isByteRangeAccessSupported = true
loadingRequest.contentInformationRequest?.contentLength = hlsKey.count
loadingRequest.dataRequest?.respond(with: hlsKey)
loadingRequest.finishLoading()
return true
}
return false
}
