Interested by improving HLS-LL with byterange ?

[ValentinBesnard]

What do you want to do with Hls.js?

Hi guys,

I do not know where to contact you so I do it here.

I work for a well-know company doing video distribution and we develop an OTT packager doing (among other things) HLS low-latency with byterange. We notice that such streams "works" in the sense that the stream is playing (no playback issue) but this cannot be used in "real" use-case because hls.js makes a lot of requests (a request for each HLS part) while we expect to enter in "steady mode" and have only one request for each segment (and use chunked transfer encoding).

We can provide you a stream. If you are interested, please contact me at valentin.besnard.49 [at] gmail.com

Regards

Valentin

What have you tried so far?

No response

[robwalch]

Hi @ValentinBesnard,

We've been working on preload-hint support, and will share that work in a PR soon. It's scheduled for a later release. It does not include support for streaming segment parts. An LL-HLS sample with chunk-transferred parts as described here https://www.akamai.com/blog/performance/-using-ll-hls-with-byte-range-addressing-to-achieve-interoperabi would help get the ball rolling (I do not know of any publicly available samples and do not have cycles to develop one internally). The best way to reach out and share off of GitHub is through video-dev Slack: https://www.video-dev.org/ in the hlsjs channel.

My preference would be that this issue be converted to a Feature Request for this functionality (or file a new Feature Request) so that we can add the the LL-HLS project https://github.com/orgs/video-dev/projects/3 and assign it a milestone https://github.com/video-dev/hls.js/milestone/70 that fits the Roadmap.

[ValentinBesnard]

Hi @robwalch

Thank for your feedbacks and for the slack link. I have join the channel.

Regarding the sample, we have setup a VM producing such a stream with one of our packager at ATEME. We can share the link with you (and you colleagues) if you provide me your public IP such that we can whitelist you (for security purpose (IT rules), the stream is not public). I can also do a capture of the stream and link it to this ticket as a zip ?
If anyone else seeing this message want to have access to the stream, send me an email at: v.besnard [at] ateme.com with your public IP address and the reason why you want to access the stream.

Here some more details of the feature request (as a description draft).

Context

(I am pretty sure you know all of this but a bit of context is never bad)
Low-latency with HLS parts (without byterange) is working well but now more and more clients want to use CMAF. The main advantage of CMAF is to share media segments between DASH manifests and HLS playlists. This work well without low-latency but when low-latency is used, this is not optimal for several reasons:

• same content store two times on CDN: DASH and HLS use different low-latency mechanisms: chunked transfer encoding (CTE) for DASH and HLS parts (without byterange) for HLS. As a result, the same content is stored two times in CDN: one time as media segments and one time as HLS parts.
• Using HLS parts without low-latency creates a lot of requests on the packager (or on the CDN) and we want to reduce this for performance reasons + badnwidth reduction. To reduce the number of requests, we can increase the duration of HLS parts but this has also a bad impact on latency.
  => To solve these issue, one idea is to use byterange such that HLS also uses chunked transfer encoding to store only complete media segments on CDN and reduce the number of requests.

Example of HLS media playlist with byterange

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:10
#EXT-X-SERVER-CONTROL:CAN-BLOCK-RELOAD=YES,HOLD-BACK=30.000,PART-HOLD-BACK=8.000
#EXT-X-PART-INF:PART-TARGET=2
#EXT-X-MEDIA-SEQUENCE:25889455
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="asset-avc1_100000=10000-init.mp4"
#EXT-X-PROGRAM-DATE-TIME:2023-03-16T11:09:00.572000Z
#EXTINF:10,
asset-avc1_100000=10000-2588945405720000.mp4
#EXTINF:10,
asset-avc1_100000=10000-2588945505720000.mp4
#EXTINF:10,
asset-avc1_100000=10000-2588945605720000.mp4
#EXT-X-PART:DURATION=2,BYTERANGE="33118@0",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945705720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="24595@33118",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945705720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="29989@57713",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945705720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="27909@87702",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945705720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="28494@115611",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945705720000.mp4"
#EXTINF:10,
asset-avc1_100000=10000-2588945705720000.mp4
#EXT-X-PART:DURATION=2,BYTERANGE="35058@0",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945805720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="28830@35058",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945805720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="29303@63888",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945805720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="25202@93191",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945805720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="34769@118393",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945805720000.mp4"
#EXTINF:10,
asset-avc1_100000=10000-2588945805720000.mp4
#EXT-X-PART:DURATION=2,BYTERANGE="23974@0",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945905720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="33793@23974",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945905720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="26712@57767",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945905720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="28913@84479",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945905720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="28749@113392",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588945905720000.mp4"
#EXTINF:10,
asset-avc1_100000=10000-2588945905720000.mp4
#EXT-X-PART:DURATION=2,BYTERANGE="29581@0",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588946005720000.mp4"
#EXT-X-PART:DURATION=2,BYTERANGE="34699@29581",INDEPENDENT=YES,URI="asset-avc1_100000=10000-2588946005720000.mp4"
#EXT-X-PRELOAD-HINT:TYPE=PART,BYTERANGE-START=64280,URI="asset-avc1_100000=10000-2588946005720000.mp4"
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_48000_qad=20001.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_96000_deu=20004.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_96000_qaa=20007.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_48000_fra=20000.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_48000_deu=20003.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_96000_qad=20006.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_48000_qaa=20002.m3u8",LAST-MSN=25889461,LAST-PART=1
#EXT-X-RENDITION-REPORT:URI="asset-mp4a_96000_fra=20005.m3u8",LAST-MSN=25889461,LAST-PART=1

Current status with hls.js

When testing to play this HLS stream with byterange using hls.js, playback is fine (no freeze, no crash, playback is smooth) but hls.js only makes requests on partial segments with a byte range already announced in the media playlist.
For instance with the media playlist above, we have 5 parts of 2 seconds for each media segment. hls.js will make 5 requests to get each part separately. This is not optimal because we still have the same number of requests as a HLS low-latency stream without byterange. We expect to have only one request for the full media segment here.

Expected behavior

We expect the following behavior:

1. (optional) When launching the playback, there is a transition phasis where the player can request parts with byte ranges already announced in the media playlist. These parts are directly delivered by the packager (without delay) because these parts are already completely available.
2. The player then enters in steady mode i.e., that it only makes one request by media segment using a byte range "0-*" (or just no byte range at all) to get the full media segment. The content of the media segment will be delivered on-the-fly using chunked transfer encoding. Usually when you have segments of 10 seconds, each request takes approximately 10 seconds to complete.
   It is not clear if the player has to fetch the media playlist one time by part (with the example of media playlist above: every 2s) or one time by media segment (with the example of media playlist above: every 10s).

We are doing this work with multiple players to make this HLS low-latency mode working. Note that some player are able to directly enter in steady mode without transition phasis. This is even better but having a transition phasis is acceptable.

If it is not clear, do not hesistate to ask questions or modify the description.

Valentin

[robwalch]

We are adding support for EXT-X-PRELOAD-HINT with #6356. While it does not include support for making open-ended byte-range requests, or pulling parts off of a streaming ArrayBuffer, the groundwork is there. (Come to think of it we should test if such a request would be attempted/error or not with the current implementation.) CC @iamboorrito.