How do I display trickplay images with hls.js that elaborated at EXT-X-IMAGE-STREAM-INF in m3u8

Displaying trickplay images using hls.js involves integrating the trickplay image data specified by the EXT-X-IMAGE-STREAM-INF tag in the M3U8 manifest. Trickplay images are typically used to provide previews or thumbnails of video content, often displayed when a user hovers over the seek bar.

Here's a step-by-step guide to achieve this:

1. Setup HLS.js: Ensure you have hls.js integrated into your project. You can include it via CDN or install it via npm/yarn.

   <script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
   

   Or install via npm/yarn:

   npm install hls.js
   

2. Initialize HLS.js: Initialize hls.js and attach it to your video element.

   <video id="video" controls></video>
   <div id="thumbnail-container">
       <img id="thumbnail" style="display: none;" />
   </div>
   

   if (Hls.isSupported()) {
     const video = document.getElementById('video');
     const hls = new Hls();
     hls.loadSource('path/to/your/playlist.m3u8');
     hls.attachMedia(video);
     hls.on(Hls.Events.MANIFEST_PARSED, function () {
       video.play();
     });
   }
   

3. Parse Trickplay Image Data: Parse the EXT-X-IMAGE-STREAM-INF tag from the M3U8 manifest. This tag will provide information about the trickplay images, including the URL, resolution, and interval.

   hls.on(Hls.Events.LEVEL_LOADED, function (event, data) {
     const imageStream = data.details.fragments.find(fragment => fragment.tag === 'EXT-X-IMAGE-STREAM-INF');
     if (imageStream) {
       const imageUrlTemplate = imageStream.url;
       const imageInterval = imageStream.attrs['RESOLUTION'];
       const width = imageStream.attrs['RESOLUTION'].split('x')[0];
       const height = imageStream.attrs['RESOLUTION'].split('x')[1];
   
       setupThumbnails(imageUrlTemplate, imageInterval, width, height);
     }
   });
   

4. Setup Thumbnail Display: Create a function to display the thumbnails when the user hovers over the seek bar.

   function setupThumbnails(imageUrlTemplate, imageInterval, width, height) {
     const video = document.getElementById('video');
     const thumbnail = document.getElementById('thumbnail');
     const thumbnailContainer = document.getElementById('thumbnail-container');
   
     video.addEventListener('mousemove', function (event) {
       const duration = video.duration;
       const videoWidth = video.clientWidth;
       const mousePosition = event.offsetX;
       const hoverTime = (mousePosition / videoWidth) * duration;
   
       const thumbnailIndex = Math.floor(hoverTime / imageInterval);
       const thumbnailUrl = imageUrlTemplate.replace('{$index}', thumbnailIndex);
   
       thumbnail.src = thumbnailUrl;
       thumbnail.style.display = 'block';
       thumbnailContainer.style.left = `${event.pageX}px`;
       thumbnailContainer.style.top = `${event.pageY - height - 10}px`;
     });
   
     video.addEventListener('mouseleave', function () {
       thumbnail.style.display = 'none';
     });
   }
   

5. Update Styles: Add some basic styles to position the thumbnail container.

   <style>
     #thumbnail-container {
       position: absolute;
       pointer-events: none;
     }
     #thumbnail {
       width: auto;
       height: auto;
       max-width: 100px;
     }
   </style>
   

6. M3U8 Manifest Example: Ensure your M3U8 manifest includes the EXT-X-IMAGE-STREAM-INF tag correctly.

   #EXTM3U
   #EXT-X-VERSION:7
   #EXT-X-IMAGE-STREAM-INF:RESOLUTION=320x180,CODECS="jpeg",URI="thumbnails_180p.m3u8"
   #EXT-X-IMAGE-STREAM-INF:RESOLUTION=640x360,CODECS="jpeg",URI="thumbnails_360p.m3u8"
   

   The thumbnails_180p.m3u8 might look like this:

   #EXTM3U
   #EXT-X-VERSION:7
   #EXT-X-TARGETDURATION:10
   #EXTINF:10.000,
   thumbnail_00001.jpg
   #EXTINF:10.000,
   thumbnail_00002.jpg
   #EXTINF:10.000,
   thumbnail_00003.jpg
   

This is a basic implementation to get you started with displaying trickplay images using hls.js. You can customize and optimize this code according to your requirements.