HLS Testing with JMeter

In this article, we will cover how to perform video streaming (downloading action) tests with JMeter.

Video Protocols

First of all, let’s talk about what happens in the background while watching any video.

Several communication standards are used to transmit and share video data. We call these video protocols. These protocols contain specific rules and guidelines to properly transmit and process video streams and files between the client and server. Here are explanations of some basic video protocols:

• HTTP (HyperText Transfer Protocol): HTTP is a basic protocol used for transmitting data such as text, images, and video through web browsers. It is mainly used for video streams on the web. When video files are transferred from the server to the client, special versions that support HTTP video streaming can be used with protocols such as HLS or MPEG-DASH.
• RTSP (Real-Time Streaming Protocol): RTSP is a communications protocol used to manage and control real-time video streams. Devices such as IP cameras and video servers deliver video streams with RTSP and play them back to clients.
• RTMP (Real-Time Messaging Protocol): RTMP is a communications protocol developed by Adobe and used for live or recorded video streams. It is supported by Flash-based video players and media servers.
• RTP (Real-time Transport Protocol): RTP is a protocol used to transmit real-time media data such as audio and video. It is frequently used in VoIP and video conferencing systems.
• SIP (Session Initiation Protocol): SIP is a communication protocol used for voice and video communication. It is used to start and end sessions in VoIP and video conferencing applications.
• WebRTC (Web Real-Time Communication): WebRTC is a collection of technologies and protocols used for browser-based real-time communication. This protocol enables voice and video calls via web browsers.
• MPEG-DASH (Dynamic Adaptive Streaming over HTTP): MPEG-DASH is a protocol used to make video streaming dynamically adaptive over HTTP. This is used to ensure the best video quality for different devices and connection speeds.
• HLS (HTTP Live Streaming): HLS is a video streaming protocol developed by Apple and used primarily on iOS devices. It offers video in different resolutions and formats.

The two most used protocols today are MPEG-DASH and HLS protocols. Let’s continue our article with videos using the HLS protocol.

HLS Protocol

Videos processed with the HLS protocol use components with manifest and .ts extensions. HLS offers dividing videos into segments and providing these segments with different quality levels.

Manifest Files

• Master Playlist (Master M3U8): It is the main manifest file used to watch HLS videos. This file contains references to all HLS streams with different streams and quality levels. The client receives this file and then selects the appropriate stream and quality level. The master playlist has a .m3u8 extension.
• Media Playlist (Media M3U8): These files contain the list of segments of the video at a certain quality level. The media player receives these files and then makes a request to retrieve these segments. There is a media playlist for each quality level.

.ts (Transport Stream) Extensions

• Videos are divided into many smaller segments called “segments”. Each segment represents a section or time period of the video. These .ts extension files contain these segments of video content. For example, “video_01.ts,” “video_02.ts,” etc.

HLS Flow Chart

Now let’s do an example with the information we have learned so far. We will use New York Times website as an example.

As you can see, when the video starts, the components we mentioned above appear as HTTP requests.

In videos with subtitle support, text files are among these requests.

HLS Plugin

Now let’s see how to do what we have learned via JMeter. To do this, we first need to download the plugins-manager.jar file from the link to the lib/ext folder in the JMeter folder on our computer. After completing this process, when we open JMeter, we can access the plugins as can be seen in the picture.

We can download the plugin we will use in video tests by typing HLS Plugin in the Available Plugins field.

Test Creation

We will use the sampler called “HLS Sampler” when creating video tests. To add the sampler, we can right-click on the Test Plan and follow the path Add → Thread Group → Add → Sampler → HLS Sampler.

Configuration

In the URL field; You need to write the URL of the manifest (.m3u8 or .mpd) that creates the video and contains general information about the video. So how do we find this?

We can capture this URL information by using the Network tab in the browser as we showed in an example above or by using JMeter’s Recording feature while the video is playing. It is usually named manifest or master. In our example, this URL value is “https://vp.nyt.com/video/hls/2023/08/01/110343_1_opdoc-clean_wg/master.m3u8".

You can set the duration you want to download from the video in the Play Option section. Duration can be “Entire video” or a custom duration in seconds.

We can determine the resolution values ​​we want to test with the Bandwith and Resolution fields. We can select the minimum and maximum fields for both values. In these cases, the plugin bases the minimum and maximum values ​​in the manifest file.

If we want to test specific values, we can select the “Custom” field and enter the values ​​we want. The values ​​entered here must be the same as the values ​​in the manifest file. For our example, let’s take the relevant value as 1920x1080.

Analysis of Result

After all these configuration settings are made, let’s add View Result Free and run the test to see the results.

For this test, we will use a Custom Video duration of 60 seconds, with bandwidth at minimum and resolution at 1080P. With these configurations, the sampler should look like the image below:

When we start the test after making the relevant settings, the plugin will start downloading the manifest and segment files from the specified URL.

And voila! We have successfully requested and watched a video with a virtual user.

To check the accuracy of the test, we can check the response of the HLS Sampler — master playlist service. The response of the relevant service will include information such as language settings, bandwidth, and resolution settings of the video URL we tested.

As you can see, the values ​​we see on the Network tab in the browser are the same as the values ​​we see when we test with JMeter.

When you finish creating your JMeter script to test your HLS videos, you can upload it to Loadium and enhance your testing abilities with better reporting, collaboration, and scalability.

Loadium also allows you to generate traffic from all over the world, testing the performance of your server for users worldwide.