VLM Example: Mosaic Streaming

This page explains how to configure a vlm file to broadcast a mosaic. It covers the following topics:

• How to create a mosaic and display it to local system?

• How to create a mosaic and stream it to the network?

• How does the code actually work?

The first two sections provide the VLM code for the specified scenarios, and the third section explains the code and gives an overview of how the mosaic works behind the scenes.

We will transcode the video streams together using the transcode module. Refer to How to transcode to understand how to perform basic transcoding operations. It will also help you are familiar with applying filters while transcoding.

For handling the audio part, we will use the Bridge framework. In this case, the use of the bridge is that we can send the audio streams to it, and get them back later when we need them.

This way, we can send the audio streams to the bridge and focus on transcoding the video streams into a mosaic as per our requirement. When the code of video is complete, we will simply get the audio streams back from the bridge and combine them with the transcoded video stream to get our final output.

The process will become more clear with the following example.

How to create a Mosaic and display it to the local system?

For this example, we will create a mosaic of four videos. To do that, we will set up five broadcast channels - one for each video and one for the mosaic. You can combine more videos together, but remember to make sure to configure the height and width parameters accordingly.

To create a mosaic, we will combine all the video streams first, and then add the audio streams over the transcoded video.

Consider the following vlm code:

new channel1 broadcast enabled
setup channel1 input "file:///C:/Users/UserName/Desktop/sample1.mkv"
setup channel1 output #duplicate{dst=mosaic-bridge{id=1,width=960,height=540},select=video,dst=bridge-out{id=11},select=audio}

new channel2 broadcast enabled
setup channel2 input "file:///C:/Users/UserName/Desktop/sample2.mkv"
setup channel2 output #duplicate{dst=mosaic-bridge{id=2,width=960,height=540},select=video,dst=bridge-out{id=12},select=audio}

new channel3 broadcast enabled
setup channel3 input "file:///C:/Users/UserName/Desktop/sample3.mkv"
setup channel3 output #duplicate{dst=mosaic-bridge{id=3,width=960,height=540},select=video,dst=bridge-out{id=13},select=audio}

new channel4 broadcast enabled
setup channel4 input "file:///C:/Users/UserName/Desktop/sample4.mkv"
setup channel4 output #duplicate{dst=mosaic-bridge{id=4,width=960,height=540},select=video,dst=bridge-out{id=14},select=audio}

new mosaic broadcast enabled
setup mosaic input "file:///C:/Users/UserName/Desktop/bg.jpg"
setup mosaic option image-duration=-1
setup mosaic output #transcode{sfilter=mosaic{width=1920,height=1080,cols=2,rows=2,order="1,2,3,4",keep-aspect-ratio=enabled,keep-picture=1,mosaic-align=5},vcodec=mp4v}:bridge-in:display

control mosaic play
control channel1 play
control channel2 play
control channel3 play
control channel4 play

In the above code, we have used the duplicate module while specifying the output for the broadcasts, because we need to send the audio and video streams to separate destinations. To implement the code, follow these steps:

• Create a text file in any text editor and copy the code.

• Change the file path and file name in the input parameter of the broadcasts as per where the files are located in your system.

• (optional) Modify the height and width in case the resolution of your system is not 1080x1920.

• Save the code with the .vlm extension (like mosaic.vlm).

• Open the terminal and navigate to the directory where you saved the vlm code.

• Run the following command to execute the vlm file:

$ vlc --vlm-conf=mosaic.vlm

As the above command is executed, the VLC Media Player will open and start playing the desired mosaic.

How to create a Mosaic and stream it over the local network?

For this example, we will do a multicast stream over RTP. Refer to Stream over RTP to understand how streaming over RTP works. We will also use SAP announcements as that makes it more convenient for the receiver(s) to connect. As a reminder, note that streaming over the local network means that only the devices connected to the same router as the streaming device will be able to receive the stream.

In order to stream the mosaic over the local network, we just need to change the final dst in the output part of the mosaic broadcast. Hence, instead of sending the final output to the display module, we will call the rtp module.

Therefore, the last line of the mosaic broadcast section will become:

setup mosaic output #transcode{sfilter=mosaic{width=1920,height=1080,cols=2,rows=2,order="1,2,3,4",keep-aspect-ratio=enabled,keep-picture=1,mosaic-align=5},vcodec=mp4v}:bridge-in:rtp{mux=ts,dst=239.255.255.250,port=5010,sap,name=testing}

Hence, follow these steps to stream the mosaic over the local network:

• Replace the setup mosaic output line in the original vlm file with the one written just above.

• Execute the file as by following the steps specified in the last section.

And then, on the receiving device(s):

• Open VLC and navigate to View -> Playlist.

• Click on Network Streams (SAP) under the “Local Network” section on the left hand side.

• A stream by the name of testing will become visible (like in the figure below).

• Double click on it to start receiving the mosaic.

How the SAP announcement looks like in the VLC window.

How does the code work?

What is happening in the code is that for all the media files, we:

• Open them using the setup input command.

• Send the video to the mosaic (by using mosaic-bridge in the dst parameter), and

• Send the audio to the bridge (by using bridge-out in the dst parameter).

Then, in the mosaic broadcast, we:

• Get the videos by setting sfilter=mosaic in the transcode block.

• Transcode the four videos together (in a 2x2 matrix) over the specified background image.

• Send the transcoded video stream to the bridge (by using the : in the sout chain).

After that, the bridge:

• Receives the video stream from the mosaic.

• Gets the audio streams from the files and attaches them to the mosaic-video output (by using the bridge-in module), and

• Sends the final video + audio combined stream to the specified output.

Refer to the following diagram to understand how the streams flow between the modules:

As we can see in the graph, all the video streams and the background image are sent to the Mosaic which transcodes them together. Because have used a subtitle filter (in the code snippet, sfilter=mosaic), the transcoded frames are actually being added as subtitles to make the video stream. This video stream is the output from the Mosaic block.

The video stream then goes to the Bridge. The Bridge combines the video stream with all the audio streams that it receives straight from the input media files, and outputs the combined audio+video stream - which is either displayed on the system or streamed over the local network.

As the video streams were transcoded together but the audio streams were not, therefore the final stream contains one video stream and multiple audio streams that are being played simultaneously.