Working with Video
The modules in the “Video” menu allow you to synthesize, manipulate, and project real-time video streams. Like other modules in the platform, video can be fed from one tool to another, and learning how to work with video data is easy, with a few important concepts to familiarize yourself with and keep in mind as you work.
Basic Video Module Layout
Video modules look similar to sound and data modules, but include some unique aspects.
Like sound and data modules, video modules include some combination of dials, number boxes, buttons, and other interfaces that allow you to control the parameters within the module. Video modules also include some combination of input and output bus addresses, that allow you to send the video stream from one module to another.
One unique feature of video modules is the ability to ‘preview’ the video feed directly in the module. Previews typically default to ‘off’, and including too many previews at one time can slow down the GPU - however, this will be a very useful tool as you work.
Currently, there is only one module to project a video onto a larger window - that’s the ‘Projector’ module (covered later in this tutorial) - so previews will be crucial to understanding how your video feed looks at various points in the signal chain.
Routing video streams is incredibly similar to routing audio signals, with a few key differences.
One small difference is that address naming is simplified. With Video modules, you only need to create a number - any number between 1 and 9999 - No need to open any dropdown menu. Simply type in the number and hit enter, or click outside of the box, to set the value. You can also click and drag the number up and down with your keyboard.
Video inputs have a ‘mint’ background with a black border and text. Outputs, on the other hand, have a black background with a mint border and text. Several VSynth video modules allow video inputs for frequency, phase, and pulse width modulation, and it’s definitely worth your time to explore these options.
Note that channel 0 is a dummy channel, and will not send or receive any video.
Video may be projected onto one or more additional windows via the ‘Projector’ module. This module accepts an incoming video feed and maps it to a window that can be resized, or made full screen. Pressing ‘Shift+F’ on your keyboard will make any projector windows currently open switch to full-screen, and pressing ‘Shift+F’ again will toggle the windows out of full-screen. The projected video can maintain it’s original resolution, or can be stretched to fill the window.
Synthesized video can be created with any of the VS modules, and several more will be added in the future. Synthesis will occur at a resolution determined by the ‘Video Size’ parameters on the console. These are measured in pixel size, and the default 640x480 values can be updated at any time - even during a performance.
VS modules are created using the VSynth library. This open-source library requires us to keep the original interface intact, so working with VSynth modules requires a slight modification to your work flow. Dials look different, and parameters are framed within the original design - However, all processes can be automated, controlled, and saved just like other modules. We’ve provided a slight variation on these module backgrounds to stay aesthetically consistent with the creator’s vision. If you’d like to learn more about Kevin Kripper’s toolbox for video synthesis, you can dive deeper at www.patreon.com/vsynth
Just like audio, video can be recorded from the console. These settings are found immediately below the ‘Video Size’ options, and include several options.
- Record Type - This menu allows you to choose whether you want to record audio, video, video with audio (Vid w/A) or video and audio separately (V+A). Recorded video is saved in your project folder, in the ‘Saved Audio’ subfolder.
- Video Channel - This number simply allows you to select a specific video channel address to record. For example, if you want to record output from a module that is routing out to V2, you would want to set ‘2’ as your selection on the console.
Currently, we only offer recording video files in the .mov format - we haven’t been able to capture stable footage in other file types yet. If you want to record video in other formats, right now we recommend using 3rd party screen capturing software like Open Broadcast System (OBS) or Nvidia ShadowPlay. This provides the additional advantage of removing demands from the MSDP processing threads.
Working with Video files
Note that this information is duplicated from the ‘Working with Audio Files’ section of the ‘Working with Audio’ tutorial. Although the images do not line up, the concept functions identically
Several modules in MSDP can access, playback, and modify files. Working with files is a breeze, and there are a variety of methods for getting video into your projects.
The media manager
The recommended method of working with files is to first add them to your project, using the Media Manager.
The Media Manager can be accessed by finding the ‘Media Manager’ button on the console. This can be found beneath the ‘Select Board File’ dropdown menu.
Adding files to the Media Manager will create a copy of the file in your project folder. This will ensure that the files travel with your project, and will make sure that projects work as expected on any computer that you open them on.
To add files to the Media Manager, you can simply find the files on your computer that you want to add, and drag-and-drop them into the Media Manager window. If you are using MSDP in full screen, you’ll need to toggle out of full screen mode, using the ‘full screen’ toggle on the console.
Alternately, you can press the ‘Search Computer for Media’ button to create an ‘Open’ dialogue window. From here you can search your computer for the file you want to add. This does come with one disadvantage - using this method, you can only add one file at a time, while you can drag-and-drop multiple files at once into the Media Manager window.
Loading files into modules
There are four ways to load files into modules, each of which will be reviewed below.
Drag and Drop from the Media Manager - This is our top recommended method - simply open the manager and drag the name of the file over the module, then drop it to within the module frame to have the file load.
Drag and Drop from Desktop - This is the easiest way to load a file that isn’t saved in the project. For this method, simply find the file on your desktop or in a finder / explorer window, and drag the file over the module. Drop the file within the module frame to load the file into the module.
IMPORTANT - If you decide to use files from outside of your project, you will not be able to properly open the files on other computers. For this reason, we recommend grabbing all of your files from the media manager.
Pressing the ‘Open’ button on any file-loading module will open a secondary window called the ‘Video Loader’. In some scenarios, this may be faster than using the above methods.
The Video Loader Menu - Any video files that exist in your project’s Media Manager will automatically appear in this dropdown menu. Selecting an option from this menu will cause the Loader window to close, and will load the file into your module.
The ‘Search Computer’ button - The video loader also allows you to search your computer for any video file. Just like dragging and dropping video from outside of your project, files opened with this method do NOT save in the project. This means that your module may fail to load the appropriate video file if opened on another computer.
Working with Webcams
Along with synthesized video and video files, MSDP also allows users to open streams from any webcam connected to your current computer. There are two modules built for this purpose: the Camera Capture module, and the VS-Camera-FM module. Both will be covered below.
Camera Stream Limitations
It is important to note that a camera stream can only be captured by one source at a time. This means that a user could have three web cams plugged in, and have one module for each stream, but a user can NOT grab the same web cam footage from three camera modules at the same time.
This shouldn’t be a problem though - instead of grabbing the footage from multiple camera modules, take the output of one camera module, and send it to multiple locations!
The Camera Capture module
This is a traditional camera stream capture. You can choose camera source and toggle the stream. It includes the following parameters:
- Start/Stop stream toggle - Use this to begin capturing, or to stop capturing, footage from a webcam.
- Select Source - Choose the appropriate camera from a list of currently connected sources. If you connect a camera after opening MSDP, it might be necessary to re-load the module, or to restart the program.
- Mode - Toggle between a normal or mirrored display.
- and 5. Video Output addresses - allows the stream to be sent to another video module.
The VSynth Camera FM module
This module works similarly to the default camera capture module, but includes the ability to modulate the camera feed with the signal of another video feed! Notice that, because VSynth modules have a slightly different signal flow, the dials and buttons will work differently within the VSynth module. The parameters include:
- Camera Source - Select your camera from the dropdown menu.
- Start / Stop stream toggle - the toggle in this module is represented as a red or gray square in the top left corner of the inner-module. When red, the camera feed is toggled to the ‘on’ state.
- Frequency Modulation X-Axis - Determines the intensity of frequency modulation along the Y-axis. Requires a modulating signal (parameter 5).
- Frequency Modulation Y-Axis - Determines the intensity of frequency modulation along the Y-axis. Requires a modulating signal (parameter 5).
- Frequency Modulation Source In - Input channel for modulating signal, should come from other video module.
- and 7. Vidio Output Adresses - allows the stream to be sent to another video module.
Changing the Video settings
Several video options can be accessed and changed in the ‘System Preferences’. This window can be accessed by going to ‘File -> System Preferences’ on the top menu.
On the bottom left of the System Preferences window, the ‘Video Settings’ section can be found. This allows you to modify two important parameters across all projects.
Target Video FPS - This number determines the target frames per second for live video synthesis and output. Setting a target does not guarantee a consistant framerate - instead it will determine the max fps allowable. A higher FPS will look nicer, if acheivable, but a lower number will reduce demands on the GPU and can result in a more stable frame rate.
Record Video FPS - This number determines how many frames will be captured per second in recorded video. The recorded values are different than the target values for a reason - our internal system for recording requires us to capture footage at time intervals that are divisible by 1000.
3rd Party Screen Capture Tools
If you want to record video at 60fps or higher, right now we recommend using 3rd party screen capturing software like Open Broadcast System (OBS) or Nvidia ShadowPlay. This provides the additional advantage of removing demands from the MSDP processing threads.
Additionally, in the System Preferences, you can set the default video settings for new projects. These options are identical to the ones on the console discussed above in the ‘Recording Video’ and ‘Synthesizing Video’ sections.