MATLAB Digest - September 2003

Rapid Prototyping of a Surveillance Video Compression System

Developing embedded video applications can be an exciting and challenging task. Without access to proper tools and technologies, however, development can be intimidating and even frustrating.

In this article, we show you how to quickly prototype a complete video application in Simulink using the frame-based, multirate, matrix-oriented signal processing capabilities in the DSP Blockset. You will learn how to approach new video processing tasks in Simulink and perhaps learn more about motion detection.

We first examine data compression, and then describe how to prototype and simulate a motion-detection algorithm for a video surveillance application with Simulink and the DSP Blockset. We then discuss how to tune the simulation, automatically generate code for an embedded target to deploy the real-time operation. Finally, we measure the target performance and verify the system.

Compressing Data for Video Surveillance

Because video surveillance systems produce large amounts of data to record, archive, and review, data compression is a vital part of next-generation surveillance systems. For example, even a modest 8-bit/pixel monochrome digital surveillance system with CIF resolution (352x240 pixels) running at 30 frames/second will send approximately 2.4 MB/second to a storage device.

There are two primary ways to decrease the space required to store the surveillance video data: store only those video frames that have certain characteristics, such as significant motion; or store compressed representations of individual video frames. In this article, we use a simple motion-detection algorithm to automatically select individual video frames for storage. This will help us reduce our example’s recording rate.

Figure 1: Source video at 30 frames/sec. Click on image to play video.

Figure 2: Recorded video at << 30 frames/sec. Click on image to play video.

Prototype of Motion-Detection Algorithm

Motion within the camera’s field of view can trigger video frame storage. Figure 1 shows a video source producing data at 30 frames/sec, even when the subject is still. Figure 2 shows a video frame where the subject is in motion; all non-motion video frames are suppressed, greatly reducing the frame rate. One of the simpler algorithms for estimating motion is to compute the difference between successive video frames. The greater the number of differences that we find, the more motion is likely to have occurred within the scene. We designed the block diagram in Figure 3 to compute the Sum of the Absolute value of Differences (SAD) between successive video frames, returning a useful estimate of motion.

Figure 3: A Simulink model depicting the SAD algorithm in block diagram form. Click on image to see enlarged view.

In Figure 3, output port #2 (SAD) produces the motion estimate. The video source is 160x120 pixels, and the operations use integer data types for memory efficiency.

The auto-scaling and overflow-reporting features and fixed-point parameters of Simulink help with investigating fixed-point effects, such as detecting dynamic range overflow conditions or assessing the impact of rounding modes on intermediate computations.

A concern to consider when designing fixed-point systems is the effect of limited dynamic range and scaling on intermediate computations. In this system, a key intermediate result is the “absolute difference” (AD) image formed immediately prior to the final summation for the SAD output. The video sequence representing the AD result may be obtained in the above block diagram from output port #1, labeled “AD.” An example AD image, shown in Figure 4 as captured on a Matrix Viewer block from the DSP Blockset, is the result of running the full block diagram system shown later in Figure 6, with the AD video stream selected in place of the final surveillance video result.

Figure 4: Absolute Differences image formed prior to final summation for the SAD motion estimate.

The image in Figure 4 displays ghost-like outlines of objects in motion. Only those portions of two video frames where differences are detected are revealed with high-intensity pixel values; portions of the image where little difference occurs from one frame to the next remain dark. Artifacts in the AD image would indicate possible errors in integer processing, often due to improper scaling, leading to overflow of integer summations and premature saturation of intensity values. The proper appearance of this image is a key intermediate result of system simulation.

Tuning the Simulation

The summation of pixel intensities in the AD image leads to the SAD estimate of motion. Figure 5 shows motion estimates plotted in a graph over time. The yellow trace is the motion estimate computed for each video frame over a span of approximately 20 seconds. The magenta trace is a threshold level that can be adjusted to discriminate the varying levels of motion. In Figure 6, we see the threshold is set via a constant applied to the InThresh port of the surveillance subsystem. If set too high, the threshold could prevent the recording of video frames containing significant motion, perhaps even missing critical events in the surveillance data. If set too low, output video rates may grow unacceptably large, defeating the compression system. Therefore, it is imperative to enable control of this threshold parameter throughout system simulation trials. Designers can expose this control to users of the final embedded surveillance system, or automate control of the threshold level in a later stage of system development.

Figure 5: Motion estimates and threshold for assessing detection sensitivity.

A Simulink subsystem encapsulates the SAD algorithm in block diagram form, providing a graphical hierarchy of the model and enabling large-scale modeling in top-down or bottom-up fashion. Figure 6 shows the system-level model of the entire surveillance video compression application.

Figure 6: Simulink model showing original video image (at left) and the selected video frames (on right) for archiving. Click on image to see enlarged view.

Generating Embedded Code

You can make several modifications to customize the Simulink model for a digital signal processor (DSP) target, such as using hardware-specific device drivers. For the target, we use the Texas Instruments TMS320C6416 DSK, a DSP Starter Kit featuring a high-performance fixed-point processor (C6416) for video and imaging applications. We incorporate Real-Time Data Exchange (RTDX), a real-time data transfer system unique to TI processors, into the Simulink model in Figure 7 by introducing blocks provided by the Embedded Target for TI C6000 DSP. These blocks automatically configure full-rate video data transfer to and from the hardware, enabling a test bench or other host-side code to be developed for validating system operation. We will address host-side visualization of the real-time data stream later in this article.

Since MATLAB Link for Code Composer Studio and the embedded target products support RTDX technology, we stream a prerecorded video clip from MATLAB into the embedded system created with Simulink, and then stream the clip out of the system and back into MATLAB for review. Similarly, we perform real-time changes to parameters, such as the motion threshold, directly from a MATLAB program without halting the target.

To switch the output video stream between the AD image (Figure 4) and the final compressed video stream, we used a Switch block representing switches found on the DSK hardware (Figure 7). A light-emitting diode (LED) driver block indicates motion detection directly from the DSK hardware.

Figure 7: Simulink model built with C6416 DSK device-driver blocks, including drivers for switches, LEDs, and Real-Time Data Exchange (RTDX). Click on image to see enlarged view.

Once we have set the final options in the Simulation Parameters dialog, we can generate code with a real-time operating system. Here, we use TI’s DSP/BIOS, high-speed RTDX and code optimizations, as shown in Figure 8. Additional options panels can set other options. Once we have generated the code, the target automatically invokes Code Composer Studio to compile, link, download, and run the application code on the target.

Figure 8: Two of the simulation parameter options panels for the Simulink surveillance model, showing a subset of the C6416 DSK targeting options in the Embedded Target for TI C6000 DSP. Click on image to see enlarged view.

Measuring Target Performance

C6416 processor utilization is approximately 4.3% when running the surveillance algorithm at 15 frames/second, owing to its low complexity and pure integer implementation. The Embedded Target can automatically produce this target profiling information as an option. The remainder of the processor computational budget would be allocated to other sections of a complete surveillance system design, such as including image compression of each still-frame, zoom/pan/tilt control, auto-focus, and to support multiple simultaneous video-surveillance channels.

High-speed RTDX has a bandwidth of approximately 2 MB/second, which sets a practical limitation on the size and rate of the video-frame transfer. After we have verified the real-time system, we detach the host PC from the system and use a direct digital video connection to the embedded system to remove the host computer from the final embedded system.

After we have verified the real-time application on the DSK, we may choose to transfer the application to a code-compatible architecture, such as TI’s DM642 digital media processor, for final embedded implementation.

Figure 9: Embedded system featuring the TI DM642 digital media processor based on the C6400 DSP architecture.

Building a GUI to Verify the System in Real Time

Because the MATLAB Link for Code Composer Studio and the Embedded Target for TI C6000 DSP support the TI DSK hardware, we use the DSK to test the real-time video system. We wrote a MATLAB M-file to create a graphical user interface (GUI) for host-side visualization of the data sent to and received from the surveillance application, by making use of the new commands made available in the MATLAB Link for Code Composer Studio. These commands enabled interactive start/stop/download of the target code, as well as bidirectional data transfer between the GUI and the target. Figure 10 shows the custom GUI developed to test the surveillance application from within MATLAB.

Figure 10: MATLAB interacting with the DSP target in real time. The target is running the video surveillance application created with Simulink. Click on image to see enlarged view.

The data in Figure 10 is “live,” changing at a rate of 15 frames/sec. The top plot shows motion estimates scrolling along as they are computed. By clicking the top graph, you can automatically download a new motion threshold to the target. The lower-left image shows video source frames transmitted from the MATLAB workspace in a loop, while the lower-right image shows captured video frames. The figure indicates that 245 video frames were sent, with only 14 frames captured for storage. The application comprises approximately 200 lines of MATLAB code, including automatic recompile and download of the target application, initialization of real-time data transfer, GUI creation, interaction with thresholds, and live data updates.

Summary

By prototyping the video application at the system level, we can identify and fix design problems and perform “what-if” simulations to manipulate key design parameters. You can implement the embedded video application on a TI C6400 processor using automatic code generation and targeting operations with Real-Time Workshop and the Embedded Target for TI C6000 DSP.

The MATLAB Link for Code Composer Studio can help you to verify that the implementation meets the original system specifications, and perform real-time hardware-in-the-loop debugging, validation, and visualization of the embedded application.

Development Phase	Principal Use	Product(s) Used
Host-side verification of target code	Interactive GUI for visualization and target parameter tuning	MATLAB MATLAB Link for Code Composer Studio

Design and simulation of target code	Simulink model	Simulink DSP Blockset

Code generation and target support	Targeting and optimization Board support package	Real-Time Workshop Embedded Target for TI C6000 DSP

Products & Services	Industries	Academia	Support	User Community	Company