QVision core classes.

Main classes for the framework architecture of the QVision. More...

Classes

class  QVApplication
 Main class for QVision applications. More...
class  QVCamera
 Base class for image and video input source objects. More...
class  QVGenericImage
 Base class for QVImage objects. More...
class  QVImage< Type, Channels >
 Image representation class for the QVision. More...
class  QVPropertyContainer
 Base class for dynamic property container objects. More...
class  QVWorker
 Base class for Worker threads. More...

Detailed Description

Main classes for the framework architecture of the QVision.

Basic program structure for QVision applications

QVision's basic programming model.

QVision defines a software framework for application development. This means programs developed using QVision should follow a common general structure, and the program code should adapt to it in a broad sense. A typical QVision application will use some image processing or computer vision algorithms to obtain refined visual information from one or more image sources, generally video inputs, and later will use that higher level information in some task, maybe just depicting it in the user interface, or any other higher level information processing task, like robotic or decision control, image characterization, physical modelling, etc....

Thus, the structure of a QVision program will generally follow this schema:

monoliticComputerVisionSystem.png

Each square block from the figure is a task in the program. The class QVWorker was introduced in the QVision library to model the Computer vision / Image processing algorithms block. It should contain the code that implements those algorithms, read its inputs from devices connected to the computer, video files, or the user interface, and share the results with the rest of the tasks in the program.

An example code for a QVision application was shown in section The first program. The main() function was the following:

int main(int argc, char *argv[])
        {
        // QVApplication object
        QVApplication app(argc, argv, "Example program for QVision library. Play a video on a canvas window.");

        QVMPlayerCamera camera("Video");        // QVCamera object
        PlayerWorker worker("Player worker");   // QVWorker object
        camera.link(&worker,"Input image");

        // GUI object
        QVGUI interface;

        // Video/image output object
        QVImageCanvas imageCanvas("Test image");
        imageCanvas.linkProperty(worker,"Output image");

        return app.exec();
        }

The class QVMPlayerCamera is a subclass for QVCamera. Both represent video or image inputs in the QVision framework, but the class QVCamera is virtual, and every class modelling video or image inputs in the QVision should inherit from it. Actually, it only has one subclass, the QVMPlayerCamera class, which is a flexible video input that can read frames from diverse video file formats, webcams and digital video cameras. It is based on the MPlayer program. To allow a worker object to read frames from a video source, it should be linked to the camera object, specifying the name of the image for the worker, with a line like this:

camera.link(&worker,"Input image");

The object QVApplication registers worker and camera objects in the system, and process command line input parameters. It inherits from the QApplication class, so it has a similar functionallity. It is the most important object in a QVision application, so it always should be created first in the main() function, before any worker, camera or graphic interface object. Only one object derived from this class should be created in a QVision application.

The class QVGUI is the main GUI widget. This object reads the workers and cameras registered in the QVApplication object, so it contains buttons and sliders to control the algorithm parameters (computer vision or video processing algorithms parameters) for the system, and the video input flow, allowing the user to stop an resume frame read from the video sources opened by the program, in execution time. Also, it can display information about cpu usage stadistics, frame rate, seconds read from the video inputs, etc... and it allows the user to stop and close the program. It is recommended to create only one object of its type for any QVision application.

The class QVImageCanvas is a video or image output widget. Any object created with it will display an image window showing an output image from the worker object. Like the camera object, it should be connected to the worker to read the resulting images, specifying the name of the output image with a line like this:

imageCanvas.linkProperty(worker,"Output image");

Warning:
Because the exec() function is not virtual in the QApplication class, make sure you call the exec() function from the QVApplication class, not from the QApplication class, in a QVision application. Basically, avoid creating and using QVApplication objects like this:
int main(int argc, char *argv[])
        {
        // QVApplication object
        QApplication *app = new QVApplication(argc, argv, "Example program for QVision library. Play a video on a canvas window.");

        [...}

        return app->exec();     // this will execute QApplication::exec() function, not QVApplication::exec() function.
        }

Dynamic properties

Dynamic properties help creating workers, cameras, user interfaces, and video output widgets like software components, increasing modularity and reusability in the QVision applications.

For example, the graphical user interface and the QVApplication objects were designed to allow the user to inspect and modify the algorithm parameters in execution time, by sliders and buttons, and by the command line interface respectively. Thus those objects should be able to find those parameters in the worker object, but it is not desirable to change them for every possible worker programmed.

Dynamic properties are an homogeneous way of sharing data between objects, without creating design dependencies between them. They are like common object properties, in the sense that they can be read or modified in execution time, but they also can be added or deleted from the objects, and be discovered by other objects in execution time, without knowing the class of the object or any other information about it, except that it can contain dynamic properties.

Workers should share their inputs (algorith parameters) using dynamic properties, so the QVGUI and the QVApplication objects can modify and read them in real time. Also, it is desirable for reasons that we will see in section Parallel programming with QVision, that workers share their output data (images, or other data structures) and image inputs through dynamic properties.

So, when linking a worker to a camera or a video output widget, actually a dynamic property from the worker and a dynamic property for the camera or the widget are being linked, telling the program that whenever a worker reads or writes an image, it actually reads it from the camera, or writes it to the video output widget. You can read more about property containers in the QVPropertyContainer class reference page. Every object capable of having dynamic properties is derived from that class.

The worker object

As it was previously commented, worker objects should contain the code for the computer vision and/or image processing algorithms, that the application uses. Every worker object must define two functions: a constructor, that adds the proper dynamic properties for the worker, and a QVWorker::iterate() function. The latter is a virtual function that should be redefined containing the code for the Computer vision / Image processing algorithms task/block in the application.

This function should read input images, parameters, and other data inputs from dynamic properties contained in the worker, process them, and store the resulting images, data structures, or values, in other dynamic properties contained in the worker object.

An example code for a QVWorker class was shown in section TheFirstProgram:

class PlayerWorker: public QVWorker
        {
        public:
                PlayerWorker(QString name): QVWorker(name)
                        {
                        addProperty< QVImage<uChar,1> >("Input image", inputFlag|outputFlag);
                        addProperty< QVImage<uChar,1> >("Output image", outputFlag);
                        }

                void iterate()
                        {
                        QVImage<uChar,1> image = getPropertyValue< QVImage<uChar,1> >("Input image");

                        // image processing / computer vision code should go here

                        setPropertyValue< QVImage<uChar,1> >("Output image", image); 
                        }
        };

This worker has no input parameters, and it just reads an input image, and stores it in an output property.

Other programming models in QVision

Todo:
Here will be explained other models for QVision applications, using triggers and GUI input.

ImageProcessing

Image processing functionality for the QVision library is contained mainly in two different packages. First is the package QVIPP. QVision makes use of Intel's IPP library, and offers many of its functionality to QVision users by means of wrapper functions. These functions make use of ROI and anchor image properties described in the section Regions of interest (or ROI's) and anchors that follows, and are contained in the package Wrapper functions for Intel(R) IPP., which you can check to learn more about it.

Second is the package QVDTA. It is an acronym that stands for QVision Data Types and Algorithms. It contains mostly Computer Vision algorithms and data types, built over the functions in the package qvipp, and other algorithms not directly implemented by the IPP library. You can check the API of the Computer Vision data types and algorithms. package, to learn more about it and know the functions and data it holds.

An example of image processing code for a QVWorker::iterate() function could be the following:

void iterate()
        {
        QVImage<uChar,1> inputImage = getPropertyValue< QVImage<uChar,1> >("Input image");

        QVImage<uChar,1> outputImage(inputImage.getCols(), inputImage.getRows());
        FilterMedian(inputImage, outputImage, 15, 15);

        setPropertyValue< QVImage<uChar,1> >("Output image", outputImage); 
        }
This iterate() function will apply a median filtering over the input image, and return the resulting filtered image in the property Output image.

Parallel programming with QVision

QVision introduces a simple and flexible framework structure for application design, that allows the user of the library to easily develop multi-threaded applications, with synchronized and thread-safe data sharing, without using of thread locks, semaphores, or any other typical complication of clasic parallel programming, and avoiding dead locks, and other problems of multi-threaded programming.

In section QVision's basic programming model. it was shown a programming model for QVision. It was a monolitic model, in the sense that all the computer vision and image processing was done in the same task. In computer vision and image processing, that task usually can be decomposed in different subtasks. For example the next figure depicts a more specific, but very common computer vision application structure:

computerVisionSystem.png

Generally computer vision applications may vary from that schema, but it can be a common guideline. As for image and video processing oriented applications, that schema can be simplified, generally eliminating feature extraction and matching tasks. The following schema corresponds to a real application, with three different tasks, that is one of the example programs included in the QVision, the rotoscoper:

rotoscoperDataFlow.png

It is a pure filtering application, that mixes component tree pruning filtering and canny operator to get rotoscoped-like effect over the frames of the input video. Component tree prunning is done in the Component tree filtering task, which sends the resulting image to the Canny operator task, and to the Contour painter task, that paints Canny contours over the pruned image.

Each different subtask or task is prone to be conveniently mapped onto a different thread, because different tasks work over separable data, and the volume of data processed by a computer vision task is commonly high enough to dedicate a thread to process it. That is why QVWorker class inherits from Qt's QThread class. So, worker objects can be considered as threads, that just call their iterate() function time and again.

By just breaking a task into several subtasks, and programming each task in a different worker, we are doing parallel computing, because every worker will be executed in a different thread, that the operative system can map onto different cores or CPU's. Also it is convenient for workers to be executed in threads separated from the main application thread, to keep the GUI responsive when a call to a worker's iterate() function takes too long.

Data sharing and synchronization amongst QVWorkers with property holders.

When programming in parallel, in general synchronization is needed when a thread should wait to another thread to process some data. Thread safe data sharing is needed for threads to share coherent data, not corrupted or half processed. Both things are eased by the QVision library to the programmed thanks to dynamic properties, and with some tricking with the QVWorker::iterate() function.

As with cameras and video output widgets, workers can link properties from one to another, with the function QVPropertyContainer::linkProperty. This function stablish a dependency between a source and a destiny properties in the same or different workers, such as when the source property changes its value to a new one, destination property will automatically set its value to the same value.

This value connection is thread-safe, so it can be used to share data thread-safely amongst worker objects. Futhermore, this linking can be set to be asynchronous or synchronous, through the fourth parameter of the function QVPropertyContainer::linkProperty. Setting it to SynchronousLink when the property holder is a QVWorker has two implications. First, for the source dynamic property for the linking, setting a new value for it doesn't inmediately stores the new value in the property, delaying that action until the actual call to QVWorker::iterate emmited finishes.

Second, the worker containing the source property and the one containing the destination property synchronize their iterate function calls, in a way that every call to iteration in the latter will not start until the last emitted iteration in the former finishes.

By this way, you can create thead-safe data sharing threads, with iterate functions synchronized when needed, and if the data path between threads is acyclic, it is also ensured that this model will not produce dead locks due to multithreading data sharing.

QVision's user interface

QVision is devoted to fast computer vision and image processing research and prototyping. Thus, it must provide:

Command line parameters.

The QVApplication object process the command line parameters, letting the user specify startup values for the algorithm parameters, contained in the the registered workers. Any input parameter of of type int, double, bool, and QString (which are considered to be algorithm parameters) for any worker created after the QVApplication object and before the call to exec() function can be initialized through the command line. Also, cameras contain properties suitable to be stablished in the command line, for example Cols, Rows which configure the dimentions of the image read by the camera, and mostly important, URL parameter, that contains an URL string identifying the input for the video (a video file, the name of a camera device, etc...).

Using predefined command line parameter --help with any program built upon the QVision library shows the parameter list the command understands. An example of the response to that command provided by the canny example program follows:

$ ./canny --help
Usage: ./canny [OPTIONS]
Example program for QVision library. Gets component tree from images.

Input parameters for Video:
  --Rows=[int] (def. 0) .............................. Rows to open the camera. 
  --Cols=[int] (def. 0) ........................... Columns to open the camera. 
  --RealTime=[true,false](def. false)  If the camera should be opened in real time mode. 
  --Deinterlaced=[true,false](def. false)  If the camera should be opened in deinterlaced mode. 
  --NoLoop=[true,false](def. false)  If the camera should be opened in no loop mode. 
  --RGBMEncoder=[true,false](def. false)  If the camera should be opened in RGB using mencoder. 
  --URL=[text] ............................ URL of the video source (see QVMPlayerCamera documentation for syntax). 

Todo:
Arreglar esto para que salgan los parĂ¡metros del worker CannyWorker. Ahora no se muestran correctamente al hacer --help.
You can see that commands should be specified with the following sintax:

--<property name>=<value>

using the name of the corresponding property, embraced by double quotes if it contains spaces or an equal sign (=), followed by an equal sign, and the value for the property. true or false are valid values for boolean properties, non-decimal numbers for integer properties, and any number for double properties.

You can consult input parameters for QVCameras in the detailed section for class QVMPlayerCamera.


Generated on Thu Mar 13 19:18:17 2008 for QVision by  doxygen 1.5.3