Classes | |
class | QVApplication |
Main class for QVision applications. More... | |
class | QVCamera |
Base class for image and video input source objects. More... | |
class | QVGenericImage |
Base class for QVImage objects. More... | |
class | QVImage< Type, Channels > |
Image representation class for the QVision. More... | |
class | QVPropertyContainer |
Base class for dynamic property container objects. More... | |
class | QVWorker |
Base class for Worker threads. More... |
Thus, the structure of a QVision program will generally follow this schema:
Each square block from the figure is a task in the program. The class QVWorker was introduced in the QVision library to model the Computer vision / Image processing algorithms block. It should contain the code that implements those algorithms, read its inputs from devices connected to the computer, video files, or the user interface, and share the results with the rest of the tasks in the program.
An example code for a QVision application was shown in section The first program. The main() function was the following:
int main(int argc, char *argv[]) { // QVApplication object QVApplication app(argc, argv, "Example program for QVision library. Play a video on a canvas window."); QVMPlayerCamera camera("Video"); // QVCamera object PlayerWorker worker("Player worker"); // QVWorker object camera.link(&worker,"Input image"); // GUI object QVGUI interface; // Video/image output object QVImageCanvas imageCanvas("Test image"); imageCanvas.linkProperty(worker,"Output image"); return app.exec(); }
The class QVMPlayerCamera is a subclass for QVCamera. Both represent video or image inputs in the QVision framework, but the class QVCamera is virtual, and every class modelling video or image inputs in the QVision should inherit from it. Actually, it only has one subclass, the QVMPlayerCamera class, which is a flexible video input that can read frames from diverse video file formats, webcams and digital video cameras. It is based on the MPlayer program. To allow a worker object to read frames from a video source, it should be linked to the camera object, specifying the name of the image for the worker, with a line like this:
camera.link(&worker,"Input image");
The object QVApplication registers worker and camera objects in the system, and process command line input parameters. It inherits from the QApplication class, so it has a similar functionallity. It is the most important object in a QVision application, so it always should be created first in the main() function, before any worker, camera or graphic interface object. Only one object derived from this class should be created in a QVision application.
The class QVGUI is the main GUI widget. This object reads the workers and cameras registered in the QVApplication object, so it contains buttons and sliders to control the algorithm parameters (computer vision or video processing algorithms parameters) for the system, and the video input flow, allowing the user to stop an resume frame read from the video sources opened by the program, in execution time. Also, it can display information about cpu usage stadistics, frame rate, seconds read from the video inputs, etc... and it allows the user to stop and close the program. It is recommended to create only one object of its type for any QVision application.
The class QVImageCanvas is a video or image output widget. Any object created with it will display an image window showing an output image from the worker object. Like the camera object, it should be connected to the worker to read the resulting images, specifying the name of the output image with a line like this:
imageCanvas.linkProperty(worker,"Output image");
int main(int argc, char *argv[]) { // QVApplication object QApplication *app = new QVApplication(argc, argv, "Example program for QVision library. Play a video on a canvas window."); [...} return app->exec(); // this will execute QApplication::exec() function, not QVApplication::exec() function. }
For example, the graphical user interface and the QVApplication objects were designed to allow the user to inspect and modify the algorithm parameters in execution time, by sliders and buttons, and by the command line interface respectively. Thus those objects should be able to find those parameters in the worker object, but it is not desirable to change them for every possible worker programmed.
Dynamic properties are an homogeneous way of sharing data between objects, without creating design dependencies between them. They are like common object properties, in the sense that they can be read or modified in execution time, but they also can be added or deleted from the objects, and be discovered by other objects in execution time, without knowing the class of the object or any other information about it, except that it can contain dynamic properties.
Workers should share their inputs (algorith parameters) using dynamic properties, so the QVGUI and the QVApplication objects can modify and read them in real time. Also, it is desirable for reasons that we will see in section Parallel programming with QVision, that workers share their output data (images, or other data structures) and image inputs through dynamic properties.
So, when linking a worker to a camera or a video output widget, actually a dynamic property from the worker and a dynamic property for the camera or the widget are being linked, telling the program that whenever a worker reads or writes an image, it actually reads it from the camera, or writes it to the video output widget. You can read more about property containers in the QVPropertyContainer class reference page. Every object capable of having dynamic properties is derived from that class.
This function should read input images, parameters, and other data inputs from dynamic properties contained in the worker, process them, and store the resulting images, data structures, or values, in other dynamic properties contained in the worker object.
An example code for a QVWorker class was shown in section TheFirstProgram:
class PlayerWorker: public QVWorker { public: PlayerWorker(QString name): QVWorker(name) { addProperty< QVImage<uChar,1> >("Input image", inputFlag|outputFlag); addProperty< QVImage<uChar,1> >("Output image", outputFlag); } void iterate() { QVImage<uChar,1> image = getPropertyValue< QVImage<uChar,1> >("Input image"); // image processing / computer vision code should go here setPropertyValue< QVImage<uChar,1> >("Output image", image); } };
This worker has no input parameters, and it just reads an input image, and stores it in an output property.
Second is the package QVDTA. It is an acronym that stands for QVision Data Types and Algorithms. It contains mostly Computer Vision algorithms and data types, built over the functions in the package qvipp, and other algorithms not directly implemented by the IPP library. You can check the API of the Computer Vision data types and algorithms. package, to learn more about it and know the functions and data it holds.
An example of image processing code for a QVWorker::iterate() function could be the following:
void iterate() { QVImage<uChar,1> inputImage = getPropertyValue< QVImage<uChar,1> >("Input image"); QVImage<uChar,1> outputImage(inputImage.getCols(), inputImage.getRows()); FilterMedian(inputImage, outputImage, 15, 15); setPropertyValue< QVImage<uChar,1> >("Output image", outputImage); }
In section QVision's basic programming model. it was shown a programming model for QVision. It was a monolitic model, in the sense that all the computer vision and image processing was done in the same task. In computer vision and image processing, that task usually can be decomposed in different subtasks. For example the next figure depicts a more specific, but very common computer vision application structure:
Generally computer vision applications may vary from that schema, but it can be a common guideline. As for image and video processing oriented applications, that schema can be simplified, generally eliminating feature extraction and matching tasks. The following schema corresponds to a real application, with three different tasks, that is one of the example programs included in the QVision, the rotoscoper:
It is a pure filtering application, that mixes component tree pruning filtering and canny operator to get rotoscoped-like effect over the frames of the input video. Component tree prunning is done in the Component tree filtering task, which sends the resulting image to the Canny operator task, and to the Contour painter task, that paints Canny contours over the pruned image.
Each different subtask or task is prone to be conveniently mapped onto a different thread, because different tasks work over separable data, and the volume of data processed by a computer vision task is commonly high enough to dedicate a thread to process it. That is why QVWorker class inherits from Qt's QThread class. So, worker objects can be considered as threads, that just call their iterate() function time and again.
By just breaking a task into several subtasks, and programming each task in a different worker, we are doing parallel computing, because every worker will be executed in a different thread, that the operative system can map onto different cores or CPU's. Also it is convenient for workers to be executed in threads separated from the main application thread, to keep the GUI responsive when a call to a worker's iterate() function takes too long.
As with cameras and video output widgets, workers can link properties from one to another, with the function QVPropertyContainer::linkProperty. This function stablish a dependency between a source and a destiny properties in the same or different workers, such as when the source property changes its value to a new one, destination property will automatically set its value to the same value.
This value connection is thread-safe, so it can be used to share data thread-safely amongst worker objects. Futhermore, this linking can be set to be asynchronous or synchronous, through the fourth parameter of the function QVPropertyContainer::linkProperty. Setting it to SynchronousLink when the property holder is a QVWorker has two implications. First, for the source dynamic property for the linking, setting a new value for it doesn't inmediately stores the new value in the property, delaying that action until the actual call to QVWorker::iterate emmited finishes.
Second, the worker containing the source property and the one containing the destination property synchronize their iterate function calls, in a way that every call to iteration in the latter will not start until the last emitted iteration in the former finishes.
By this way, you can create thead-safe data sharing threads, with iterate functions synchronized when needed, and if the data path between threads is acyclic, it is also ensured that this model will not produce dead locks due to multithreading data sharing.
Using predefined command line parameter --help with any program built upon the QVision library shows the parameter list the command understands. An example of the response to that command provided by the canny example program follows:
$ ./canny --help Usage: ./canny [OPTIONS] Example program for QVision library. Gets component tree from images. Input parameters for Video: --Rows=[int] (def. 0) .............................. Rows to open the camera. --Cols=[int] (def. 0) ........................... Columns to open the camera. --RealTime=[true,false](def. false) If the camera should be opened in real time mode. --Deinterlaced=[true,false](def. false) If the camera should be opened in deinterlaced mode. --NoLoop=[true,false](def. false) If the camera should be opened in no loop mode. --RGBMEncoder=[true,false](def. false) If the camera should be opened in RGB using mencoder. --URL=[text] ............................ URL of the video source (see QVMPlayerCamera documentation for syntax).
--<property name>=<value>
using the name of the corresponding property, embraced by double quotes if it contains spaces or an equal sign (=), followed by an equal sign, and the value for the property. true or false are valid values for boolean properties, non-decimal numbers for integer properties, and any number for double properties.
You can consult input parameters for QVCameras in the detailed section for class QVMPlayerCamera.