PARP Research Group University of Murcia, Spain


User interface

Graphical user interface widgets

Creating graphical user interfaces in applications developed over the QVision is quite simple. The framework offers a set of classes, contained in the Graphical User Interface group, to create graphical widgets easily integrable with any previously created processing block structure, composed by several workers and other kind of input/output blocks.

These widgets can discover dynamic properties contained in the worker objects thanks to the self introspection technique, and offer different sliders, buttons, checkboxes and other means for the user to inspect and modify their values at execution time. Generally the widgets offered in the QVision will offer such a possibility with dynamic properties of common or scalar types, such as integer, double, character string, etc...

The developer of a QVision application or a worker class should take care of storing the values of the parameters of the algorithms he or she wants to be modificable by the user at execution time in dynamic properties contained in the worker object.

The default graphical user interface

Class QVDefaultGUI can be used to create an interactive widget that will offer the user the possibility to control the execution of the different workers registered in the application, the flow of the input video camera objects, and modify the scalar input parameters defined and not linked at the workers (see DynamicProperties ).

This is a sample snapshot for a QVDefaultGUI window:

qvisioninterface_rotoscoper.png

To be correctly initialized a sole instance object of the QVDefaultGUI class should be created after the QVApplication object in the main function, and before the call to exec(). For a general usage of this class in a real QVision application, see ProgrammingModel section.

There are two widget areas in the QVDefaultGUI window:

  • Control area: contains a camera widget and a worker widget for each camera and worker object created in the application, respectively. These widgets can be used to stop, resume, and pause the execution and video input flow for the workers and the cameras of the application, amongst other things.
  • Workers input area: contains a set of widgets allowing the user to modify the values of the parameters for the workers created by the application.

Camera widgets contain buttons to control its input flow. An example of these widgets is depicted below:

qvisioninterface_rotoscoper_camera.png

Follows a description of the buttons and their functionality:

Pause button
qvisioninterface_pause.png
This button stops the camera from publishing new frames from the video source. If the camera is on real-time mode it will keep reading frames, but won't send them to the workers connected to it.

Resume button
qvisioninterface_play.png
This button resumes grabbing of the images that the registered workers read from the cameras.

Step button
qvisioninterface_step.png
When the camera is paused, this button makes it read the next frame in the video input, but keeps the camera paused.

Stop button
qvisioninterface_stop.png
This button stops the camera from reading frames from the input source.

Camera extended controls button
qvisioninterface_cameramaximize.png
This button opens a new window, with extended control functionality for the camera object, including buttons to control the playing speed. Also, it displays the following information: fps, reproduction speed, size of the video images, seconds elapsed (actual/total) and frames grabbed (read from the video source) / read (sent to the workers).
qvisioninterface_cameraextended.png

As for the cameras, every worker object registered in the system has its own widget in the interface. They contain several buttons that control its execution. An example of these widgets is depicted below:

qvisioninterface_rotoscoper_cannyworker.png

You can see a preview plot of CPU usage, and some buttons. Execution for the workers can be paused, resumed, stepped, and stopped with them, as if workers were cameras.

Note:
CPU stat plot depends on execution time between time flags, so when two or more workers compete for one CPU, times can differ than the used when each worker is run in its own CPU.
Follows a list with the detailed description of the buttons and their functionality:

Pause button
qvisioninterface_pause.png
This button suspends the processing of the worker. Every worker synchronized with it will be stopped as well, until the resume button is pushed.

Resume button
qvisioninterface_play.png
If the worker is paused, pushing this button resumes its execution.

Step button
qvisioninterface_step.png
This button can be pushed when a worker is paused to make it process it its inputs one time. The worker will still be paused after that.

Stop button
qvisioninterface_stop.png
This button finish the processing of the worker. Its properties will be unlinked and with a freezed value, so any worker connected to it will read the same values from then on.

CPU statistics button
qvisioninterface_cpustats.png
This button opens a detailed cpu usage plot for the worker. It shows a window with the cpu time statistical plot of the time flags defined in the iterate() function of the worker. You can see an example of this plot in the following figure:

qvisioninterface_cpustat_canny.png

Function QVWorker::timeFlag() can be used in the body of the QVWorker::iterate() function to configure a time flag in the execution of a worker.

On the left of the widget there is a space containing a tab for each worker created by the application. Each tab shows a set of widgets offering control for the worker's input parameters. The user can modify the values of these input parameters at will in execution time.

Each of these widgets is connected to each input property (see section DynamicProperties) from the set of created workers

The image canvas widget

Class QVImageCanvas can be used to create a widget that depicts a QVImage object.

This widget is a property container, and it will read the image to be displayed from an output dynamic property contained in another property container (normally a QVWorker object).

Usage is as follows: first create the QVImageCanvas object in the main() function of a program, then link a QVImage type property from a property holder:

[...]

CannyWorker cannyWorker("Canny");

[...]

QVImageCanvas imageCanvas;
imageCanvas.linkProperty(cannyWorker,"Canny image");

[...]

and latter call for the exec() function of the QVApplication object of the program. This will automatically create a window like the following:

qvimagecanvas_penguin.png

You can see it has a zoom number indicator (z = 1), horizontal and vertical rules (in pixels), and some buttons. These latter control zooming and moving around a zoomed area of the image mostly, and a detailed explanation of each one follows:

Zoom in button
qvimagecanvas_zoomin.png
This button zooms the image in. Canvas window doesn't change size, but the area of the image displayed becomes smaller by a factor of 4 (width and height gets divided by 2). The label of the left bottom corner displays the text
z=<zoom>
where <zoom> is a number indicating the zoom factor that divides width and height in the actual zoom level.

Zoom out button
qvimagecanvas_zoomout.png
This button divides zoom factor by 2, if it is equal or greater than 2. If the canvas window is bigger than the image at the final zoom, it is adjusted to the size of the latter.

Zoom restore button
qvimagecanvas_zoomrestore.png
This button sets zoom factor by 1, and resizes canvas window to its original size.

Select zoom region button
qvimagecanvas_zoomregion.png
This button lets you select a rectangle in the image to zoom, adjusting zoom factor and canvas window size to it.

Select polyline button
qvimagecanvas_polyline.png
This button lets you select points in the image. That points list can be get as the "poly select" imageCanvas's property. You can explain this button by pushing it for a few seconds, and you can select a polyline representation, a point list representation, or generate a circular polyline, by a click and drag mode.

Select ROI button
qvimagecanvas_roi.png
This button lets you select a rectangle in the image, that rectangle can be get as the "rect select" imageCanvas's property. And can be use to apply operations only to this rectangle.

Drag mode
qvimagecanvas_drag.png
This button activates drag mode. With it you can move around the image displayed in the canvas window, if the zoom factor forces the canvas window to show only a sub-region of the image, by holding click and dragging over the depicted image.

In the following figure you can see a zoomed canvas window depicting a sub-region of the original image:

qvimagecanvas_penguinpixels.png

Another interesting feature of image canvas is that at a zoom factor bigger or equal to 32, the canvas renders the gray-scale pixel value over every pixel if the image is gray-scale, or the three different values for each of the RGB channels over every pixel, if the image is RGB, as depicted below:

qvimagecanvas_penguinpixelnumbers.png

Numeric plot widget

Class QVNumericPlot shows a widget which will read a set of integer or double values and plot their evolution over the time of a worker execution.

The abscissa axis will display the number of iterations, or the number of seconds elapsed, depending on the boolean time parameter of the constructor QVNumericPlot::QVNumericPlot. The ordinate axis will scale to the maximum value of the integer or double values read from the worker (or workers) in the time interval displayed. One QVNumericPlot widget can read integer or double values from several workers, but they must be synchronized, or the QVNumericPlot should display the elapsed time in the abscissa axis.

To show the usage of the class, first we need a worker producer of the integer or double values. The following code is an example:

class MyWorker: public QVWorker
        {
        public:
                MyWorker(QString name): QVWorker(name)
                        {
                        addProperty<int>("Max pixel", outputFlag);
                        addProperty<int>("Min pixel", outputFlag);
                        [...]
                        }

                void iterate()
                        {
                        int     min, max;
                        [...]
                        // Calculate values for 'min' and 'max' variables.
                        [...]
                        setPropertyValue<int>("Max pixel", max);
                        setPropertyValue<int>("Min pixel", min);
                        [...]
                        }
        }

And then, in the main function, we can create a QVNumericPlot object and link it with those properties, like this:

#include <QVNumericPlot>

void main()
        {
        [...]
        MyWorker myWorker("worker");
        QVNumericPlot numericPlot("MinMax");
        numericPlot.linkProperty(myWorker, "Max pixel");
        numericPlot.linkProperty(myWorker, "Min pixel");
        [...]
        }

The result will be an application showing the following window widget:

qvision_numericplot_window.png

Histogram plot widget

Class QVHistogramPlot shows a widget which will read a QList<double> worker's output property, and plot their evolution over the time of a worker execution.

The ordinate axis will scale to the maximum value of the double values in the list read from the worker (or workers) in the time interval displayed. One QVHistogramPlot widget can read QList<double> values from several workers, but they must be synchronized if the iterations are based on worker's iterations.

To show the usage of the class, first we need a worker producer of the QList<double> values. The following code is an example:

class MyWorker: public QVWorker
        {
        public:
                MyWorker(QString name): QVWorker(name)
                        {
                        addProperty<QList<double> >("MinMaxList", outputFlag);
                        addProperty<QList<double> >("FirstRow", outputFlag);
                        [...]
                        }

                void iterate()
                        {
                        QList<double> minmaxlist;
                        QList<double> firstrow;
                        [...]
                        // Calculate values for 'minmaxlist' and 'firstrow' variables.
                        [...]
                        setPropertyValue<QList<double> >("MinMaxList", minmaxlist);
                        setPropertyValue<QList<double> >("FirstRow", firstrow);
                        [...]
                        }
        }

And then, in the main function, we can create a QVHistogramPlot and link it with that property, like this:

#include <QVHistogramPlot>

void main()
        {
        [...]
        MyWorker myWorker("worker");
        QVHistogramPlot histPlot("histFirstRow", false, 10, 300);
        histPlot.linkProperty(myWorker, "FirstRow");
        [...]
        }

The result will be an application showing the following window widget:

qvision_histogramplot_window.png

CPU performance plot widget

The class QVCPUPlot can be used to display the CPU usage statistics of a worker. The developer of a worker class can divide the processing of each call to the QVWorker::iterate() function in a set of different time stages using calls to the QVWorker::timeFlag() function. For example, the following code of the iterate method of a worker:

MyWorker::iterate()
        {
        [...]
        timeFlag("Read parameters");
        [...]
        timeFlag("Call to getComponentTree for low areas");
        [...]
        timeFlag("Prune low areas from image");
        [...]
        }

Will stablish some performance breakpoints in the function. The QVWorker::timeFlag() function registrates the time elapsed between each two of those break points, and stores some time statistics in the worker, which can be later displayed using the widget QVCPUPlot. For example, the following main function:

void main()
        {
        [...]
        MyWorker myWorker("name");
        [...]
        QVCPUPlot cpuPlot("CPU Plot", true, 10);
        cpuPlot.linkProperty(myWorker); 
        }

Will create in execution time the following window, displaying time statistics for the different time segments specified with the QVWorker::timeFlag() method:

componenttree_cpustat.png

Command line parameters in QVision applications

Dynamic properties of certain types, contained in workers or camera objects in an application are authomatically detected by the QVApplication object before the call to the QVApplication::exec() method, again thanks to their self introspection quality. The QVApplication object also parses the input console command line used to execute the application. By doing both things, this object allows the user to stablish initial values for those input dynamic properties through parameters in the command line. These command line assignable dynamic properties line must be input properties of type integer (int), boolean (bool), or character strings (QString).

Using the --help command line parameter, every QVision application displays an usage reference text, including a description of the command line assignable dynamic properties detected in the application, including their valid value range, their default value, and a short description for them. The properties are displayed grouped with other properties contained in the same block, or worker object.

For example, the rotoscoper example application can be executed with the following command line:

 ./rotoscoper --help 

Which will make it display the following message through the command line:

Usage: ./rotoscoper [OPTIONS]
Composes component tree image filtering and canny operator for making animation like images from real images.

Input parameters for Video:
  --Rows=[int] (def. 0)                                                      Rows to open the camera.
  --Cols=[int] (def. 0)                                                   Columns to open the camera.
  --RealTime=[true,false](def. false)               If the camera should be opened in real time mode.
  --Deinterlaced=[true,false](def. false)        If the camera should be opened in deinterlaced mode.
  --NoLoop=[true,false](def. false)                   If the camera should be opened in no loop mode.
  --URL=[text] (def. '')                                                     URL of the video source.

Input parameters for Canny operator:
  --max worker iterations=[int] (def. -1)  Stablishes maximal number of iterations to execute worker.
  --stats enabled=[true,false] (def. true)      Stablishes if the worker's cpu stats will be enabled.
  --Threshold high=[50...1000] (def. 150)                          High threshold for Canny operator.
  --Threshold low=[10...500] (def. 50)                              Low threshold for Canny operator.

Input parameters for Contour painter:
  --max worker iterations=[int] (def. -1)  Stablishes maximal number of iterations to execute worker.
  --stats enabled=[true,false] (def. true)      Stablishes if the worker's cpu stats will be enabled.

Input parameters for Component Tree:
  --max worker iterations=[int] (def. -1)  Stablishes maximal number of iterations to execute worker.
  --stats enabled=[true,false] (def. true)      Stablishes if the worker's cpu stats will be enabled.
  --Maximal area to prune=[5...10000] (def. 50)  Maximal size of the areas to be pruned in the image.

We can see for example that the worker Component Tree includes an input property named Maximal area to prune, of integer type. This property has a valid range of values between the 5 and 10000, and its default value is 50. Along with a short text definition for the property (Maximal size of the areas to be pruned in the image), these characteristics are specified in the call to the method QVPropertyContainer::addProperty which adds the property in the constructor of the worker object:

class ComponentTreeWorker: public QVWorker
        {
        public:
                ComponentTreeWorker(QString name): QVWorker(name)
                        {
                        addProperty<int>("Maximal area to prune", inputFlag, 50,
                                "Maximal size of the areas to be pruned in the image", 5, 10000);
                        [...]
                        }
        [...]
        }

The following command line starts the rotoscoper application, specifying some initial values for the property URL in the camera object, and the property Maximal area to prune in the Component Tree object:

 ./rotoscoper --URL=http://perception-inf.um.es/videos/misc/penguin.dv --"Maximal area to prune"=100 

The command line parameters specifying values for the block properties must be separated by spaces in the command line. They must start with a double dash, followed by the name of the property, which must be quoted if it contains spaces. Next must follow an = (equal sign), and the value we want to initially store in the property.

Optionally, the name of the block or worker can be specified in the parameter, between the double dash and the property name, to resolve name conflicts, when two workers have a property referenced with the same name. For example, the previous command would be equivalent to the following:

 ./rotoscoper --Video:URL=http://perception.inf.um.es/public_data/videos/misc/penguin.dv --"Component Tree":"Maximal area to prune"=100 



QVision framework. PARP research group, copyright 2007, 2008.