This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow. More...

Inheritance diagram for Tracking:

Public Slots
virtual void	startProcess ()
	Initializes a tracking analysis and triggers its execution. Constructs from the path to a folder where the image sequence is stored, detects the image format and processes the first image to detect objects. First, it computes the background by averaging images from the sequence, then it subtracts the background from the first image and then binarizes the resulting image. It detects the objects by contour analysis and extracts features by computing the object moments. It triggers the analysis of the second image of the sequence.

void	updatingParameters (const QHash< QString, QString > &)
	Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface.

virtual void	imageProcessing ()
	Processes an image from an images sequence and tracks and matchs objects according to the previous image in the sequence. Takes a new image from the image sequence, substracts the background, binarises the image and crops according to the defined region of interest. Detects all the objects in the image and extracts the object features. Then matches detected objects with objects from the previous frame. This function emits a signal to display the images in the user interface.

Signals
void	progress (int)
	Emitted when an image is processed.

void	backgroundProgress (int)
	Emitted when an image to compute the background is processed.

void	finishedProcessFrame ()
	Emitted when the first image has been processed to trigger the starting of the analysis.

void	finished ()
	Emitted when all images have been processed.

void	forceFinished (QString message)
	Emitted when a crash occurs during the analysis.

void	statistic (long long int time)
	Emitted at the end of the analysis.

Public Member Functions
	Tracking (QWidget *parent=nullptr)

	Tracking (const string &path, const string &background, int startImage=0, int stopImage=-1)
	Constructs the tracking object from a path to an image sequence and an optional path to a background image.

	Tracking (const string &path, const UMat &background, int startImage=0, int stopImage=-1)
	Constructs the tracking object from a list of path, a background image and a range of image.

	Tracking (const Tracking &)=delete

Tracking &	operator= (const Tracking &T)=delete

Tracking &	operator= (Tracking &&T)=delete

	Tracking (Tracking &&T)=delete

	~Tracking ()
	Destructs the tracking object.

Point2d	curvatureCenter (const Point3d &tail, const Point3d &head) const
	Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object.

double	curvature (Point2d center, const Mat &image) const
	Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse.

double	divide (double a, double b) const
	Computes the float division and handle the division by 0 by returning 0.

bool	objectDirection (const UMat &image, vector< double > &information) const
	Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection.

vector< double >	objectInformation (const UMat &image) const
	Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation.

vector< Point3d >	reassignment (const vector< Point3d > &past, const vector< Point3d > &input, const vector< int > &assignment) const
	Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i].

vector< vector< Point3d > >	objectPosition (const UMat &frame, int minSize, int maxSize) const
	Computes the positions of the objects and extracts the object's features.

vector< int >	costFunc (const vector< vector< Point3d > > &prevPos, const vector< vector< Point3d > > &pos, double LENGHT, double ANGLE, double LO, double AREA, double PERIMETER) const
	Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem".

void	cleaning (const vector< int > &occluded, vector< int > &lostCounter, vector< int > &id, vector< vector< Point3d > > &input, double param_maximalTime) const
	Cleans the data if an object is lost more than a certain time.

vector< Point3d >	prevision (vector< Point3d > past, vector< Point3d > present) const
	Predicts the next position of an object from the previous position.

vector< int >	findOcclusion (vector< int > assignment) const
	Finds the objects that are occluded during the tracking.

Static Public Member Functions
static double	modul (double angle)
	Computes the usual mathematical modulo 2*PI of an angle.

static double	angleDifference (double alpha, double beta)
	Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention.

static UMat	backgroundExtraction (VideoReader &video, int n, const int method, const int registrationMethod)
	Computes the background of an image sequence by averaging n images.

static void	registration (UMat imageReference, UMat &frame, int method)
	Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation.

static void	binarisation (UMat &frame, char backgroundColor, int value)
	Binarizes the image by thresholding.

static bool	exportTrackingResult (const QString &path, const QSqlDatabase &db)
	Exports the tracking data from the database to a text file.

static bool	importTrackingResult (const QString &path, QSqlDatabase db)
	Imports the tracking data from a text file to the database.

Public Attributes
const QString	connectionName

UMat	m_binaryFrame

vector< vector< Point3d > >	m_out

vector< vector< Point3d > >	m_outPrev

Private Attributes
QSharedPointer< QElapsedTimer >	m_timer

bool	m_statusBinarisation

string	m_path

string	m_backgroundPath

UMat	m_background

int	m_displayTime

QString	m_savingPath

VideoReader *	video

int	m_im

QString	m_error

int	m_startImage

int	m_stopImage

Rect	m_ROI

QFile	m_logFile

vector< cv::String >	m_files

vector< int >	m_id

vector< int >	m_lost

int	m_idMax

bool	m_isMorph

QHash< QString, QString >	m_parameters

Detailed Description

This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow.

Author: Benjamin Gallois

Version

Revision: 4.0

Contact: gallo.nosp@m.is.b.nosp@m.enjam.nosp@m.in08.nosp@m.@gmai.nosp@m.l.co.nosp@m.m

Constructor & Destructor Documentation

◆ Tracking() [1/2]

Tracking::Tracking	(	const string &	path,
		const string &	backgroundPath,
		int	startImage = 0,
		int	stopImage = -1 )

Constructs the tracking object from a path to an image sequence and an optional path to a background image.

Parameters

[in]	path	The path to a folder where images are stocked.
[in]	backgroundPath	The path to a background image.
[in]	startImage	Index of the beginning image.
[in]	stopImage	Index of the ending image.

◆ Tracking() [2/2]

Tracking::Tracking	(	const string &	path,
		const UMat &	background,
		int	startImage = 0,
		int	stopImage = -1 )

Constructs the tracking object from a list of path, a background image and a range of image.

Parameters

[in]	imagePath	List of path to the images.
[in]	background	A background image.
[in]	startImage	Index of the beginning image.
[in]	stopImage	Index of the ending image.

Member Function Documentation

◆ angleDifference()

double Tracking::angleDifference	(	double	alpha,
		double	beta )

static

Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention.

Parameters

[in]	alpha	Input angle.
[in]	beta	Input angle.

Returns: Least difference.

◆ backgroundExtraction()

UMat Tracking::backgroundExtraction	(	VideoReader &	video,
		int	n,
		const int	method,
		const int	registrationMethod )

static

Computes the background of an image sequence by averaging n images.

Parameters

[in]	VideoReader	A VideoReader object containing the movie.
[in]	n	The number of images to average to computes the background.
[in]	Method	0: minimal projection, 1: maximal projection, 2: average projection.
[in]	registrationMethod	Method of registration.
[in]	isError	Is a least one error as occured during the process.

Returns: The background image. TO DO: currently opening all the frames, to speed-up the process and if step is large can skip frames by replacing nextImage by getImage.

◆ backgroundProgress

void Tracking::backgroundProgress ( int )

signal

Emitted when an image to compute the background is processed.

Parameters

int	The number of processed image.

◆ binarisation()

void Tracking::binarisation	(	UMat &	frame,
		char	backgroundColor,
		int	value )

static

Binarizes the image by thresholding.

Parameters

[in,out]	frame	The image to binarize.
[in]	backgroundColor	If equals to 'w' the thresholded image will be inverted, if equal to 'b' it will not be inverted.
[in]	value	The value at which to threshold the image.

◆ cleaning()

void Tracking::cleaning	(	const vector< int > &	occluded,
		vector< int > &	lostCounter,
		vector< int > &	id,
		vector< vector< Point3d > > &	input,
		double	param_maximalTime ) const

Cleans the data if an object is lost more than a certain time.

Parameters

[in]	occluded	The vector with the index of object missing in the current image.
[in]	input	The vector at current image of size m <= n to be sorted.
[in]	lostCounter	The vector with the number of times each objects are lost consecutively.
[in]	id	The vector with the id of the objects.
[in]	param_maximalTime

Returns: The sorted vector.

◆ costFunc()

vector< int > Tracking::costFunc	(	const vector< vector< Point3d > > &	prevPos,
		const vector< vector< Point3d > > &	pos,
		double	LENGTH,
		double	ANGLE,
		double	LO,
		double	AREA,
		double	PERIMETER ) const

Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem".

Parameters

[in]	prevPos	The vector of objects parameters at the previous image.
[in]	pos	The vector of objects parameters at the current image that we want to sort in order to conserve objects identity.
[in]	LENGTH	The typical displacement of an object in pixels.
[in]	ANGLE	The typical reorientation angle in radians.
[in]	LO	The maximal assignment distance in pixels.

Returns: The assignment vector containing the new index position to sort the pos vector.

◆ curvature()

double Tracking::curvature	(	Point2d	center,
		const Mat &	image ) const

Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse.

Parameters

[in]	center	Center of the curvature.
[in]	image	Binary image CV_8U.

Returns: Radius of curvature.

◆ curvatureCenter()

Point2d Tracking::curvatureCenter	(	const Point3d &	tail,
		const Point3d &	head ) const

Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object.

Parameters

[in]	tail	The parameters of the tail ellipse: coordinate and direction of the major axis.
[in]	head	The parameters of the head ellipse: coordinate and direction of the major axis.

Returns: Coordinate of the curvature center.

◆ divide()

double Tracking::divide	(	double	a,
		double	b ) const

Computes the float division and handle the division by 0 by returning 0.

Parameters

[in]	a	Dividend.
[in]	a	Divisor.

Returns: Division result, 0 if b = 0.

◆ exportTrackingResult()

bool Tracking::exportTrackingResult	(	const QString &	path,
		const QSqlDatabase &	db )

static

Exports the tracking data from the database to a text file.

Parameters

[in]	path	The path to a folder where to write the text file.
[in]	db	The database where tracking results are stored already opened.

◆ findOcclusion()

vector< int > Tracking::findOcclusion ( vector< int > assignment ) const

Finds the objects that are occluded during the tracking.

Parameters

[in] assignment The vector with the new indexes that will be used to sort the input vector.

Returns: The vector with the indexes of occluded objects.

◆ importTrackingResult()

bool Tracking::importTrackingResult	(	const QString &	path,
		QSqlDatabase	db )

static

Imports the tracking data from a text file to the database.

Parameters

[in]	path	The path to a folder where to write the text file.
[in]	db	The database where tracking results are stored already opened.

◆ modul()

double Tracking::modul ( double angle )

static

Computes the usual mathematical modulo 2*PI of an angle.

Parameters

[in] angle Input angle.

Returns: Output angle.

◆ objectDirection()

bool Tracking::objectDirection	(	const UMat &	image,
		vector< double > &	information ) const

Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection.

Parameters

[in]	image	Binary image CV_8U.
[in,out]	information	The parameters of the object (x coordinate, y coordinate, orientation).

Returns: True if the direction angle is the orientation angle. False if the direction angle is the orientation angle plus pi.

◆ objectInformation()

vector< double > Tracking::objectInformation ( const UMat & image ) const

Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation.

Parameters

[in] image Binary image CV_8U.

Returns: The equivalent ellipse parameters: the object center of mass coordinate and its orientation.

Note: : This function computes the object orientation, not its direction.

◆ objectPosition()

vector< vector< Point3d > > Tracking::objectPosition	(	const UMat &	frame,
		int	minSize,
		int	maxSize ) const

Computes the positions of the objects and extracts the object's features.

Parameters

[in]	frame	Binary image CV_8U.
[in]	minSize	The minimal size of an object.
[in]	maxSize	The maximal size of an object.

Returns: All the parameters of all the objects formated as follows: one vector, inside of this vector, four vectors for parameters of the head, tail, body and features with number of object size. { { Point(xHead, yHead, thetaHead), ...}, Point({xTail, yTail, thetaHead), ...}, {Point(xBody, yBody, thetaBody), ...}, {Point(curvature, 0, 0), ...}}

◆ prevision()

vector< Point3d > Tracking::prevision	(	vector< Point3d >	past,
		vector< Point3d >	present ) const

Predicts the next position of an object from the previous position.

Parameters

past	The previous position parameters.
present	The current position parameters.

Returns: The predicted positions.

◆ progress

void Tracking::progress ( int )

signal

Emitted when an image is processed.

Parameters

int	Index of the processed image.

◆ reassignment()

vector< Point3d > Tracking::reassignment	(	const vector< Point3d > &	past,
		const vector< Point3d > &	input,
		const vector< int > &	assignment ) const

Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i].

Parameters

[in]	past	The vector at the previous image.
[in]	input	The vector at current image of size m <= n to be sorted.
[in]	assignment	The vector with the new indexes that will be used to sort the input vector.
[in]	lostCounter	The vector with the number of times each objects are lost consecutively.
[in]	id	The vector with the id of the objects.

Returns: The sorted vector.

◆ registration()

void Tracking::registration	(	UMat	imageReference,
		UMat &	frame,
		int	method )

static

Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation.

Parameters

[in]	imageReference	The reference image for the registration.
[in,out]	frame	The image to register.
[in]	method	The method of registration: 0 = simple (phase correlation), 1 = ECC, 2 = Features based.

◆ updatingParameters

void Tracking::updatingParameters ( const QHash< QString, QString > & parameterList )

slot

Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface.

Parameters

[in] parameterList The list of all the parameters used in the tracking.

Member Data Documentation

◆ m_background

UMat Tracking::m_background

private

Background image CV_8U.

◆ m_backgroundPath

string Tracking::m_backgroundPath

private

Path to an image background.

◆ m_binaryFrame

UMat Tracking::m_binaryFrame

Binary image CV_8U

◆ m_displayTime

int Tracking::m_displayTime

private

Binary image CV_8U.

◆ m_error

QString Tracking::m_error

private

QString containing unreadable images.

◆ m_files

vector<cv::String> Tracking::m_files

private

Vector containing the path for each image in the images sequence.

◆ m_id

vector<int> Tracking::m_id

private

Vector containing the objets Id.

◆ m_im

int Tracking::m_im

private

Index of the next image to process in the m_files list.

◆ m_isMorph

bool Tracking::m_isMorph

private

Is morphological application selected.

◆ m_logFile

QFile Tracking::m_logFile

private

Path to the file where to save logs.

◆ m_lost

vector<int> Tracking::m_lost

private

Vector containing the lost objects.

◆ m_out

vector<vector<Point3d> > Tracking::m_out

Objects information at iteration minus one

◆ m_outPrev

vector<vector<Point3d> > Tracking::m_outPrev

Objects information at current iteration

◆ m_parameters

QHash<QString, QString> Tracking::m_parameters

private

map of all the parameters for the tracking.

◆ m_path

string Tracking::m_path

private

Path to an image sequence.

◆ m_ROI

Rect Tracking::m_ROI

private

Rectangular region of interest.

◆ m_savingPath

QString Tracking::m_savingPath

private

Folder where to save files.

◆ m_startImage

int Tracking::m_startImage

private

Index of the next image to process in the m_files list.

◆ m_statusBinarisation

bool Tracking::m_statusBinarisation

private

True if wite objects on dark background, flase otherwise.

◆ m_stopImage

int Tracking::m_stopImage

private

Index of the next image to process in the m_files list.

◆ m_timer

QSharedPointer<QElapsedTimer> Tracking::m_timer

private

Timer that measured the time during the analysis execution.

The documentation for this class was generated from the following files:

src/tracking.h
src/tracking.cpp

Public Slots

Signals

Public Member Functions

Static Public Member Functions

Public Attributes

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ Tracking() [1/2]

◆ Tracking() [2/2]

Member Function Documentation

◆ angleDifference()

◆ backgroundExtraction()

◆ backgroundProgress

◆ binarisation()

◆ cleaning()

◆ costFunc()

◆ curvature()

◆ curvatureCenter()

◆ divide()

◆ exportTrackingResult()

◆ findOcclusion()

◆ importTrackingResult()

◆ modul()

◆ objectDirection()

◆ objectInformation()

◆ objectPosition()

◆ prevision()

◆ progress

◆ reassignment()

◆ registration()

◆ updatingParameters

Member Data Documentation

◆ m_background

◆ m_backgroundPath

◆ m_binaryFrame

◆ m_displayTime

◆ m_error

◆ m_files

◆ m_id

◆ m_im

◆ m_isMorph

◆ m_logFile

◆ m_lost

◆ m_out

◆ m_outPrev

◆ m_parameters

◆ m_path

◆ m_ROI

◆ m_savingPath

◆ m_startImage

◆ m_statusBinarisation

◆ m_stopImage

◆ m_timer