FastTrack  5.0.0
Tracks multiples objects dealing with occlusion and identities.
Public Slots | Signals | Public Member Functions | Static Public Member Functions | Public Attributes | Private Attributes | List of all members
Tracking Class Reference

This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow. More...

Inheritance diagram for Tracking:

Public Slots

void startProcess ()
 Initializes a tracking analysis and triggers its execution. Constructs from the path to a folder where the image sequence is stored, detects the image format and processes the first image to detect objects. First, it computes the background by averaging images from the sequence, then it subtracts the background from the first image and then binarizes the resulting image. It detects the objects by contour analysis and extracts features by computing the object moments. It triggers the analysis of the second image of the sequence.
 
void updatingParameters (const QMap< QString, QString > &)
 Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface. More...
 
void imageProcessing ()
 Processes an image from an images sequence and tracks and matchs objects according to the previous image in the sequence. Takes a new image from the image sequence, substracts the background, binarises the image and crops according to the defined region of interest. Detects all the objects in the image and extracts the object features. Then matches detected objects with objects from the previous frame. This function emits a signal to display the images in the user interface.
 

Signals

void progress (int)
 Emitted when an image is processed. More...
 
void backgroundProgress (int)
 Emitted when an image to compute the background is processed. More...
 
void finishedProcessFrame ()
 Emitted when the first image has been processed to trigger the starting of the analysis.
 
void finished ()
 Emitted when all images have been processed.
 
void forceFinished (QString message)
 Emitted when a crash occurs during the analysis.
 
void error (int code)
 Emitted when an error occurs.
 
void statistic (long long int time)
 Emitted at the end of the analysis.
 

Public Member Functions

 Tracking (string path, string background, int startImage=0, int stopImage=-1)
 Constructs the tracking object from a path to an image sequence and an optional path to a background image. More...
 
 Tracking (string path, UMat background, int startImage=0, int stopImage=-1)
 Constructs the tracking object from a list of path, a background image and a range of image. More...
 
 ~Tracking ()
 Destructs the tracking object.
 
Point2d curvatureCenter (const Point3d &tail, const Point3d &head)
 Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object. More...
 
double curvature (Point2d center, const Mat &image)
 Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse. More...
 
double divide (double a, double b)
 Computes the float division and handle the division by 0 by returning 0. More...
 
bool objectDirection (const UMat &image, vector< double > &information)
 Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection. More...
 
vector< double > objectInformation (const UMat &image)
 Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation. More...
 
vector< Point3d > reassignment (const vector< Point3d > &past, const vector< Point3d > &input, const vector< int > &assignment)
 Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i]. More...
 
UMat backgroundExtraction (VideoReader &video, int n, const int method, const int registrationMethod)
 Computes the background of an image sequence by averaging n images. More...
 
void registration (UMat imageReference, UMat &frame, int method)
 Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation. More...
 
void binarisation (UMat &frame, char backgroundColor, int value)
 Binarizes the image by thresholding. More...
 
vector< vector< Point3d > > objectPosition (const UMat &frame, int minSize, int maxSize)
 Computes the positions of the objects and extracts the object's features. More...
 
vector< int > costFunc (const vector< vector< Point3d >> &prevPos, const vector< vector< Point3d >> &pos, double LENGHT, double ANGLE, double LO, double AREA, double PERIMETER)
 Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem". More...
 
void cleaning (const vector< int > &occluded, vector< int > &lostCounter, vector< int > &id, vector< vector< Point3d >> &input, double param_maximalTime)
 Cleans the data if an object is lost more than a certain time. More...
 
vector< Point3d > prevision (vector< Point3d > past, vector< Point3d > present)
 Predicts the next position of an object from the previous position. More...
 
vector< Point3i > color (int number)
 Computes a random set of colors. More...
 
vector< int > findOcclusion (vector< int > assignment)
 Finds the objects that are occluded during the tracking. More...
 

Static Public Member Functions

static double modul (double angle)
 Computes the usual mathematical modulo 2*PI of an angle. More...
 
static double angleDifference (double alpha, double beta)
 Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention. More...
 

Public Attributes

UMat m_binaryFrame
 
UMat m_visuFrame
 
vector< vector< Point3d > > m_out
 
vector< vector< Point3d > > m_outPrev
 

Private Attributes

QElapsedTimer * timer
 
UMat m_background
 
bool statusBinarisation
 
VideoReadervideo
 
int m_im
 
QString m_error
 
int m_startImage
 
int m_stopImage
 
Rect m_ROI
 
QTextStream m_savefile
 
QFile m_outputFile
 
QFile m_logFile
 
vector< cv::String > m_files
 
vector< Point3i > m_colorMap
 
vector< vector< Point > > m_memory
 
vector< int > m_id
 
vector< int > m_lost
 
int m_idMax
 
string m_path
 
string m_backgroundPath
 
int m_displayTime
 
int param_n
 
int param_maxArea
 
int param_minArea
 
int param_spot
 
double param_len
 
double param_angle
 
double param_area
 
double param_perimeter
 
double param_lo
 
double param_to
 
int param_thresh
 
double param_nBackground
 
int param_methodBackground
 
int param_methodRegistrationBackground
 
int param_registration
 
int param_x1
 
int param_y1
 
int param_x2
 
int param_y2
 
int param_kernelSize
 
int param_kernelType
 
int param_morphOperation
 
QMap< QString, QString > parameters
 

Detailed Description

This class is intended to execute a tracking analysis on an image sequence. It is initialized with the path to the folder where images are stored. This class can be used inside an application by creating a new thread and calling the method startProcess. The tracking workflow can be changed by reimplementing the method startProcess and imageProcessing. This class can also be used as a library by constructing Tracking tracking("", "") to access the public class members and builds a new workflow.

Author
Benjamin Gallois
Version
Revision
4.0

Contact: gallo.nosp@m.is.b.nosp@m.enjam.nosp@m.in08.nosp@m.@gmai.nosp@m.l.co.nosp@m.m

Constructor & Destructor Documentation

◆ Tracking() [1/2]

Tracking::Tracking ( string  path,
string  backgroundPath,
int  startImage = 0,
int  stopImage = -1 
)

Constructs the tracking object from a path to an image sequence and an optional path to a background image.

Parameters
[in]pathThe path to a folder where images are stocked.
[in]backgroundPathThe path to a background image.
[in]startImageIndex of the beginning image.
[in]stopImageIndex of the ending image.

◆ Tracking() [2/2]

Tracking::Tracking ( string  path,
UMat  background,
int  startImage = 0,
int  stopImage = -1 
)

Constructs the tracking object from a list of path, a background image and a range of image.

Parameters
[in]imagePathList of path to the images.
[in]backgroundA background image.
[in]startImageIndex of the beginning image.
[in]stopImageIndex of the ending image.

Member Function Documentation

◆ angleDifference()

double Tracking::angleDifference ( double  alpha,
double  beta 
)
static

Computes the least difference between two angles, alpha - beta. The difference is oriented in the trigonometric convention.

Parameters
[in]alphaInput angle.
[in]betaInput angle.
Returns
Least difference.

◆ backgroundExtraction()

UMat Tracking::backgroundExtraction ( VideoReader video,
int  n,
const int  method,
const int  registrationMethod 
)

Computes the background of an image sequence by averaging n images.

Parameters
[in]filesList of paths to each image in the images sequence.
[in]nThe number of images to average to computes the background.
[in]Method0: minimal projection, 1: maximal projection, 2: average projection.
Returns
The background image. TO DO: currently opening all the frames, to speed-up the process and if step is large can skip frames by replacing nextImage by getImage.

◆ backgroundProgress

void Tracking::backgroundProgress ( int  )
signal

Emitted when an image to compute the background is processed.

Parameters
intThe number of processed image.

◆ binarisation()

void Tracking::binarisation ( UMat &  frame,
char  backgroundColor,
int  value 
)

Binarizes the image by thresholding.

Parameters
[in,out]frameThe image to binarize.
[in]backgroundColorIf equals to 'w' the thresholded image will be inverted, if equal to 'b' it will not be inverted.
[in]valueThe value at which to threshold the image.

◆ cleaning()

void Tracking::cleaning ( const vector< int > &  occluded,
vector< int > &  lostCounter,
vector< int > &  id,
vector< vector< Point3d >> &  input,
double  param_maximalTime 
)

Cleans the data if an object is lost more than a certain time.

Parameters
[in]occludedThe vector with the index of object missing in the current image.
[in]inputThe vector at current image of size m <= n to be sorted.
[in]lostCounterThe vector with the number of times each objects are lost consecutively.
[in]idThe vector with the id of the objects.
[in]param_maximalTime
Returns
The sorted vector.

◆ color()

vector< Point3i > Tracking::color ( int  number)

Computes a random set of colors.

Parameters
[in]numberThe number of colors to generate.
Returns
The vector containing the n colors.

◆ costFunc()

vector< int > Tracking::costFunc ( const vector< vector< Point3d >> &  prevPos,
const vector< vector< Point3d >> &  pos,
double  LENGTH,
double  ANGLE,
double  LO,
double  AREA,
double  PERIMETER 
)

Computes a cost function and use a global optimization association to associate targets between images. Method adapted from: "An effective and robust method for Tracking multiple fish in video image based on fish head detection" YQ Chen et al. Uses the Hungarian method implemented by Cong Ma, 2016 "https://github.com/mcximing/hungarian-algorithm-cpp" adapted from the Matlab implementation by Markus Buehren "https://fr.mathworks.com/matlabcentral/fileexchange/6543-functions-for-the-rectangular-assignment-problem".

Parameters
[in]prevPosThe vector of objects parameters at the previous image.
[in]posThe vector of objects parameters at the current image that we want to sort in order to conserve objects identity.
[in]LENGTHThe typical displacement of an object in pixels.
[in]ANGLEThe typical reorientation angle in radians.
[in]LOThe maximal assignment distance in pixels.
Returns
The assignment vector containing the new index position to sort the pos vector.

◆ curvature()

double Tracking::curvature ( Point2d  center,
const Mat &  image 
)

Computes the radius of curvature of the object defined as the inverse of the mean distance between each pixel of the object, and the center of the curvature. The center of curvature is defined as the intersection of the two minor axes of the head and tail ellipse.

Parameters
[in]centerCenter of the curvature.
[in]imageBinary image CV_8U.
Returns
Radius of curvature.

◆ curvatureCenter()

Point2d Tracking::curvatureCenter ( const Point3d &  tail,
const Point3d &  head 
)

Computes the center of the curvature, defined as the intersection of the minor axis of the head ellipse with the minor axis of the tail ellipse of the object.

Parameters
[in]tailThe parameters of the tail ellipse: coordinate and direction of the major axis.
[in]headThe parameters of the head ellipse: coordinate and direction of the major axis.
Returns
Coordinate of the curvature center.

◆ divide()

double Tracking::divide ( double  a,
double  b 
)

Computes the float division and handle the division by 0 by returning 0.

Parameters
[in]aDividend.
[in]aDivisor.
Returns
Division result, 0 if b = 0.

◆ findOcclusion()

vector< int > Tracking::findOcclusion ( vector< int >  assignment)

Finds the objects that are occluded during the tracking.

Parameters
[in]assignmentThe vector with the new indexes that will be used to sort the input vector.
Returns
The vector with the indexes of occluded objects.

◆ modul()

double Tracking::modul ( double  angle)
static

Computes the usual mathematical modulo 2*PI of an angle.

Parameters
[in]angleInput angle.
Returns
Output angle.

◆ objectDirection()

bool Tracking::objectDirection ( const UMat &  image,
vector< double > &  information 
)

Computes the direction of the object from the object parameter (coordinate of the center of mass and orientation). To use this function, the object major axis has to be the horizontal axis of the image. Therefore, it is necessary to rotate the image before calling objectDirection.

Parameters
[in]imageBinary image CV_8U.
[in,out]informationThe parameters of the object (x coordinate, y coordinate, orientation).
Returns
True if the direction angle is the orientation angle. False if the direction angle is the orientation angle plus pi.

◆ objectInformation()

vector< double > Tracking::objectInformation ( const UMat &  image)

Computes the equivalent ellipse of an object by computing the moments of the image. If the image is a circle, return nan as the orientation.

Parameters
[in]imageBinary image CV_8U.
Returns
The equivalent ellipse parameters: the object center of mass coordinate and its orientation.
Note
: This function computes the object orientation, not its direction.

◆ objectPosition()

vector< vector< Point3d > > Tracking::objectPosition ( const UMat &  frame,
int  minSize,
int  maxSize 
)

Computes the positions of the objects and extracts the object's features.

Parameters
[in]frameBinary image CV_8U.
[in]minSizeThe minimal size of an object.
[in]maxSizeThe maximal size of an object.
Returns
All the parameters of all the objects formated as follows: one vector, inside of this vector, four vectors for parameters of the head, tail, body and features with number of object size. { { Point(xHead, yHead, thetaHead), ...}, Point({xTail, yTail, thetaHead), ...}, {Point(xBody, yBody, thetaBody), ...}, {Point(curvature, 0, 0), ...}}

◆ prevision()

vector< Point3d > Tracking::prevision ( vector< Point3d >  past,
vector< Point3d >  present 
)

Predicts the next position of an object from the previous position.

Parameters
pastThe previous position parameters.
presentThe current position parameters.
Returns
The predicted positions.

◆ progress

void Tracking::progress ( int  )
signal

Emitted when an image is processed.

Parameters
intIndex of the processed image.

◆ reassignment()

vector< Point3d > Tracking::reassignment ( const vector< Point3d > &  past,
const vector< Point3d > &  input,
const vector< int > &  assignment 
)

Sorts a vector accordingly to a new set of indexes. The sorted vector at index i is the input at index assignment[i].

Parameters
[in]pastThe vector at the previous image.
[in]inputThe vector at current image of size m <= n to be sorted.
[in]assignmentThe vector with the new indexes that will be used to sort the input vector.
[in]lostCounterThe vector with the number of times each objects are lost consecutively.
[in]idThe vector with the id of the objects.
Returns
The sorted vector.

◆ registration()

void Tracking::registration ( UMat  imageReference,
UMat &  frame,
int  method 
)

Register two images. To speed-up, the registration is made in a pyramidal way: the images are downsampled then registered to have a an approximate transformation then upslampled to have the precise transformation.

Parameters
[in]imageReferenceThe reference image for the registration.
[in,out]frameThe image to register.
[in]methodThe method of registration: 0 = simple (phase correlation), 1 = ECC, 2 = Features based.

◆ updatingParameters

void Tracking::updatingParameters ( const QMap< QString, QString > &  parameterList)
slot

Updates the private members from the external parameters. This function links the tracking logic with the graphical user interface.

Parameters
[in]parameterListThe list of all the parameters used in the tracking.

Member Data Documentation

◆ m_background

UMat Tracking::m_background
private

Background image CV_8U.

◆ m_backgroundPath

string Tracking::m_backgroundPath
private

Path to an image background.

◆ m_binaryFrame

UMat Tracking::m_binaryFrame

Binary image CV_8U

◆ m_colorMap

vector<Point3i> Tracking::m_colorMap
private

Vector containing RBG color.

◆ m_displayTime

int Tracking::m_displayTime
private

Binary image CV_8U.

◆ m_error

QString Tracking::m_error
private

QString containing unreadable images.

◆ m_files

vector<cv::String> Tracking::m_files
private

Vector containing the path for each image in the images sequence.

◆ m_id

vector<int> Tracking::m_id
private

Vector containing the objets Id.

◆ m_im

int Tracking::m_im
private

Index of the next image to process in the m_files list.

◆ m_logFile

QFile Tracking::m_logFile
private

Path to the file where to save logs.

◆ m_lost

vector<int> Tracking::m_lost
private

Vector containing the lost objects.

◆ m_memory

vector<vector<Point> > Tracking::m_memory
private

Vector containing the last 50 tracking data.

◆ m_out

vector<vector<Point3d> > Tracking::m_out

Objects information at iteration minus one

◆ m_outPrev

vector<vector<Point3d> > Tracking::m_outPrev

Objects information at current iteration

◆ m_outputFile

QFile Tracking::m_outputFile
private

Path to the file where to save tracking data.

◆ m_path

string Tracking::m_path
private

Path to an image sequence.

◆ m_ROI

Rect Tracking::m_ROI
private

Rectangular region of interest.

◆ m_savefile

QTextStream Tracking::m_savefile
private

Stream to output tracking data.

◆ m_startImage

int Tracking::m_startImage
private

Index of the next image to process in the m_files list.

◆ m_stopImage

int Tracking::m_stopImage
private

Index of the next image to process in the m_files list.

◆ m_visuFrame

UMat Tracking::m_visuFrame

Image 8 bit CV_8U

◆ param_angle

double Tracking::param_angle
private

Maximal change in direction of an object between two images.

◆ param_area

double Tracking::param_area
private

Normalization area.

◆ param_kernelSize

int Tracking::param_kernelSize
private

Size of the kernel of the morphological operation.

◆ param_kernelType

int Tracking::param_kernelType
private

Type of the kernel of the morphological operation.

◆ param_len

double Tracking::param_len
private

Maximal length travelled by an object between two images.

◆ param_lo

double Tracking::param_lo
private

Maximal distance allowed by an object to travel during an occlusion event.

◆ param_maxArea

int Tracking::param_maxArea
private

Maximal area of an object.

◆ param_methodBackground

int Tracking::param_methodBackground
private

The method used to compute the background.

◆ param_methodRegistrationBackground

int Tracking::param_methodRegistrationBackground
private

The method used to register the images for the background.

◆ param_minArea

int Tracking::param_minArea
private

Minimal area of an object.

◆ param_morphOperation

int Tracking::param_morphOperation
private

Type of the morphological operation.

◆ param_n

int Tracking::param_n
private

Number of objects.

◆ param_nBackground

double Tracking::param_nBackground
private

Number of images to average to compute the background.

◆ param_perimeter

double Tracking::param_perimeter
private

Normalization perimeter.

◆ param_registration

int Tracking::param_registration
private

Method of registration.

◆ param_spot

int Tracking::param_spot
private

Which spot parameters are used to computes the cost function. 0: head, 1: tail, 2: body.

◆ param_thresh

int Tracking::param_thresh
private

Value of the threshold to binarize the image.

◆ param_to

double Tracking::param_to
private

Maximal time.

◆ param_x1

int Tracking::param_x1
private

Top x corner of the region of interest.

◆ param_x2

int Tracking::param_x2
private

Bottom x corner of the region of interest.

◆ param_y1

int Tracking::param_y1
private

Top y corner of the region of interest.

◆ param_y2

int Tracking::param_y2
private

Bottom y corner of the region of interest.

◆ parameters

QMap<QString, QString> Tracking::parameters
private

map of all the parameters for the tracking.

◆ statusBinarisation

bool Tracking::statusBinarisation
private

True if wite objects on dark background, flase otherwise.

◆ timer

QElapsedTimer* Tracking::timer
private

Timer that measured the time during the analysis execution.


The documentation for this class was generated from the following files: