A common task after image segmentation is object detection, classification, and tracking. Understanding the mechanisms of basic biological processes requires understanding the proliferation, location, and pathway of cells and subcellular bodies in molecular and cellular imaging.
For example, our understanding of chemotaxis in immunology comes from the study of the stepwise movement of neutrophil cells and experimental modifications that slow or impede cell movement.
Given the diversity and large numbers of cellular elements in complex biological processes, an automatic approach to detect, classify, and track objects of interest in cellular videos is crucial to advancing our understanding of fundamental biological processes.
Deep learning has changed how objects are computationally detected and tracked and has provided leaps in how cellular video is interpreted.
In addition to distinguishing between background and foreground, computer vision tasks involving visual information require understanding objects represented in an image and video. In cell microscopy, cell types and similar populations can be represented in multiple shapes, sizes, and forms.
Machine learning models must understand the intrinsic features that define different cellular populations to detect and track cell movement. Understanding the diverse representations that an individual object can have in visual information from different perspectives and opinions is important in detecting and appropriately identifying objects in visual data.
This often becomes more computationally difficult in medical imaging where a single field of view can contain many even hundreds of examples of objects of interest. Proper detection and registration of objects requires analysis of movement and trajectories at the population level.
In a recent development from the deep learning community, the task of object detection/instance segmentation is solved through a complex framework that includes multiple prediction phases and vertices responsible for generating proposals and defining bounding boxes and object labels.
Some well-known models include models such as RCNN, Mask-RCNN, and YOLO. These models are typically trained in a supervised learning framework with identical images and labels in the form of signal-defined bounding boxes.
In medical data, especially in cell imaging, the use of these advanced neural structures is typically limited by the lack of human instruction, variation in human nomenclature, and the large amount of relatively homogeneous objects in images, which makes human instruction tedious.
An alternative solution is proposed that relies on generalizing the pixel map output from semantic image segmentation and separates cases based on heuristics. Each segmentation can be classified into relevant cell types, although an important biological challenge in cell imaging is the challenge of segmenting large groups of often overlapping organisms.
In cases where cells are distinct from each other, semantic segmentation can be sufficient. However, when there are pairs of cells that overlap or groups of cells that are closely connected, even a few segmentation errors can lead to errors in classification and clustering of cellular states.
A wide range of segmentation methods have been proposed to overcome the challenge of separating close and overlapping objects in medical imaging. Some existing solutions are based on the cell shape assumption, where strategies such as Gaussian Laplacian, radial symmetry transform, and interobject distances are used to distinguish between individual instances.
Other researchers have experimented with feature-based or boundary-based approaches to distinguish between different objects. The wide variety of proposed techniques highlights the challenge of detecting cellular organisms, and each method has demonstrated impressive results in specific imaging media and domain-specific questions.
Video data provides additional temporal information that can be used for object detection and object tracking information even in frames that may be difficult to process independently.
The additional information can make tracking more difficult as well – especially when there are multiple objects in the same frame, and the need to marry each object with the same object in the previous or next frame to generate accurate tracks.
This task is usually solved under the framework of a linear sorting problem: matching a specified number of objects from one frame of a video to the same objects in the next frame. In this context, a cost matrix is determined based on how likely a pair of cells are in two image frames of the same cell.
This matrix is usually determined/calculated based on factors including distance between locations, similarity of appearance, and surrounding environment.
Although these elements are usually selected and weighted empirically in the final cost matrix, there are recent works that apply neural network-based methods to improve the formation of the cost matrix.
Additional optimization can also be performed after generating an initial set of paths from frame-to-frame matches. This step can help mitigate partitioning errors and also take into account events such as cell merging and splitting.
Additionally, deep learning can help researchers explore the complex relationship between visual phenomena and genes. Integrative neural networks provide a powerful and unbiased tool to organize and quantify complex morphological properties of cells from optical data.
Morphological and morphodynamic states are often highly correlated with gene expression – the relationship between optical phenomena, gene expression and functional states of cells can be easily extracted using an integrative network quantization tool.
Cellular videos enable a wide range of studies on time-related behavior in biological systems that cannot be performed using conventional microscopy.
There is an increasing trend toward recognizing that cellular systems are incredibly dynamic throughout the cell cycle and additional information can be gained in the dynamic analysis of cellular morphology.
In live cell imaging, tracking cell trajectories opens opportunities for detailed analysis of the dynamic state and temporal change of individual cells during development and immune processes.