Smart Human-in-the-loop Segmentation

Bioimaging techniques yield deeper insights into the molecular and cellular processes of the human body than ever before. The procedure of identifying and detecting the boundaries of specific areas of interest such as single cells within a microscopic imaging is called segmentation. In recent years, image segmentation based on deep learning has achieved great success in biomedical research. The algorithm usually learns via a large amount of “ground truth” – training data annotated by humans. This supervised learning method requires researchers to label a large amount of data. Even though unsupervised methods that can train deep learning models without human annotation exist, their performance is often less accurate than that of supervised methods – and thus not sufficient for biomedical research. That is why scientist working on the project »Smart Human-in-the-loop Segmentation« aim to develop a powerful deep learning model that is trained with a minimum amount of human effort.

Open source deep learning workflow enables more efficient image segmentation

The overall method generalises and further develops the iterative deep learning workflow of the Allen Cell and Structure Segmenter. The mentioned Segmenter is a toolkit for the 3D segmentation of intracellular structures in fluorescence microscope images, co-developed by Dr Jianxu Chen and the Allen Institute for Cell Science. The first step of the »Smart Human-in-the-loop Segmentation« is the development of effective strategies to automatically select representative samples for the initial curation or annotation based on the project team’s previous work on suggestive annotation, a deep active learning framework that combines a fully convolutional network with active learning. Next, a few samples will be selected for a rough manual annotation by the researchers. These are used for the initial training of the algorithm for the segmentation of images, which will then be applied to all images in the data set. After that, a number of images will be revised manually and entered into the workflow to train the algorithm.

For each iteration, the researchers are going to develop methods to enable the deep learning model to automatically detect potential errors that will then initiate human curation. Finally, effective active learning and continual learning methods will be developed to avoid so called catastrophic forgetting. Meaning the drastic loss of previously learned information after being trained on a new task, in the iterative deep learning workflow. This enables accurate image segmentation with a low amount of human intervention. Eventually, the developed algorithms will be open source and made available as a graphical user interface tool for the broader biomedical image analysis community.