Bioimaging techniques yield deeper insights into the molecular and cellular processes of the human body than ever before. The procedure of identifying and detecting the boundaries of specific areas of interest such as single cells within a microscopic imaging is called segmentation. In recent years, image segmentation based on deep learning has achieved great success in biomedical research. The algorithm usually learns via a large amount of “ground truth” – training data annotated by humans. This supervised learning method requires researchers to label a large amount of data. Even though unsupervised methods that can train deep learning models without human annotation exist, their performance is often less accurate than that of supervised methods – and thus not sufficient for biomedical research. That is why scientist working on the project »Smart Human-in-the-loop Segmentation« aim to develop a powerful deep learning model that is trained with a minimum amount of human effort.
Open source deep learning workflow enables more efficient image segmentation
The overall method generalises and further develops the iterative deep learning workflow of the Allen Cell and Structure Segmenter. The mentioned Segmenter is a toolkit for the 3D segmentation of intracellular structures in fluorescence microscope images, co-developed by Dr Jianxu Chen and the Allen Institute for Cell Science. The first step of the »Smart Human-in-the-loop Segmentation« is the development of effective strategies to automatically select representative samples for the initial curation or annotation based on the project team’s previous work on suggestive annotation, a deep active learning framework that combines a fully convolutional network with active learning. Next, a few samples will be selected for a rough manual annotation by the researchers. These are used for the initial training of the algorithm for the segmentation of images, which will then be applied to all images in the data set. After that, a number of images will be revised manually and entered into the workflow to train the algorithm.
For each iteration, the researchers are going to develop methods to enable the deep learning model to automatically detect potential errors that will then initiate human curation. Finally, effective active learning and continual learning methods will be developed to avoid so called catastrophic forgetting. Meaning the drastic loss of previously learned information after being trained on a new task, in the iterative deep learning workflow. This enables accurate image segmentation with a low amount of human intervention. Eventually, the developed algorithms will be open source and made available as a graphical user interface tool for the broader biomedical image analysis community.
Share
Select publications
JMIR Serious Games, Vol. 12, 2024, P. e52785
Egger J, Gsaxner C, Luijten G, Chen J, Chen X, Bian J, Kleesiek J, Puladi B.
Is the Apple Vision Pro the Ultimate Display? A First Perspective and Survey on Entering the Wonderland of Precision Medicine
https://doi.org/10.2196/52785
Nature Methods, Vol. 21, No. 8, 2024, P. 1390-1393
Zhang S, Dai G, Huang T, Chen J.
Multimodal large language models for bioimage analysis
https://doi.org/10.1038/s41592-024-02334-2
Journal of Imaging Informatics in Medicine, 2024
Krieger K, Egger J, Kleesiek J, Gunzer M, Chen J.
Multisensory Extended Reality Applications Offer Benefits for Volumetric Biomedical Image Analysis in Research and Medicine
https://doi.org/10.1007/s10278-024-01094-x
2024 IEEE International Symposium on Biomedical Imaging (ISBI), 2024
Zhou Y, Zhao S, Sonneck J, Chen J.
2D Label-Free Prediction of Multiple Organelles Across Different Transmitted-Light Microscopy Images with Bag-of-Experts
https://doi.org/10.1109/ISBI56570.2024.10635298
2024 IEEE International Symposium on Biomedical Imaging (ISBI), 2024
Yao J, Hagemann N, Xiong Q, Chen J, Hermann DM, Chen C.
Topological Analysis of Mouse Brain Vasculature via 3d Light-Sheet Microscopy Images
https://doi.org/10.1109/ISBI56570.2024.10635226
Nature Methods, Vol. 21, No. 2, 2024, P. 182-194
Reinke A, Tizabi MD, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Kavur AE, Rädsch T, Sudre CH, Acion L, Antonelli M, Arbel T, Bakas S, Benis A,…
Understanding metric-related pitfalls in image analysis validation
https://doi.org/10.1038/s41592-023-02150-0
GigaScience, 2024
Sonneck J, Zhou Y, Chen J.
MMV_Im2Im: an open-source microscopy machine vision toolbox for image-to-image transformation
https://doi.org/10.1093/gigascience/giad120
Nature Methods, Vol. 21, No. 3, 2024, P. 368-369
Zhou Y, Cao J, Sonneck J, Banerjee S, Dörr S, Grüneboom A, Lorenz K, Zhang S, Chen J.
EfficientBioAI: making bioimaging AI models efficient in energy and latency
https://doi.org/10.1038/s41592-024-02167-z
Nature Communications, Vol. 14, 2023, P. 1-13
Cibir Z, Hassel J, Sonneck J, Kowitz L, Beer A, Kraus A, Hallekamp G, Rosenkranz M, Raffelberg P, Olfen S, Smilowski K, Burkard R, Helfrich I, Tuz…
ComplexEye:a multi-lens array microscope for high-throughput embedded immune cell migration analysis
https://doi.org/10.1038/s41467-023-43765-3
Nature Machine Intelligence, Vol. 5, 2023, P. 687-698
Jia Z, Chen J, Xu X, Kheir J, Hu J, Xiao H, Peng S, Hu X, Chen D, Shi Y.
The importance of resource awareness in artificial intelligence for healthcare
https://doi.org/10.1038/s42256-023-00670-0
Nature Methods, Vol. 20, 2023, P. 968-970
Chen J, Viana M, Rafelski S.
When seeing is not believing: application-appropriate validation matters for quantitative bioimage analysis
https://doi.org/10.1038/s41592-023-01881-4
Cell Reports Methods, Vol. 3, No. 3, 2023, P. 100436
Spangenberg P, Hagemann N, Squire A, Förster N, Krauß SD, Qi Y, Mohamud Yusuf A, Wang J, Grüneboom A, Kowitz L, Korste S, Totzeck M, Cibir Z, Tuz AA,…
Rapid and fully automated blood vasculature analysis in 3D light-sheet image volumes of different organs
https://doi.org/10.1016/j.crmeth.2023.100436
IEEE Transactions on Medical Imaging, Vol. 41, No. 10, 2022, P. 2582-2597
Liang P, Zhang Y, Ding Y, Chen J, Madukoma CS, Weninger T, Shrout JD, Chen DZ.
H-EMD: A Hierarchical Earth Mover's Distance Method for Instance Segmentation
https://doi.org/10.1109/TMI.2022.3169449
Immunological Reviews, Vol. 306, No. 1, 2021, P. 293-303
Grüneboom A, Aust O, Cibir Z, Weber F, Hermann DM, Gunzer M.
Imaging innate immunity
https://doi.org/10.1111/imr.13048