Machine learning based medical image analysis for prognosis of brain disease course and therapy efficacy (J2-2500)

General information

Title
Machine learning based medical image analysis for prognosis of brain disease course and therapy efficacy
Period
Sep 1, 2020 -- Aug 31, 2023
Range
1.80 FTE
Activity
2.06 - Engineering Sciences and Technologies / Systems and Cybernetics / Medical informatics

Abstract

This project will directly address the priority field Research on neurodegenerative diseases (JPND) as set by the Slovenian research agency. Without loss of generality, this project will focus on multiple sclerosis (MS), since it involves two common pathologies also found in cerebrovascular disease, Alzheimer's and other dementias, depression, schizophrenia and bipolar disorder, etc. The two pathologies of interest are scar tissue or lesions and progressive loss of neurons or neurodegeneration, which typically are one of the few paraclinical symptoms that appear from several months up to a few years before any clinical symptoms can even be recognized. Magnetic resonance (MR) tomographic imaging is by far the most sensitive soft tissue imaging technique and therefore extensively used for the assessment of normal brain structure status and the detection of pathologic lesions. Based on brain MR scans the lesion accumulation and degree of neurodegeneration may be quantified in vivo. This is also the reason why information-rich brain MR images are being increasingly utilized in large-scale clinical trials, for instance, for testing new drugs and therapies the MR image based measures are already used as surrogates of clinical outcome measures. In this way, drug development proceeds faster and is thus less hazardous for the enrolled patients. Whereas clinical trials involve between-group comparisons, clinical management of individual MS patients is more challenging and the use of MR image based measures or imaging biomarkers has not yet proliferated. Main difficulties are high inter​ - and intra-​ - observer variability of visual assessment of the MR images and high variability in quality of standard-of-care MR images. The emerging technical solution leading to more objective and reliable quantitative assessment is a computational analysis of MR images. In this project, we will leverage advanced machine learning tools for MR image analysis with the goal to develop accurate, interpretable and robust prediction models that address the need for personalized early prognosis of disease course and treatment efficacy for MS and other neurodegenerative diseases from standard-of-care MR images. The proposed project has 8 deliverables, which involve (1) acquisition and annotation of brain and spinal cord MR images of MS patients and (2) collection of associated set of clinical, laboratory, gait, balance and self-reported outcome measures. For the standard-of- care MR images (3) automated pipeline including MR sequence identification and image quality assessment and (4) adversarial based image enhancement will be developed, followed by (5) feature-rich MR image description based on brain and spinal cord segmentation, radiomics and shape features, and autoencoder representations. Next, (6) based on features and enhanced MR images we will develop novel and improved prediction models, focusing also on interpretability and robustness. Besides (7) expected impactful research publications, the ultimate goal is to (8) develop, integrate and prospectively validate a decision support system for managing MS based on the state-of-the-art prediction models and verify its capabilities using routine, standard-of-care MR images. Preliminary results suggest that MR image based treatment efficacy prediction enables the clinicians to determine the optimal treatment within 1.2 years on average, compared to current average of 3.9 years based on using clinical outcome measures like the extended disability status score (EDSS) or relapse occurrence. Besides, through the search for the optimal prediction models we also aim to elucidate the rather notorious associations between the MR imaging data and clinical outcomes in individual MS patients. The proposed methods and systems have a clear benefit for the patient and clinical MS disease management, but may also help decrease the high socioeconomic costs associated with the MS disease.

Researchers

Phases of the project and their realization

Work Packages (WPs)
WP I
Data collection and annotation
WP II
Machine learning based image enhancement
WP III
Machine learning based image feature extraction
WP IV
Machine learning based prediction models
WP V
Clinical translation and result dissemination
2020
Brain and spinal cord image acquisition (WP I)
Completed
MR sequence identification (WP II)
Completed
Adversarial image enhancement (WP II)
Completed
2021
Intra- and inter-scanner reacquisition (WP I)
Completed
Image data annotation (WP I)
Completed
Clinical outcome data collection (WP I)
Completed
Adversarial image enhancement (WP II)
Completed
Image quality assessment (WP II)
Completed
Structural healthy brain parcellation (WP III)
Completed
Pathological lesion segmentation (WP III)
Completed
Radiomics and shape feature extraction (WP III)
Completed
Representation based feature extraction (WP III)
Completed
Feature based prediction models (WP IV)
Completed
Image based prediction models (WP IV)
Completed
Result reporting and publication (WP V)
Completed
2022
Intra- and inter-scanner reacquisition (WP I)
Completed
Image data annotation (WP I)
Completed
Clinical outcome data collection (WP I)
Completed
Feature based prediction models (WP IV)
Completed
Image based prediction models (WP IV)
Completed
Multi-level fused prediction models (WP IV)
Completed
Design for model interpretability (WP IV)
Completed
Design for model robustness (WP IV)
Completed
Long-term disease course prediction (WP V)
Completed
Treatment efficacy prediction (WP V)
Completed
Decision support system integration (WP V)
Completed
Result reporting and publication (WP V)
Completed
2023
Treatment efficacy prediction (WP V)
In progress
Decision support system integration (WP V)
In progress
Result reporting and publication (WP V)
In progress

Bibliographics records

1.
Lara Dular, Gregor Brecl Jakob, Lina Savšek, Jožef Magdič, Bojan Rojc, Žiga Špiclin: Napovedovanje kliničnega poteka multiple skleroze iz magnetnoresonančnih slik: predhodni rezultati raziskave. Medicinski razgledi, 62:(S1):51-57, 2023
2.
Matija Kosin: Povečanje ločljivosti magnetno resonančnih slik z generativnimi nevronskimi mrežami. Master's thesis, University of Ljubljana, Faculty of Electrical Engineering (supervisor: Žiga Špiclin), 2021 [COBISS-ID:79977731 ]
3.
Lara Dular, Žiga Špiclin: Mixup augmentation improves age prediction from T1-weighted brain MRI scans. 5th International Workshop on Predictive Intelligence in Medicine - PRIME 2022, held in conjunction with the 25th International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2022, Lecture Notes in Computer Science vol. 13564, pp. 60-70, 2022 [COBISS-ID:125058819 ] [doi:10.1007/978-3-031-16919-9_6 ]
4.
Lara Dular, Žiga Špiclin: Improving across dataset brain age predictions using transfer learning. 4th International Workshop on Predictive Intelligence in Medicine - PRIME 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2021, Lecture Notes in Computer Science vol. 12928, pp. 243-254, 2021 [COBISS-ID:93994243 ] [doi:10.1007/978-3-030-87602-9_23 ]
5.
Žiga Bizjak, Franjo Pernuš, Žiga Špiclin: Deep shape features for predicting future intracranial aneurysm growth. Frontiers in Physiology, 12:644349, 2021 [COBISS-ID:69012739 ] [doi:10.3389/fphys.2021.644349 ]
6.
Martin Žukovec, Lara Dular, Žiga Špiclin: Modeling multi-annotator uncertainty as multi-class segmentation problem. 7th International Brain Lesion Workshop - BrainLes 2021, held in conjunction with the 24th International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2021, Lecture Notes in Computer Science vol. 12962, pp. 112-123, 2022 [COBISS-ID:125030403 ] [doi:10.1007/978-3-031-08999-2_9 ]
7.
Stefan Nenner, Ashkan Khakzar, Moiz Sajid, Mahdi Saleh, Žiga Špiclin, Seong Tae Kim, Nassir Navab: Spatio-temporal learning from longitudinal data for multiple sclerosis lesion segmentation. 6th International Brain Lesion Workshop - BrainLes 2020, v povezavi z the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2020, Lecture Notes in Computer Science vol. 12658, pp. 111-121, 2021 [COBISS-ID:147363843 ] [doi:10.1007/978-3-030-72084-1_11 ]
8.
Hennadii Madan, Rok Berlot, Nicola J. Ray, Franjo Pernuš, Žiga Špiclin: Practical priors for Bayesian inference of latent biomarkers. IEEE Journal of Biomedical and Health Informatics, 24(2):396-406, 2020 [COBISS-ID:12721492 ] [doi:10.1109/JBHI.2019.2945077 ]