ARTICLE 26.03.2021

Watch the Replay: Introducing 2Be3-Net for Enhanced 3D PET Scan Predictions

If you missed Ronan Thomas’s presentation at the 2nd International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2021), the full video replay is now available online.

Dive into the intersection of Computer Vision and Healthcare as Ronan presents the team's paper: "2Be3-Net: Combining 2D and 3D convolutional neural networks for 3D PET scans predictions."

The Challenge: Radiomics vs. Deep Learning in Oncology

When developing predictive models from 3D Positron Emission Tomography (PET) scans for cancer patients, researchers typically face a crossroads with two main approaches, each carrying significant limitations:

Radiomics: While extracting high-dimensional features from clinical images is the standard approach, it relies on highly accurate tumor segmentation. This process is intensely time-consuming and highly vulnerable to inter-observer variability.
Deep Convolutional Neural Networks (CNN): While data-driven approaches bypass manual segmentation, standard 3D CNNs struggle to achieve high performance on PET images. This is primarily due to the massive size of 3D networks combined with a scarcity of large, publicly available PET datasets.

The 2Be3-Net Architecture

To bridge this gap, the research team assembled several public datasets to create a robust PET dataset of 2,800 scans. From this foundation, they developed 2Be3-Net, an innovative deep learning architecture that combines the strengths of 2D and 3D processing:

2D Feature Extraction: The model leverages a 2D pre-trained model to efficiently extract feature maps from 2D PET slices.
3D Prediction: A 3D CNN is then applied on top of the concatenated feature maps to compute highly accurate, patient-wise predictions.

Experiments indicate that 2Be3-Net exploits spatial information significantly better than isolated 2D or 3D CNN solutions.

Proven Clinical Impact

The team evaluated the 2Be3-Net architecture on predicting clinical outcomes for head-and-neck cancer. The proposed pipeline outperforms traditional PET radiomics approaches in predicting both loco-regional recurrences and overall patient survival.