3D Autoencoder for Glioblastom

Cheatsheet Content

### Overview: 3D Autoencoder with MLP Classifier This cheatsheet outlines a deep learning pipeline for glioblastoma (GBM) MRI analysis, incorporating a 3D Autoencoder for feature extraction, Gaussian Mixture Model (GMM) for immune profiling, and an MLP classifier for downstream tasks. #### Model Architecture 1. **3D Autoencoder:** Learns a compressed representation (latent space) from 3D MRI scans. 2. **GMM-based Immune Profiling:** Processes immune cell scores to define a posterior boundary for binary splitting. 3. **MLP Classifier Head:** Predicts clinical outcomes or subtypes using the concatenated latent space features and GMM posterior boundary. ### Data Ingestion: 3D Glioblastoma MRI - **Input Data:** 3D Volumetric MRI scans of glioblastoma tumors. - **Modalities:** T1, T1ce, T2, FLAIR (common for GBM). - **Preprocessing:** - **Registration:** Align MRI sequences to a common space (e.g., T1ce). - **Normalization:** Intensity normalization (e.g., Z-score, min-max scaling) per volume. - **Resampling:** Standardize voxel spacing and image dimensions. - **Skull Stripping/Brain Extraction:** Remove non-brain tissue. - **Tumor Segmentation (Optional):** Isolate tumor region for focused analysis or masking. - **Shape:** `(Batch_Size, Depth, Height, Width, Channels)` where `Channels` is the number of MRI modalities. ### 3D Autoencoder - **Purpose:** Dimensionality reduction and learning robust, low-dimensional representations of 3D MRI volumes. - **Encoder Architecture:** - Sequence of 3D Convolutional layers (`Conv3D`). - Activation functions (e.g., ReLU, Leaky ReLU). - Pooling layers (e.g., `MaxPool3D` or `Strided Conv3D` for downsampling). - Batch Normalization. - Outputs a latent vector $\mathbf{z}$. - Example: `Conv3D -> ReLU -> MaxPool3D -> ... -> Flatten -> Dense` - **Decoder Architecture:** - Mirrors the encoder. - Sequence of 3D Transposed Convolutional layers (`Conv3DTranspose`) or Upsampling layers. - Reconstructs the original 3D MRI from $\mathbf{z}$. - Example: `Dense -> Reshape -> Conv3DTranspose -> ReLU -> ... -> Conv3D (output channel=input channel, e.g., sigmoid/tanh for output)` - **Loss Function:** Reconstruction loss (e.g., Mean Squared Error (MSE), Binary Cross-Entropy for normalized data). $$ L_{AE} = ||X - \hat{X}||^2 $$ - **Output:** Latent space vector $\mathbf{z}$ (e.g., 128-512 dimensions). ### GMM-based Immune Profiling - **Input:** Immune cell scores (e.g., from bulk RNA-seq, scRNA-seq, or IHC data). - Assumed to be a vector of scores for various immune cell types or immune-related pathways. - **Gaussian Mixture Model (GMM):** - Models the distribution of immune cell scores as a mixture of Gaussian distributions. - **Clustering:** Can be used to identify distinct immune profiles (e.g., "hot" vs. "cold" tumors). - **Posterior Probability:** For each sample $x_{immune}$, calculate the posterior probability of belonging to each cluster $k$: $$ P(C_k | x_{immune}) = \frac{P(x_{immune} | C_k) P(C_k)}{\sum_j P(x_{immune} | C_j) P(C_j)} $$ - **Binary Split & Boundary:** - Assume a pre-defined binary split (e.g., high vs. low immune infiltration). - The "learned posterior boundary" refers to using the posterior probabilities from a GMM (trained on immune scores) to define a boundary that separates these two groups. - This could be: - The probability $P(C_{high} | x_{immune})$ for the "high infiltration" cluster. - A ratio like $\frac{P(C_1 | x_{immune})}{P(C_2 | x_{immune})}$. - A decision boundary derived from the GMM components. - **Output:** A scalar value or a vector representing the sample's position relative to the immune boundary (e.g., posterior probability of being in the "high immune" group). Let's call this $\mathbf{p}_{immune}$. ### MLP Classifier Head - **Purpose:** Predicts a target variable (e.g., survival, treatment response, tumor subtype) based on latent features and immune profile. - **Input:** Concatenation of the autoencoder's latent space vector $\mathbf{z}$ and the GMM-derived immune boundary value $\mathbf{p}_{immune}$. $$ \mathbf{f}_{combined} = [\mathbf{z}, \mathbf{p}_{immune}] $$ - **Architecture:** - Sequence of Dense (Fully Connected) layers. - Activation functions (e.g., ReLU). - Dropout for regularization. - Output layer: - **Binary Classification:** `Dense(1, activation='sigmoid')` - **Multi-class Classification:** `Dense(num_classes, activation='softmax')` - **Regression:** `Dense(1, activation='linear')` - **Loss Function:** - **Binary:** Binary Cross-Entropy - **Multi-class:** Categorical Cross-Entropy - **Regression:** Mean Squared Error (MSE) - **Training:** End-to-end training (fine-tuning the AE with the classifier) or sequential training (train AE first, then classifier). ### Training Considerations - **Phased Training:** 1. Train 3D Autoencoder independently on MRI data. 2. Train GMM on immune cell scores. Extract posterior probabilities/boundary. 3. Train MLP classifier using fixed latent features from AE and GMM output. 4. (Optional) Fine-tune the entire model end-to-end. - **Optimization:** Adam, SGD with momentum. - **Regularization:** L1/L2 regularization, Dropout. - **Evaluation Metrics:** - **AE:** PSNR, SSIM, MSE (for reconstruction). - **Classifier:** Accuracy, F1-score, AUC-ROC, Precision, Recall (for classification); R2, MSE (for regression). - **Hardware:** Requires substantial GPU memory for 3D convolutions.