at carnegie mellon
Attention-augmented Dual-View Mammogram Alignment for Enhanced BI-RADS Classification
Aligns and fuses CC and MLO mammograms with spatial transformers, then predicts BI-RADS categories from both views.
Project Report
This project presents a deep learning pipeline for automatic alignment and classification of dual-view mammograms using Spatial Transformer Networks (STNs) and transfer learning. Mammography typically includes two standard views: Craniocaudal (CC) and Mediolateral Oblique (MLO). These views differ significantly in geometry, making direct comparison difficult. This project addresses that challenge by learning spatial transformations that align the two views before classification.
Objectives
- Align MLO mammograms to CC views using learned transformations
- Generate fused representations of dual-view breast images
- Predict BI-RADS categories (0–5) using a deep neural network
- Handle class imbalance and preserve ordinal relationships in labels
Spatial Alignment using a Spatial Transformer
- Input: concatenated CC + MLO images
- CNN localization network
- Fully connected regression head
- Outputs: affine transformation matrix (2×3) Transformation uses grid sampling to warp MLO images Produces: Aligned MLO and a Transformation matrix Output: Fused tensor [CC, aligned MLO], saved for downstream trainingCollaborator: Elissa Matlock