Add GradCAM attention heatmap visualization to Gradio interface using pytorch-grad-cam

[Copilot]

Overview

This PR adds attention heatmap visualization to the Gradio interface using GradCAM (Gradient-weighted Class Activation Mapping), allowing users to see which regions of fundus images the model focuses on when making predictions.

What's New

The Gradio interface now displays two outputs side-by-side:

1. Prediction probabilities - Classification results across all disease classes
2. Attention heatmap - Visual overlay showing which image regions influenced the prediction

Example Usage

When a user uploads a fundus image and clicks "Analyze Image":

• The model generates disease predictions as before
• A new attention heatmap is created showing focus areas:
  • Red/Yellow regions: High attention (features the model considers most important)
  • Blue/Green regions: Low attention (less influential areas)

This helps users understand and validate the model's decision-making process.

Implementation Details

Using pytorch-grad-cam Library

This implementation uses the well-maintained pytorch-grad-cam library instead of a custom implementation:

• Professional, battle-tested library with 10k+ GitHub stars
• Active maintenance and community support
• Access to multiple CAM variants (GradCAM++, EigenCAM, LayerCAM, etc.)
• Better compatibility with diverse model architectures
• Optimized performance and visualization utilities

Multi-Architecture Support

Automatically selects the appropriate layer for GradCAM based on model type:

• MobileNetV4: Last convolutional layer in features
• LeViT: Last transformer block
• EfficientViT: Last stage
• GENet: Last stage
• RegNetX: Last trunk layer
• Fallback: Automatic detection for unknown architectures

Robust Error Handling

• Gracefully degrades if heatmap generation fails
• Falls back to standard prediction without heatmap
• Comprehensive logging for debugging
• Returns original image if processing errors occur

Changes

Modified Files

• pyproject.toml: Added grad-cam>=1.5.0, opencv-python>=4.8.0, and pillow>=10.0.0 as dependencies (UV-managed)
• uv.lock: Updated with resolved dependency graph (revision 3, 98 packages)
• requirements.txt: Auto-generated by UV (287 lines with full dependency tree)
• gradio-inference.py: Integrated pytorch-grad-cam library with ~140 net lines added for UI updates and integration
• README.md: Documented feature with usage guide and interpretation instructions
• .gitignore: Added test_images/ directory

Dependency Management

This PR uses UV for dependency management following best practices:

• Dependencies declared in pyproject.toml (single source of truth)
• Lock file (uv.lock) ensures reproducible builds
• requirements.txt auto-generated using uv pip compile for pip compatibility
• All dependencies pinned to exact versions with full transitive dependency tree

Key Functions

from pytorch_grad_cam import GradCAM
from pytorch_grad_cam.utils.image import show_cam_on_image

def get_target_layers(model, model_type):
    """Selects appropriate layers for GradCAM based on architecture"""
    
def apply_heatmap_on_image(img, cam):
    """Overlays heatmap on original image using library utilities"""
    
def predict_image(image, model_type) -> tuple[dict, np.ndarray]:
    """Now returns (predictions, heatmap) instead of just predictions"""

Benefits

• Interpretability: Makes AI decisions transparent and explainable
• Clinical Validation: Verifies model focuses on relevant anatomical features
• Trust Building: Users can see and validate the reasoning behind predictions
• Debugging: Helps identify if model learns spurious correlations or artifacts
• Educational: Teaches users about AI decision-making processes
• Maintainability: Uses well-tested library instead of custom implementation
• Future-proof: Easy access to advanced CAM variants when needed
• Reproducible: UV-based dependency management ensures consistent builds

Testing

All tests pass successfully:

• ✅ Import verification
• ✅ GradCAM generation with pytorch-grad-cam library
• ✅ Heatmap overlay creation
• ✅ Output type validation
• ✅ Syntax and structure verification
• ✅ Error handling and fallback mechanisms

Backwards Compatibility

This PR maintains full backwards compatibility:

• No breaking changes to existing functionality
• Existing prediction code paths preserved
• Error handling ensures graceful degradation
• UI enhanced without replacing existing components

Screenshots/Examples

The Gradio interface now shows:

┌──────────────────┬──────────────────────────┐
│  Input           │  Outputs                 │
├──────────────────┼──────────────────────────┤
│  Upload Image    │  Prediction Probabilities│
│  Select Model    │  ┌─────────────────────┐ │
│  [Analyze]       │  │ Healthy: 85%        │ │
│                  │  │ Diabetic Ret.: 8%   │ │
│                  │  └─────────────────────┘ │
│                  │                          │
│                  │  Attention Heatmap       │
│                  │  ┌─────────────────────┐ │
│                  │  │  [Fundus image with │ │
│                  │  │   colored overlay]  │ │
│                  │  └─────────────────────┘ │
└──────────────────┴──────────────────────────┘

Related

Implements the feature requested in issue: "add the attentionmap.heatmap to display in gradio"

[!WARNING]

Original prompt

add the attentionmap.heatmap to dispplay in gradio

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Copilot]

Copilot wasn't able to review any files in this pull request.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• astral.sh
  • Triggering command: curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[Copilot]

Pull Request Overview

Copilot reviewed 4 out of 7 changed files in this pull request and generated 4 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

Moving the model to device on every prediction call is inefficient. The model should be moved to device once during loading in the load_model function, not repeatedly in predict_image.

Suggested change

model.to(device)

[Copilot]

[nitpick] Explicitly deleting cam_extractor is unnecessary in Python as it will be automatically garbage collected when it goes out of scope. This deletion doesn't provide significant benefit and adds clutter.

Suggested change

	del cam_extractor

[Copilot]

The traceback module is imported inside the exception handler. This import should be moved to the top of the file with other imports for better code organization and to avoid repeated imports during runtime.

[Copilot]

The traceback module is imported inside the exception handler. This import should be moved to the top of the file with other imports for better code organization and to avoid repeated imports during runtime.