FasterRCNN bug with grayscale input

[adeschemps]

The following code doesn't fail, even though I believe it should:

from pl_bolts.models.detection import FasterRCNN
import torch 

model = FasterRCNN()
print(model)
image = torch.randn((1,1,256,256))
result = model(image)

as print(model) returns:

self.model: FasterRCNN(
  (transform): GeneralizedRCNNTransform(
      Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
      Resize(min_size=(800,), max_size=1333, mode='bilinear')
  )
  (backbone): BackboneWithFPN(
    (body): IntermediateLayerGetter(
      (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
      (bn1): FrozenBatchNorm2d(64, eps=1e-05)
      (relu): ReLU(inplace=True)
      (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
      (layer1): Sequential(
        (0): Bottleneck(
          (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): FrozenBatchNorm2d(64, eps=1e-05)
          (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): FrozenBatchNorm2d(64, eps=1e-05)
          (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): FrozenBatchNorm2d(256, eps=1e-05)
          (relu): ReLU(inplace=True)
          (downsample): Sequential(
            (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (1): FrozenBatchNorm2d(256, eps=1e-05)
          )
        )

showing that the first convolution of the backbone expects an input with 3 channels instead of one. This is confirmed by https://github.com/pytorch/vision/blob/3300692c6e7c2023d2f2356a69ec22ca91e38790/torchvision/models/resnet.py#L323:

self.conv1 = nn.Conv2d(3, self.inplanes, kernel_size=7, stride=2, padding=3, bias=False)

Because of this, I don't understand what the model is actually computing, which is very confusing. Any help would be much appreciated

[i008]

Seems like it converts it automatically to 3 channels during the normalization (see below). This does not have to be the desired way of doing it, one could also just copy the single channel 3 times. Or change the architecture.
https://github.com/pytorch/vision/blob/main/torchvision/models/detection/transform.py#L127

[tchaton]

Dear @adeschemps,

I believe you should open this issue on Bolts directly.

Best,
T.C