[Bug]: PPTXReader crashes when handling WMF/EMF images

[Leon-Sander]

Bug Description

PPTXReader can't handle WMF/EMF images inside the power point.

fix/workaround by #17819

Version

0.12.17

Steps to Reproduce

from llama_index.readers.file.slides.base import PptxReader
reader = PptxReader()
reader.load_data("power_point_with_wmf.pptx")

Relevant Logs/Tracbacks

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[1], line 3
      1 from llama_index.readers.file.slides.base import PptxReader
      2 reader = PptxReader()
----> 3 reader.load_data("test.pptx")

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/llama_index/readers/file/slides/base.py:175, in PptxReader.load_data(self, file, extra_info, fs)
    173     f.write(image_bytes)
    174     f.close()
--> 175     result += f"\n Image: {self.caption_image(f.name)}\n\n"
    176 finally:
    177     os.unlink(f.name)

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/llama_index/readers/file/slides/base.py:138, in PptxReader.caption_image(self, tmp_image_file)
    135 if i_image.mode != "RGB":
    136     i_image = i_image.convert(mode="RGB")
--> 138 pixel_values = feature_extractor(
    139     images=[i_image], return_tensors="pt"
    140 ).pixel_values
    141 pixel_values = pixel_values.to(device)
    143 output_ids = model.generate(pixel_values, **gen_kwargs)

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/transformers/image_processing_utils.py:41, in BaseImageProcessor.__call__(self, images, **kwargs)
     39 def __call__(self, images, **kwargs) -> BatchFeature:
     40     """Preprocess an image or a batch of images."""
---> 41     return self.preprocess(images, **kwargs)

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/transformers/utils/generic.py:852, in filter_out_non_signature_kwargs.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    843         cls_prefix = ""
    845     warnings.warn(
    846         f"The following named arguments are not valid for `{cls_prefix}{func.__name__}`"
    847         f" and were ignored: {invalid_kwargs_names}",
    848         UserWarning,
    849         stacklevel=2,
    850     )
--> 852 return func(*args, **valid_kwargs)

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/transformers/models/vit/image_processing_vit.py:237, in ViTImageProcessor.preprocess(self, images, do_resize, size, resample, do_rescale, rescale_factor, do_normalize, image_mean, image_std, return_tensors, data_format, input_data_format)
    225 validate_preprocess_arguments(
    226     do_rescale=do_rescale,
    227     rescale_factor=rescale_factor,
   (...)
    233     resample=resample,
    234 )
    236 # All transformations expect numpy arrays.
--> 237 images = [to_numpy_array(image) for image in images]
    239 if is_scaled_image(images[0]) and do_rescale:
    240     logger.warning_once(
    241         "It looks like you are trying to rescale already rescaled images. If the input"
    242         " images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again."
    243     )

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/transformers/models/vit/image_processing_vit.py:237, in <listcomp>(.0)
    225 validate_preprocess_arguments(
    226     do_rescale=do_rescale,
    227     rescale_factor=rescale_factor,
   (...)
    233     resample=resample,
    234 )
    236 # All transformations expect numpy arrays.
--> 237 images = [to_numpy_array(image) for image in images]
    239 if is_scaled_image(images[0]) and do_rescale:
    240     logger.warning_once(
    241         "It looks like you are trying to rescale already rescaled images. If the input"
    242         " images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again."
    243     )

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/transformers/image_utils.py:216, in to_numpy_array(img)
    213     raise ValueError(f"Invalid image type: {type(img)}")
    215 if is_vision_available() and isinstance(img, PIL.Image.Image):
--> 216     return np.array(img)
    217 return to_numpy(img)

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/PIL/Image.py:742, in Image.__array_interface__(self)
    740         new["data"] = self.tobytes("raw", "L")
    741     else:
--> 742         new["data"] = self.tobytes()
    743 except Exception as e:
    744     if not isinstance(e, (MemoryError, RecursionError)):

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/PIL/Image.py:802, in Image.tobytes(self, encoder_name, *args)
    799 if encoder_name == "raw" and encoder_args == ():
    800     encoder_args = self.mode
--> 802 self.load()
    804 if self.width == 0 or self.height == 0:
    805     return b""

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/PIL/WmfImagePlugin.py:163, in WmfStubImageFile.load(self, dpi)
    158     x0, y0, x1, y1 = self.info["wmf_bbox"]
    159     self._size = (
    160         (x1 - x0) * self.info["dpi"] // self._inch,
    161         (y1 - y0) * self.info["dpi"] // self._inch,
    162     )
--> 163 return super().load()

File ~/anaconda3/envs/api_service/lib/python3.11/site-packages/PIL/ImageFile.py:377, in StubImageFile.load(self)
    375 if loader is None:
    376     msg = f"cannot find loader for this {self.format} file"
--> 377     raise OSError(msg)
    378 image = loader.load(self)
    379 assert image is not None

OSError: cannot find loader for this WMF file

[dosubot]

The PptxReader class in the LlamaIndex library currently does not support WMF/EMF image formats, which leads to the error you're encountering. The error occurs because the PIL library, used for image processing, does not have a loader for WMF/EMF files.

To address this issue, you can consider the following workaround:

1. Convert WMF/EMF Images: Before processing the PowerPoint file with PptxReader, convert WMF/EMF images to a supported format like JPEG or PNG. This can be done using external tools or libraries that support these formats.

2. Modify the Code: If you have control over the code, you could modify the PptxReader class to skip WMF/EMF images or handle them differently, such as by logging a warning instead of attempting to process them.

3. Use a Different Library: Consider using a library that supports WMF/EMF formats for image extraction and processing, and integrate it with your current setup.

These approaches can help you bypass the current limitation until native support for WMF/EMF is potentially added to the library or its dependencies.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}