[Feature] Image fields for multi-modal models

[athewsey]

I'm trying to evaluate Claude v3's performance for some document understanding tasks, with a workflow that includes passing the image of the page in as one of the inputs.

Is fmeval considering native handling for image/multi-modal fields in input datasets?

[keerthanvasist]

Hey @athewsey, thanks for reaching out. At this time, we only support LLMs (that text based models).

Adding support for multi-model fields is a future roadmap item.