Fix bug in main_nlg_prompt_batch.py

[Nevermetyou65]

What this PR do?

• Fix bug in main_nlg_prompt_batch.py
• When running n shot for NLG tasks, the code breaks when trying to do evaluation for iapp_squad_seacrowd_qa

My investigation:

I found that this line task_type != Tasks.QUESTION_ANSWERING evaluate to True for iapp dataset

from evaluation.data_utils import load_nlg_datasets
from nusacrowd.utils.constants import Tasks

cfg_name_to_dset_map = load_nlg_datasets()
a ,b  = cfg_name_to_dset_map["iapp_squad_seacrowd_qa"]
b # <Tasks.QUESTION_ANSWERING: 'QA'>
b != Tasks.QUESTION_ANSWERING # True

And then len(sample["text_1"]) < 20 will be evaluated, but iapp dataset dose not have text_1 column.

Solution

• Change from task_type != Tasks.QUESTION_ANSWERING to task_type.value != "QA"
• I don't know if this is the best way to do this. I am open to suggestions.