Testbed - Index out of Range

[gotsysdba]

Follow-up with giskard, llama3.1/mxbai with +2 q's may result in less q's being generated due to following:

2025-Mar-06 07:10:48 - ERROR    - (giskard.rag): list index out of range
Traceback (most recent call last):
  File ".venv/lib/python3.11/site-packages/giskard/llm/client/litellm.py", line 144, in complete
    json_dict = _parse_json_output(response_message.content, self, caller_id=caller_id)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/giskard/llm/client/litellm.py", line 73, in _parse_json_output
    parsed_dict = json.loads(_trim_json(out.content), strict=False)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "3.11/lib/python3.11/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
           ^^^^^^^^^^^^^^^^^^^
  File "3.11/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "3.11/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 16 (char 15)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".venv/lib/python3.11/site-packages/giskard/rag/question_generators/base.py", line 80, in generate_questions
    yield self._modify_question(question, knowledge_base, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/giskard/rag/question_generators/complex_questions.py", line 118, in _modify_question
    out = self._llm_complete(messages=messages)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/giskard/rag/question_generators/base.py", line 48, in _llm_complete
    return parse_json_output(out.content, self._llm_client, caller_id=self.__class__.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/giskard/rag/question_generators/utils.py", line 45, in parse_json_output
    out = llm_client.complete(
          ^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/giskard/llm/client/litellm.py", line 150, in complete
    response_message = completion.choices[i + 1].message
                       ~~~~~~~~~~~~~~~~~~^^^^^^^
IndexError: list index out of range