Issues running ChatPromptBuilder in a serialized pipeline

[lfunderburk]

Hello team,

I am hoping you can help me identify what I am doing wrong when I make a post request to my pipelines. I was able to deploy the pipeline but I am having issues making a post request. I thought I might simply want to pass the topic but I get an error

ValueError: The ChatPromptBuilder expects a list containing only ChatMessage instances. The provided list contains other types. Please ensure that all elements in the list are ChatMessage instances.

Here is a simple pipeline I built

from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import HuggingFaceLocalChatGenerator
from haystack import Pipeline 
from dotenv import load_dotenv
import os

template = [ChatMessage.from_system("""
                            Please create a summary about the following topic:
                            {{ topic }}
                """)]
prompt_builder = ChatPromptBuilder(template=template)
llm = HuggingFaceLocalChatGenerator(model="Qwen/Qwen2.5-1.5B-Instruct", generation_kwargs={"max_new_tokens": 150})


# Inititalize pipeline
pipe = Pipeline()
# Add components
pipe.add_component(instance=prompt_builder, name="prompt_builder")
pipe.add_component(instance=llm, name="llm" )
pipe.connect("prompt_builder.prompt", "llm.messages")

response = pipe.run({"prompt_builder": {"topic": "The history of the internet"}})
print(response)

# Serialize the pipeline
yaml_pipeline = pipe.dumps()

# Save the pipeline to a file
with open("chat_pipeline.yaml", "w") as f:
    f.write(yaml_pipeline)

The serialized version

components:
  llm:
    init_parameters:
      generation_kwargs:
        max_new_tokens: 150
        return_full_text: false
        stop_sequences: []
      huggingface_pipeline_kwargs:
        device: mps
        model: Qwen/Qwen2.5-1.5B-Instruct
        task: text-generation
      streaming_callback: null
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
    type: haystack.components.generators.chat.hugging_face_local.HuggingFaceLocalChatGenerator
  prompt_builder:
    init_parameters:
      required_variables: null
      template:
      - content: "\n                            Please create a summary about the\
          \ following topic:\n                            {{ topic }}\n          \
          \      "
        meta: {}
        name: null
        role: system
      variables: null
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
connections:
- receiver: llm.messages
  sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}

Sample post command

curl -X 'POST' \
  'http://localhost:1416/chat_pipeline' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "llm": {
    "generation_kwargs": {}
  },
  "prompt_builder": {
    "topic": "",
    "template": [
      {
        "content": "string",
        "role": "assistant",
        "name": "string",
        "meta": {}
      }
    ],
    "template_variables": {}
  }
}'

[anakin87]

Hello!

As I mentioned on Discord, this is related to the recent ChatMessage refactoring.

If I use Haystack 2.9.0, your code produces the following YAML (please note that I am using ChatMessage.from_user because it seems more appropriate).

components:
  llm:
    init_parameters:
      chat_template: null
      generation_kwargs:
        max_new_tokens: 150
        return_full_text: false
        stop_sequences: []
      huggingface_pipeline_kwargs:
        model: Qwen/Qwen2.5-1.5B-Instruct
        task: text-generation
      streaming_callback: null
      token:
        env_vars:
        - HF_API_TOKEN
        - HF_TOKEN
        strict: false
        type: env_var
    type: haystack.components.generators.chat.hugging_face_local.HuggingFaceLocalChatGenerator
  prompt_builder:
    init_parameters:
      required_variables: null
      template:
      - _content:
        - text: "\n                            Please create a summary about the following\
            \ topic:\n                            {{ topic }}\n                "
        _meta: {}
        _name: null
        _role: user
      variables: null
    type: haystack.components.builders.chat_prompt_builder.ChatPromptBuilder
connections:
- receiver: llm.messages
  sender: prompt_builder.prompt
max_runs_per_component: 100
metadata: {}

(The template serialization differs)

This pipeline can be successfully deployed on Hayhooks.

The example request body (in API docs) is

{
  "llm": {
    "generation_kwargs": {}
  },
  "prompt_builder": {
    "topic": "",
    "template": [
      {
        "_role": "user",
        "_content": [
          {
            "text": "string"
          },
          {
            "tool_name": "string",
            "arguments": {},
            "id": "string"
          },
          {
            "result": "string",
            "origin": {
              "tool_name": "string",
              "arguments": {},
              "id": "string"
            },
            "error": true
          }
        ],
        "_name": "string",
        "_meta": {}
      }
    ],
    "template_variables": {}
  }
}

Using

{
  "llm": {
    "generation_kwargs": {}
  },
  "prompt_builder": {
    "topic": "NLP"
 
  }
}

I get a correct response:

{
  "llm": {
    "replies": [
      {
        "_role": "assistant",
        "_content": [
          {
            "text": "Sure! Here's a concise summary of NLP..."
          }
        ],
        "_name": null,
        "_meta": {
          "finish_reason": "length",
          "index": 0,
          "model": "Qwen/Qwen2.5-1.5B-Instruct",
          "usage": {
            "completion_tokens": 150,
            "prompt_tokens": 44,
            "total_tokens": 194
          }
        }
      }
    ]
  }
}

Let me know if this helps...

[lfunderburk]

This helps - thank you!