Best way to store structured conversation context

[rafaelpierre]

Question

I was wondering what's the best practice to store conversation context for multiturn use cases.

Just for disambiguation purposes - I'm referring here to arbitrary, structured context that I want to persist for individual sessions, specially for multiturn conversational use cases. This is not related to the conversation history per se.

To make it a bit more concrete - let's say I have a use case where I want to implement agents for a Customer Service use case, where I want to ask some questions to users to understand if they fulfill the needs to be routed to a human agent.

Let's say I have a ConversationContext class which inherits from pydantic.BaseModel:

class ConversationContext(BaseModel):
   intent: Literal["unknown", "insurance", "mortgage", "credit_card"]
   vat_registration: int

My idea is to update this context as the conversation turns go.

Initially I assumed that the context object passed to the runner would be the way to go. Now reading the documentation a bit closer, I don't think that's the case - given that the RunContextWrapper is not really shared/changed by the LLM.

I had two ideas:

1. Serializing/deserializing this ConversationContext object and including it as part of the system prompt and/or conversation history - but that sounds too... prone to hallucination / non-deterministic?

2. Creating a function_tool that reads the existing ConversationContext from an external DB, and updates it if needed

Is anyone implementing something similar? If yes, any suggestions?

[apiltamang]

I am using a multiturn using a conversation states like so:

Conversation States

[
  {
    "id": "1_intro",
    "description": "Begin each conversation with a warm, friendly greeting, identifying the service and offering help.",
    "instructions": [
        "Use the company name 'Snowy Peak Boards' and provide a warm welcome."
    ],
    "examples": [
      "Hello, this is Snowy Peak Boards returns specialist. How can I help you?"
    ],
    "transitions": [{
      "next_step": "2_get_first_name",
      "condition": "Once greeting is complete."
    }]
  },
  {
    "id": "2_get_first_name",
    "description": "Ask for the user’s name (first name only).",
    "instructions": [
      "Politely ask, 'Who do I have the pleasure of speaking with?'",
      "Do NOT verify or spell back the name; just accept it."
    ],
    "examples": [
      "Who do I have the pleasure of speaking with?"
    ],
    "transitions": [{
      "next_step": "3_get_and_verify_phone",
      "condition": "Once name is obtained, OR name is already provided."
    }]
  },
  {
    "id": "3_post_name_phone_gathering",
    "description": "After getting name and number, proceed to assist with the user’s request.",
    "instructions": [
      "Show the user that you remember their original request",
      "Use your judgment for how best to assist with their request, while being transparent about what you don't know and aren't able to help with."
    ],
    "examples": [
      "Great, now I'd love to help you with {user's original intent}."
    ],
    "transitions": [{
      "next_step": "4_fetch_order_items",
    }]
  },
  {
    "id": "4_fetch_order_items",
    "description": "fetch order items for this user",
    "instructions": [
        "call the tool 'fetch_recent_orders' with parameter phonNumber to fetch list of recent orders. Inform the user you are fetching items and to wait a while."
    ],
    "examples": [
      "Fetching your orders. Please wait for just a little while."
    ],
    "transitions": [{
      "next_step": "5_mention_items_retrieved",
    }]
  },
  {
     "id": "5_mention_items_retrieved",
    "description": "mention order items fetched for this user",
    "instructions": [
        "Mention the list of items that were fetched for the user from the previous step"
    ],
    "transitions": [{
      "next_step": "call the tool 'forward_call' with parameter SUPPORT",
    }]
  }
]

For the most part, it seems to work. For example, when the model now calls a tool that requires 'name', 'phone' etc. the model seems to populate the parameters just fine. Almost seems like the model remembers what the 'name' and phone should be. Let me know if this is what you intended, or if this seems to help.