langgraph/tutorials/rag/langgraph_agentic_rag/

[giscus[bot]]

langgraph/tutorials/rag/langgraph_agentic_rag/

Build language agents as graphs

https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/

[andresucv96]

Hello,

Is it only me that whenever the decision is to reformulate the question and the state goes back to the agent, it stops there. It is not going back to the retriever to get more and better context...

[hinthornw]

By any chance, do you have a LangSmith trace you could share to illustrate the issue?

[100stacks]

Hi @hinthornw,

It appears this warning in #722 (comment):

USER_AGENT environment variable not set, consider setting it to identify your requests.

was introduced with this community: add user agent for web scraping loaders PR. This PR is referring to User Agent in a HTTP Request Header to support web scrapping use cases for LLMs.

Unfortunately, this warning is rendered whenever anyone imports and uses langchain_community.document_loaders that calls
user_agent.py. Btw v0.1 docs reference web scrapping directly, though Idk if the page was ported to v0.2 docs. 🤔

I don't think this environment variable is needed for a majority of LangChain use cases, and recommend it be made an optional environment variable or simply do not render a warning if USER_AGENT is not present.

[VladAndronik]

@andresucv96 It is because Agent node sees full history of messages after rewrite, so he always comes to decision to use existing context for the answer without the need for new retrieval, basically taking up the role of Grader. If you only give him the rewritten question he would decide upon that. However, the loop could become infinite and you need to write stop condition.

[bnbabu55]

@andresucv96, I am facing the same issue, I see that Rewrite is setting the end condition, not able to figure out why.

[bnbabu55]

@hinthornw, here is my langsmith trace for the issue above. https://smith.langchain.com/o/8045ed60-fa03-5794-b09c-62794425fd1a/projects/p/f7928a5e-dad0-4325-9fde-8ae6719672aa?timeModel=%7B%22duration%22%3A%227d%22%7D&peek=eca9a236-582b-48bf-b8da-d55ee786fa8a&tab=0&Volume=Success&Latency=P50&Tokens=P50&Cost=P50&Streaming=P50

[raselai]

Whenever I run this Agentic RAG I am getting this error bellow:

"USER_AGENT environment variable not set, consider setting it to identify your requests."

[Mooo49]

I face the same issue wile using streamlit , but it works well on google colab . so we need a solution with streamlit ??

[bnbabu55]

I am just running this as a standalone python script, even then this is happening.

[Haopeng138]

export USER_AGENT=myagent  

This work for me

[Muyuu77]

import os
os.environ['USER_AGENT'] = 'myagent'

Maybe try to put these lines at the beginning of your python file.

[pengyulong]

Setting "USE_AGENT" like this seems to be of no use.

[Dumplingisabeast]

when defining the state class, when would you use:
Annotated[Sequence[BaseMessage],...] vs. Annotated[list[AnyMessage],...]?

[pguatibonza]

Hello! I want to build an agentic rag with other tools as well, but I dont know yet how to handle it. For example, If I have another tool(i.e web search), how can I ensure that it follows another path, instead of the rag path(grade, rewrite, generate)

[woodswift]

format_docs() is defined, but it is not used? Could you please explain why?

[sreedevi1249]

create web_search node and in the grade_doucment node if documents are not relavent then you can return web_search instead of rewrite that way you can achieve adding one more tool web_search

[Jimmy-L99]

I use my own deployed llm and db, meet error as follow:

AttributeError: 'NoneType' object has no attribute 'binary_score'

[yuehua-s]

You can try replacing score = scored_result.binary_score with score = scored_result["binary_score"], which works for me.

[minmie]

I have this error too, the scored_result is None.
dose any one has a solution？

  File "/home/chenjq/miniconda3/envs/RAG/lib/python3.10/site-packages/langgraph/utils/runnable.py", line 176, in invoke
    ret = context.run(self.func, input, **kwargs)
  File "/home/chenjq/pythonWork/RAG/langgraphDemo/langgraph_agentic_rag.py", line 136, in grade_documents
    score = scored_result.binary_score
AttributeError: 'NoneType' object has no attribute 'binary_score'

[minmie]

i think this may cause by your llm dosen't support structured output.

[ricardo-casa]

what benefits does this architecture have over adaptive rag?

[sreedevi1249]

i tried little different way where if the agent is unable to answer the question meaning out of context we can include the web search which is almost similar to adaptive rag

[Meshinfo]

For all the examples, the index had already been created and did not require updating. However, in a real chat assistant scenario, the index is only created after the user uploads files. It is only at this point that the retrieval tool can be instantiated. Do you have any idea to handle such situations?

[itsihsan]

The following two issues need to be addressed within the code:

Handling Compound Questions:

1 - When a user asks a compound question (for instance., What is an agent, and what is prompt engineering?), the Agent node splits the question into multiple (in this case two) sub-questions (e.g., What is an agent?, and What is prompt engineering?).
2 - The retrieve node retrieves documents for each sub-question, resulting in two tool messages (each containing the retrieved documents for corresponding sub-question in their content part).
3 - Currently, the grade_documents node is grading the contents of the last message only, i.e., the retrieved documents from the last tool message (ignoring the documents that were retrieved in the earlier tool message(s).)

last_message = messages[-1]
docs = last_message.content 

This content is then graded against the original compound question in the grade_documents node, i.e., question = messages[0].content.

I believe the following modifications will address this problem for compound questions:
4 - Ensure the grading logic handles multiple sub-questions. This means to generalize the code to ensure that documents retrieved as a result of the retrieve node for each sub-question are graded against their corresponding sub-question in the grade_documents node.
5 - If all retrieved documents for the compound question are deemed relevant, update the generate node to consider all documents, and not just the ones from the last tool message. Currently,

messages = state["messages"]
question = messages[0].content
last_message = messages[-1]
docs = last_message.content

Issue with Memory Checkpoint:

1 - When a memory checkpoint is added to the graph as below,

from langgraph.checkpoint.memory import MemorySaver

memory = MemorySaver()
graph = workflow.compile(checkpointer=memory)

the question, i.e., question = messages[0].content will always reference the first question in the conversation, instead of the most recent user question. This causes the rewrite node to engage incorrectly due to the outdated questions.

This can be addressed by capturing the user's question and storing/calling it appropriately as below,

class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], add_messages]
    user_question: str

def agent(state):
    print("---CALL AGENT---")
    messages = state["messages"]
    user_question = messages[-1].content
    model = ChatOpenAI(temperature=0, streaming=True, model="gpt-4-turbo")
    model = model.bind_tools(tools)
    response = model.invoke(messages)
    # We return a list, because this will get added to the existing list
    return {"messages": [response], "user_question":user_question}

Calling the stored question in the generate node as below:

def generate(state: State):
    print("---GENERATE---")
    question = state["user_question"]
    # The rest of the code stays the same

I would appreciate feedback from anyone who has addressed the first part. If so, please share the updated code.

[kishorekkota]

Trying to implement this with Azure Search and Azure Hosted Open AI. My retrieval call is not being invoked at all and straight getting responses from Open AI vs sourcing from Vector Store.

[kishorekkota]

Any help would be appreciated. I do not think I am missing any step.

I have two python program created for this.
rag_assistant.py ( this is the program that creates the LangGraph Workflow.

# rag_assistant.py

import os
import uuid
import logging

os.environ["AZURESEARCH_FIELDS_ID"] = "chunk_id"
os.environ["AZURESEARCH_FIELDS_CONTENT"] = "chunk"
os.environ["AZURESEARCH_FIELDS_CONTENT_VECTOR"] = "text_vector"

from typing import Literal

from langchain_openai import AzureChatOpenAI
from langchain.schema import HumanMessage
from langgraph.graph import START, END, StateGraph, MessagesState
from langgraph.prebuilt import ToolNode, tools_condition
from langgraph.checkpoint.postgres import PostgresSaver
from langchain_core.output_parsers import StrOutputParser
from langchain.prompts import PromptTemplate
from langchain import hub

from db_connection import pool
from chat_response import ChatBot
from azure_search import create_vector_store, create_vector_store_tool
from agent_state import AgentState
import prettyprinter as pp

logger = logging.getLogger(__name__)
logger.setLevel(logging.ERROR)


class RAGAIAssistant:
    def __init__(self, thread_id: str, new_conversation: bool = True):
        self.azure_openai_api_key = os.environ.get("AZURE_OPENAI_API_KEY")
        self.azure_openai_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")

        if not self.azure_openai_api_key:
            raise ValueError("AZURE_OPENAI_API_KEY environment variable not set.")
        if not self.azure_openai_endpoint:
            raise ValueError("AZURE_OPENAI_ENDPOINT environment variable not set.")

        self.thread_id = thread_id
        self.user_message = None
        self.new_conversation = new_conversation

        # Initialize the language model
        self.llm = AzureChatOpenAI(
            deployment_name='gpt-35-turbo',
            openai_api_key=self.azure_openai_api_key,
            openai_api_version="2024-05-01-preview",
            openai_api_type="azure",
            azure_endpoint=self.azure_openai_endpoint,
            temperature=0
        )

        # Create vector store and retriever tool
        vector_store = create_vector_store()
        self.retrieval_tool = create_vector_store_tool(vector_store)

        # Bind the retriever tool to the LLM
        self.model_tool = self.llm.bind_tools([self.retrieval_tool])

        # Set up the workflow
        self._setup_workflow()

    def _setup_workflow(self):
        logger.info("Setting up workflow...")

        # Initialize the StateGraph with AgentState
        workflow = StateGraph(AgentState)

        # Add nodes to the workflow
        workflow.add_node("agent", self.agent)
        retrieve_node = ToolNode([self.retrieval_tool])
        workflow.add_node("retrieve", retrieve_node)
        workflow.add_node("rewrite", self.rewrite)
        workflow.add_node("generate", self.generate)

        # Define the workflow edges
        workflow.add_edge(START, "agent")

        # Conditional edges based on the agent's decision
        workflow.add_conditional_edges(
            "agent",
            tools_condition,
            {
                "tools": "retrieve",
                END: END,
            },
        )

        # Conditional edges after retrieval
        workflow.add_conditional_edges(
            "retrieve",
            self.grade_documents,
            {
                "generate": "generate",
                "rewrite": "rewrite",
            },
        )

        workflow.add_edge("generate", END)
        workflow.add_edge("rewrite", "agent")

        # Set up memory with PostgresSaver
        memory = PostgresSaver(pool)
        memory.setup()
        logger.info("Workflow setup completed.")

        self.workflow = workflow.compile(checkpointer=memory)

    def start_new_session(self):
        self.thread_id = uuid.uuid4().hex
        self.new_conversation = False
        logger.info(f"Started new session with thread_id: {self.thread_id}")

    def run(self, user_message: str) -> ChatBot:
        self.user_message = user_message
        if self.new_conversation:
            self.start_new_session()
        config = {"configurable": {"thread_id": self.thread_id}}

        logger.info(f"Running workflow for message: {user_message}")
        response_state = self.workflow.invoke(
            {"messages": [HumanMessage(content=self.user_message)]},
            config=config
        )

        messages = response_state.get("messages", [])
        if messages:
            last_message = messages[-1].content
        else:
            last_message = ""

        response = ChatBot(self.user_message, last_message, self.thread_id)

        return response

    def grade_documents(self, state: MessagesState) -> Literal["generate", "rewrite"]:
        logger.info("Starting document grading process.")

        class Grade(BaseModel):
            binary_score: str = Field(description="Relevance score 'yes' or 'no'.")

        try:
            model = AzureChatOpenAI(
                deployment_name='gpt-35-turbo',
                openai_api_key=self.azure_openai_api_key,
                openai_api_base=self.azure_openai_endpoint,
                openai_api_version="2024-05-01-preview",
                openai_api_type="azure",
                temperature=0
            )
            llm_with_tool = model.with_structured_output(Grade)

            prompt = PromptTemplate(
                template=(
                    "You are a grader assessing relevance of a retrieved document to a user question.\n"
                    "Here is the retrieved document:\n{context}\n\n"
                    "Here is the user question: {question}\n"
                    "If the document contains keyword(s) or semantic meaning related to the user question, grade it as relevant.\n"
                    "Give a binary 'yes' or 'no' score to indicate whether the document is relevant to the question."
                ),
                input_variables=["context", "question"],
            )

            chain = prompt | llm_with_tool

            messages = state["messages"]
            if not messages:
                logger.error("No messages found in state.")
                return "rewrite"

            last_message = messages[-1]
            question = messages[0].content
            docs = last_message.content

            scored_result = chain.invoke({"question": question, "context": docs})
            score = scored_result.binary_score.strip().lower()

            logger.info(f"Grading result: {score}")

            if score == "yes":
                logger.info("Decision: Documents are relevant. Proceeding to generate.")
                return "generate"
            else:
                logger.info("Decision: Documents are not relevant. Proceeding to rewrite.")
                return "rewrite"

        except Exception as e:
            logger.error(f"An error occurred during document grading: {e}")
            return "rewrite"

    def agent(self, state: MessagesState):
        """
        Invokes the agent model to generate a response based on the current state. Given
        the question, it will decide to retrieve using the retriever tool, or simply end.

        Args:
            state (messages): The current state

        Returns:
            dict: The updated state with the agent response appended to messages
        """
        logger.info("Invoking agent.")
        try:
            messages = state["messages"]
            if not messages:
                logger.error("No messages found in state.")
                return {"messages": []}
      
            pp.pprint(self.model_tool)
            response = self.model_tool.invoke(messages)
            logger.info("Agent invoked successfully.")

            print(response.pretty_print())

            return {"messages": [response]}

        except Exception as e:
            logger.error(f"An error occurred in agent: {e}")
            return {"messages": []}

    def rewrite(self, state: MessagesState):
        logger.info("Starting query rewrite process.")

        try:
            messages = state["messages"]
            if not messages:
                logger.error("No messages found in state.")
                return {"messages": []}

            question = messages[0].content

            msg = [
                HumanMessage(
                    content=(
                        "Look at the input and try to reason about the underlying semantic intent/meaning.\n"
                        "Here is the initial question:\n"
                        "-------\n"
                        f"{question}\n"
                        "-------\n"
                        "Formulate an improved question:"
                    ),
                )
            ]

            model = AzureChatOpenAI(
                deployment_name='gpt-35-turbo',
                openai_api_key=self.azure_openai_api_key,
                openai_api_base=self.azure_openai_endpoint,
                openai_api_version="2024-05-01-preview",
                openai_api_type="azure",
                temperature=0
            )
            response = model.invoke(msg)
            logger.info("Query rewritten successfully.")

            return {"messages": [response]}

        except Exception as e:
            logger.error(f"An error occurred during query rewrite: {e}")
            return {"messages": []}

    def generate(self, state: MessagesState):
        logger.info("Starting answer generation process.")

        try:
            messages = state["messages"]
            if not messages:
                logger.error("No messages found in state.")
                return {"messages": []}

            question = messages[0].content
            last_message = messages[-1]
            docs = last_message.content

            prompt = hub.pull("rlm/rag-prompt")
            logger.debug("Prompt pulled from hub.")

            model = AzureChatOpenAI(
                deployment_name='gpt-35-turbo',
                openai_api_key=self.azure_openai_api_key,
                openai_api_base=self.azure_openai_endpoint,
                openai_api_version="2024-05-01-preview",
                openai_api_type="azure",
                temperature=0
            )

            rag_chain = prompt | model | StrOutputParser()

            response = rag_chain.invoke({"context": docs, "question": question})
            logger.info("Answer generated successfully.")

            return {"messages": [response]}

        except Exception as e:
            logger.error(f"An error occurred during answer generation: {e}")
            return {"messages": []}

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)

    
    rag_assistant = RAGAIAssistant(thread_id="test_thread_id", new_conversation=True)
    #rag_assistant.retriever_tool(" What is the leave policy ? ")
    create_vector_store().search(" What is the leave policy ? ",search_type='similarity')
    response = rag_assistant.run("What is process for taking extended leave in New York?")
    print(f"Assistant: {response}")

    ```


Azure_search.py which creates Azure Search as retriever tool.

azure_search.py

import os
import logging

os.environ["AZURESEARCH_FIELDS_ID"] = "chunk_id"
os.environ["AZURESEARCH_FIELDS_CONTENT"] = "chunk"
os.environ["AZURESEARCH_FIELDS_CONTENT_VECTOR"] = "text_vector"

from langchain_community.vectorstores.azuresearch import AzureSearch
from langchain_openai import AzureOpenAIEmbeddings
from langchain.tools.retriever import create_retriever_tool
from azure.identity import DefaultAzureCredential
from langchain_core.tools import tool
from azure.search.documents.indexes.models import SimpleField

Configure logging

logging.basicConfig(level=logging.ERROR)
logger = logging.getLogger(name)

def create_vector_store() -> AzureSearch:
"""
Creates and returns an AzureSearch vector store instance.

Returns:
    AzureSearch: An instance of the AzureSearch vector store.

Raises:
    ValueError: If required environment variables are missing.
"""
print("Creating AzureSearch vector store...")
# Retrieve environment variables
azure_search_endpoint = os.getenv('AZURE_SEARCH_ENDPOINT')
azure_search_key = os.getenv('AZURE_SEARCH_KEY')
search_index_name = os.getenv('SEARCH_INDEX_NAME')
embedding_model_name = os.getenv('EMBEDDING_MODEL_NAME', 'text-embedding-3-large')

print(azure_search_endpoint, azure_search_key, search_index_name, embedding_model_name)

if not azure_search_endpoint or not azure_search_key or not search_index_name:
    logger.error("Missing Azure Search configuration environment variables.")
    raise ValueError("Missing Azure Search configuration environment variables.")

try:
    # Initialize embeddings
    embeddings = AzureOpenAIEmbeddings(model=embedding_model_name,azure_deployment="text-embedding-3-large")

    # Create the vector store
    vector_store = AzureSearch(
        azure_search_endpoint=azure_search_endpoint,
        azure_search_key=azure_search_key,
        index_name=search_index_name,
        embedding_function=embeddings.embed_query,
        vector_field_name="nasa-ai-vector-1730294759623",
        search_type="similarity",
        fields=[SimpleField(id="text_vector", name="text_vector", type="Edm.String", key=True)]
    )

    logger.info("AzureSearch vector store created successfully.")
    return vector_store

except Exception as e:
    logger.error(f"An error occurred while creating the vector store: {e}")
    raise

def create_vector_store_tool(vector_store: AzureSearch):
"""
Creates and returns a retriever tool from the provided vector store.

Args:
    vector_store (AzureSearch): The vector store instance.

Returns:
    Tool: A retriever tool instance.
"""
print("Retrieving documents for query: {query}")
try:
    retriever_ = vector_store.as_retriever()


    retrieve_policy_document = create_retriever_tool(
        retriever=retriever_,
        name="retrieve_policy_document",
        description="Search and Return  tool for HR Related questions and Leave Policy in NewYork.",
    )
    logger.info("Retriever tool created successfully.")

    print(retrieve_policy_document)
    print(type(retrieve_policy_document))
    print("--------------------------------")
    return retrieve_policy_document

except Exception as e:
    logger.error(f"An error occurred while creating the retriever tool: {e}")
    raise

def test_query_vector_store():
"""
Tests querying the AzureCognitiveSearch vector store directly.
"""
print("--------------------------------")
print("Testing query on AzureCognitiveSearch vector store...")
try:
# Create the vector store
vector_store = create_vector_store()

    # Get the retriever from the vector store
    retriever = vector_store.as_retriever()

    # Define a test query
    test_query = "What is the company's leave policy?"

    print(f"Running test query: {test_query}")

    # Retrieve relevant documents
    results = retriever.get_relevant_documents(test_query)

    # Print the retrieved documents
    if results:
        print(f"Retrieved {len(results)} documents:")
        for i, doc in enumerate(results, start=1):
            print(f"\nDocument {i} Content:\n{doc.page_content}\n")
    else:
        print("No documents were retrieved.")

except Exception as e:
    logger.error(f"An error occurred during the test query: {e}")

def test_retriever_tool():
"""
Tests the retriever tool created from the vector store.
"""
logger.info("Testing the retriever tool...")
try:
# Create the vector store
vector_store = create_vector_store()

    # Create the retriever tool
    retriever_tool = create_vector_store_tool(vector_store)

    # Define a test query
    test_query = "What is the company's leave policy?"

    results = retriever_tool.invoke(test_query)

    logger.info(f"Running test query using the retriever tool: {test_query}")

    print(results)

    # import inspect

    # logger.info("Listing all methods of retriever_tool:")
    # methods = [method_name for method_name in dir(retriever_tool)
    #            if callable(getattr(retriever_tool, method_name)) and not method_name.startswith("__")]
    # for method_name in methods:
    #     print(method_name)

    # # Alternatively, using inspect.getmembers to get more detailed information
    # logger.info("Detailed inspection of retriever_tool methods and attributes:")
    # for name, member in inspect.getmembers(retriever_tool):
    #     if inspect.ismethod(member) or inspect.isfunction(member):
    #         print(f"Method: {name}")
    #     elif not name.startswith("__"):
    #         print(f"Attribute: {name}")

    # Use the retriever tool to retrieve relevant documents
    # results = retriever_tool.retriever.get_relevant_documents(test_query)

    # Print the retrieved documents
    # if results:
    #     logger.info(f"Retrieved {len(results)} documents:")
    #     for i, doc in enumerate(results, start=1):
    #         print(doc)
    # else:
    #     logger.info("No documents were retrieved.")

except Exception as e:
    logger.error(f"An error occurred during the retriever tool test: {e}")

if name == "main":
logger.info("Starting test execution for azure_search.py...")
#test_query_vector_store()
test_retriever_tool()

I am successfully validate Azure search retriever tool independently. However, I am getting invalid tool call when trying to do this. 


There two different issue I am seeing depending on the code. If i create my retriever like below, then it executes fine, but does not invoke retriever tool, instead it returns reponse from LLM without invokving the tool for querying my RAG source.

    retrieve_policy_document = create_retriever_tool(
        retriever=retriever_,
        name="retriever",
        description="Search and Return  tool for HR Related questions and Leave Policy in NewYork.",
    )

When I change the name='retriever_policy_document', it runs into an error saying invalid tool call, as it is trying to find a method called retriever for tool call via function.

    retrieve_policy_document = create_retriever_tool(
        retriever=retriever_,
        name="retriever_policy_document",
        description="Search and Return  tool for HR Related questions and Leave Policy in NewYork.",
    )

Lets me know if more details are needed.

[tsuzukia21]

I demonstrated an article, but the last message is a HumanMessage instead of an AIMessage. Is this the correct behavior? When I want to accumulate conversation history, this specification requires editing.

[pankajxyz]

I used Bedrock with Claude. When retrieved docs are not relevant to question, it tries to reformulate the question. However, the response format from "rewrite" is not in correct format for "agent" to understand. Do I need to modify the response using regex etc or is there an elegant way? Below is the error, I am getting:

EventStreamError('An error occurred (validationException) when calling the InvokeModelWithResponseStream operation: Your API request included an assistant message in the final position, which would pre-fill the assistant response. When using tools, pre-filling the assistant response is not supported.')Traceback (most recent call last):
........
botocore.exceptions.EventStreamError: An error occurred (validationException) when calling the InvokeModelWithResponseStream operation: Your API request included an assistant message in the final position, which would pre-fill the assistant response. When using tools, pre-filling the assistant response is not supported.

[kush2022]

I want to get the final output when am using print.

`import pprint

inputs = {
"messages": [
("user", "What does Lilian Weng say about the types of agent memory?"),
]
}
for output in graph.stream(inputs):
for key, value in output.items():
pprint.pprint(f"Output from node '{key}':")
pprint.pprint("---")
pprint.pprint(value, indent=2, width=80, depth=None)
pprint.pprint("\n---\n") `


am getting this out:

---CALL AGENT---
"Output from node 'agent':"
'---'
{ 'messages': [ AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_j1pe', 'function': {'arguments': '{"query": "Lilian Weng agent memory types"}', 'name': 'retrieve_blog_posts'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 22, 'prompt_tokens': 280, 'total_tokens': 302, 'completion_time': 0.08, 'prompt_time': 0.038491926, 'queue_time': 0.18712725, 'total_time': 0.118491926}, 'model_name': 'llama-3.3-70b-versatile', 'system_fingerprint': 'fp_4e32347616', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-fbf0acc1-b2c2-4ed8-bdf7-2bc0f86d83eb-0', tool_calls=[{'name': 'retrieve_blog_posts', 'args': {'query': 'Lilian Weng agent memory types'}, 'id': 'call_j1pe', 'type': 'tool_call'}], usage_metadata={'input_tokens': 280, 'output_tokens': 22, 'total_tokens': 302})]}
'\n---\n'
---CHECK RELEVANCE---
---DECISION: DOCS NOT RELEVANT---
no
"Output from node 'retrieve':"
'---'
{ 'messages': [ ToolMessage(content='', name='retrieve_blog_posts', id='634613d9-6458-48a6-94fd-3b5f1367bedb', tool_call_id='call_j1pe')]}
'\n---\n'
---TRANSFORM QUERY---
"Output from node 'rewrite':"
'---'
{ 'messages': [ AIMessage(content='To create an improved question, let's break down the initial query and consider how it can be refined for better clarity and specificity:\n\n1. Subject: The subject of the question is Lilian Weng, presumably a researcher or expert in a field related to agents and their memory types.\n2. Topic: The topic is about the types of agent memory, which suggests a context of artificial intelligence, cognitive science, or a related field where "agents" can refer to software programs, robots, or other entities that can perceive their environment and take actions.\n3. Information Sought: The question seeks information on what Lilian Weng has stated or written about the types of agent memory.\n\nGiven these components, an improved version of the question could be more specific about the context or the type of information being sought. For example:\n\n- Improved Question: What specific types of agent memory does Lilian Weng discuss in her research or publications, and how does she categorize or differentiate between them?\n\nThis reformulated question aims to:\n- Specify that the question is about Lilian Weng's research or publications, which could help in narrowing down the sources to consult.\n- Ask for specific types of agent memory, which could elicit a more detailed and informative response.\n- Inquire about how Lilian Weng categorizes or differentiates between these types, which could provide insight into her perspective or contributions to the field.\n\nBy refining the question in this way, the inquiry becomes more targeted and is likely to yield a more precise and useful answer.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 313, 'prompt_tokens': 90, 'total_tokens': 403, 'completion_time': 1.138181818, 'prompt_time': 0.012991012, 'queue_time': 0.007083786000000002, 'total_time': 1.15117283}, 'model_name': 'llama-3.3-70b-versatile', 'system_fingerprint': 'fp_4e32347616', 'finish_reason': 'stop', 'logprobs': None}, id='run-b734b154-1151-4c98-927e-534b30264bb3-0', usage_metadata={'input_tokens': 90, 'output_tokens': 313, 'total_tokens': 403})]}
'\n---\n'
---CALL AGENT---
"Output from node 'agent':"
'---'
{ 'messages': [ AIMessage(content='', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 1, 'prompt_tokens': 624, 'total_tokens': 625, 'completion_time': 0.003636364, 'prompt_time': 0.079468607, 'queue_time': -9223372036.934244, 'total_time': 0.083104971}, 'model_name': 'llama-3.3-70b-versatile', 'system_fingerprint': 'fp_fcc3b74982', 'finish_reason': 'stop', 'logprobs': None}, id='run-1289a22f-ae47-4a70-9a91-9855fe1634e9-0', usage_metadata={'input_tokens': 624, 'output_tokens': 1, 'total_tokens': 625})]}
'\n---\n'

when I use print:

for ouput in graph.stream(inputs): for key, value in output.items(): if key == 'generate': print(value['messages'][0])

am getting this output:

---CALL AGENT---
---CHECK RELEVANCE---
---DECISION: DOCS NOT RELEVANT---
no
---TRANSFORM QUERY---
---CALL AGENT---

how can I get only the message generated as the final answer ?

[nterizakis]

I might be missing something but :

Post-processing

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

does not tie with:

# Chain
rag_chain = prompt | llm | StrOutputParser()

I believe the StrOutputParser cal should have been StrOutputParser(format_docs). Otherwise, how does StrOutputParser know how to parse? The function has not been made into a runable.

This would explain why people are getting errors as well.

[bnbabu55]

I think format_docs has to do with prompt not StrOutputParser, to prepare the prompt you extract the page_content text from the document and then call LLM with that prompt, you should be able to get the output.

StrOutputParser() does not take any arguments, you just call and your response will be a string.

[nterizakis]

Either I am blind, or I cannot see where it is called then (format_docs that is).

It would appear that StrOutputParser() does take parameters: https://python.langchain.com/api_reference/core/output_parsers/langchain_core.output_parsers.string.StrOutputParser.html

I have used a formatting function for it in the past, if I recall. Will scour my code to find it.