feat(core): Capture raw provider response

[AmazingcatAndrew]

Summary

Resolves #33884

Design Decisions

1. Configuration Location

• Choice: Place include_raw_response in model wrapper classes, NOT core message classes
• Rationale:
  • This is a usage-time option, not a core messaging concern
  • Different providers can independently support it
  • Keeps core message design clean

2. Data Structure Flexibility

• raw_response Type: dict[str, Any] | list[dict[str, Any]] | None
• Rationale:
  • Single response → dict (simpler API)
  • Multiple responses → list[dict] (clear aggregation)
  • None → unset/disabled

3. Stream Processing Strategy

• Trigger: Call add_ai_message_chunks() only when chunk_position="last" received
• Rationale: Ensures complete stream before aggregation

4. Backward Compatibility

• All new parameters have defaults
• Existing code runs unchanged
• raw_response completely optional

Key Changes

1. Core Message System (libs/core/langchain_core/messages/ai.py)

AIMessage Class

• New Field: raw_response: dict[str, Any] | list[dict[str, Any]] | None = None
• Constructor Support: Updated __init__() to accept and store raw_response parameter
• Serialization: Modified dict() method to include non-None raw_response values
• Documentation: Added parameter documentation

AIMessageChunk Class

• New Field: raw_response: dict[str, Any] | None = None
• Purpose: Store individual chunk's raw response during streaming

New Function: add_ai_message_chunks()

Aggregates multiple AIMessageChunk objects into a single final AIMessage

Key Features:

• Intelligent raw_response merging:
  • Single response → use directly (dict)
  • Multiple responses → preserve as list
  • None values → filtered out
• Triggered when chunk_position="last" marker received
• Handles all other chunk fields (content, tools, metadata)

Example Usage:

from langchain_core.messages.ai import add_ai_message_chunks

# Aggregate chunks during streaming
final_message = add_ai_message_chunks(chunk1, chunk2, chunk3)
# If chunks have raw_response, they're merged intelligently
print(final_message.raw_response)  # dict or list[dict] depending on count

2. Message Utilities (libs/core/langchain_core/messages/utils.py)

message_chunk_to_message() Function

• Added Logic: Transfer raw_response from chunk to final message
• Condition: Only transfer if value is not None
• Code:

  if hasattr(chunk, "raw_response") and chunk.raw_response is not None:
      data["raw_response"] = chunk.raw_response

3. Public API Exports (libs/core/langchain_core/messages/__init__.py)

all List

• Added: "add_ai_message_chunks"

_dynamic_imports Dictionary

• Added mapping: "add_ai_message_chunks": "ai"

4. OpenAI Integration (libs/partners/openai/langchain_openai/chat_models/base.py)

BaseChatOpenAI Class

New Configuration Field (line ~686):

include_raw_response: bool = False

• Controls whether to capture raw OpenAI API responses
• Default: False (maintains backward compatibility)
• Type: Pydantic model field with documentation

Implementation Locations:

1. _convert_chunk_to_generation_chunk() (~line 1071)

   • Captures streaming chunk's raw response
   • Code: if self.include_raw_response: message_chunk.raw_response = chunk

2. _stream() (~line 1274)

   • Handles sync streaming
   • Transfers final chunk to generation message
   • Code: if self.include_raw_response and chunk_position == "last": ...

3. _create_chat_result() (~line 1440)

   • Handles non-streaming calls
   • Attaches response_dict to message
   • Code: if self.include_raw_response and isinstance(message, AIMessage): ...

4. _astream() (~line 1529)

   • Handles async streaming
   • Same logic as sync stream

Scope

Currently, only the OpenAI model class has been adapted.
If this PR is accepted, similar adaptations will be implemented for other partner integrations (Anthropic, Google, Bedrock, etc.) in follow-up PRs.

Testing

• All new and existing tests pass locally.
• Added isolated unit tests under tests/unit_tests/core/ and tests/unit_tests/partners/openai/.
• No external API calls are made; all model interactions are fully mocked.

[codspeed-hq]

CodSpeed Performance Report

Merging #33922 will improve performances by 31.32%

_{Comparing AmazingcatAndrew:feat/capture-raw-provider-response (40d9386) with master (52b1516)}

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

⚡ 9 improvements
✅ 10 untouched
⏩ 15 skipped¹

Benchmarks breakdown

	Mode	Benchmark	BASE	HEAD	Change
⚡	WallTime	test_async_callbacks_in_sync	24.3 ms	18.5 ms	+31.32%
⚡	WallTime	test_import_time[BaseChatModel]	510.7 ms	460.5 ms	+10.89%
⚡	WallTime	test_import_time[CallbackManager]	446.1 ms	403.7 ms	+10.51%
⚡	WallTime	test_import_time[ChatPromptTemplate]	580.5 ms	524.1 ms	+10.75%
⚡	WallTime	test_import_time[Document]	184.8 ms	166.1 ms	+11.28%
⚡	WallTime	test_import_time[HumanMessage]	262.2 ms	237.3 ms	+10.47%
⚡	WallTime	test_import_time[InMemoryVectorStore]	592.4 ms	537.6 ms	+10.19%
⚡	WallTime	test_import_time[PydanticOutputParser]	519.2 ms	458.7 ms	+13.19%
⚡	WallTime	test_import_time[tool]	504.8 ms	447.2 ms	+12.87%

Footnotes

1. 15 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[AmazingcatAndrew]

I will handle the failing checks

[AmazingcatAndrew]

So far, all the tests (make format, make lint, make test) pass locally. But there are still some CI failiings I need to check.

[AmazingcatAndrew]

Status Update

✅ All core tests passing
✅ Lint, format, and type checks passing

Remaining: Pydantic Compatibility Snapshots

The following tests need snapshot updates due to the new raw_response field in AIMessage schema:

• test-pydantic (libs/core, 2.7.0, 3.12)
• test-pydantic (libs/core, 2.8.0, 3.12)
• test-pydantic (libs/core, 2.9.0, 3.12)

The schema changes are expected - the new field is now included in the serialized message format.

Could a maintainer help regenerate these snapshots? I attempted to update them locally but encountered environment setup issues on Windows/WSL.

Thank you!