fix: Update OpenAPIServiceConnector to new ChatMessage

[vblagoje]

Why:

Update OpenAPIServiceConnector to handle new ChatMessage format

• fixes OpenAPIServiceConnector does not support the v2.9.x ChatMessage structure #8774

What:

• Patched the request method in OpenAPI to handle raw responses more effectively.
• Reworked the processing of chat messages to extract tool calls and handle them for OpenAPI function invocations.

How can it be used:

The changes affect how OpenAPIServiceConnector handles new ChatMessage format involving build in tool_calls property.

Example usage:

connector.run(messages=[message], service_openapi_spec=a_spec_as_a_dict_object)

Where message should now contain tool_calls property.

How did you test it:

Tests verify new behavior of message processing with checks for valid tool calls. Patched OpenAPI methods are mocked to assess correct invocation and response handling. Integration tests are skipped unless specific API keys are set.

Notes for the reviewer:

Pay special attention to the new patch applied to the request method of OpenAPI, specifically concerning raw response handling. Ensure the test coverage is sufficient for edge cases in tool call processing and response verification.

[coveralls]

Pull Request Test Coverage Report for Build 13198769367

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 83 unchanged lines in 7 files lost coverage.
• Overall coverage increased (+0.9%) to 92.178%

---

Files with Coverage Reduction	New Missed Lines	%
components/generators/hugging_face_api.py	1	96.3%
core/pipeline/pipeline.py	1	96.43%
components/embedders/sentence_transformers_document_embedder.py	2	96.77%
components/embedders/sentence_transformers_text_embedder.py	2	96.3%
components/generators/chat/hugging_face_api.py	3	96.46%
core/pipeline/base.py	19	94.59%
components/connectors/openapi_service.py	55	62.59%

Totals
Change from base Build 13141979385:	0.9%
Covered Lines:	9027
Relevant Lines:	9793

---

💛 - Coveralls

[vblagoje]

@richardpaulhudson would you please give this branch a spin in your project by directly installing it via pip install git+https://github.com/deepset-ai/haystack.git@open_api_chat_message

[vblagoje]

@mpangrazzi The reason you'll see this ugly monkey patch is better described at Dorthu/openapi3#124

For better integration with LLMs handling REST endpoint responses I believe that response validation should be optional. Currently the openapi3 implementation doesn’t offer a way to disable model validation, which can sometimes lead to unintended issues and poor overall UX. In response to user feedback we'll soon deprecate OpenAPIServiceConnector for OpenAPIConnector and OpenAPITool (see internal https://github.com/deepset-ai/haystack-private/issues/104 for details).

[vblagoje]

@dfokina I don't think there are any doc changes required here as we are not changing any public API signatures - we are just updating this component to work with the latest ChatMessage

[anakin87]

Just a (slightly unrelated) curiosity: OpenAPIConnector vs OpenAPITool
How do envision them? What's the difference?

[vblagoje]

Of course @anakin87 , shared this during brainstorming with Luke and Julian so here is the copy for the record:

Difference between OpenAPIConnector and OpenAPITool

• OpenAPIConnector requires you to generate and pass parameters manually. It then invokes the remote REST endpoint.
• OpenAPITool generates invocation parameters automatically from your chat message and directly invokes the remote REST endpoint.

---

OpenAPIConnector

Pros:

• Easy to set up and explicit
• No hallucination of invocation parameters
• More deterministic behavior

Cons:

• Requires manual preparation of REST invocation parameters (coming from other components in the pipeline or run pipeline parameters)
• You need to structure the parameters into a JSON payload yourself

---

OpenAPITool

Pros:

• Quick and easy to set up
• Easy to configure and use
• Just another tool in our new tooling architecture

Cons:

• The LLM might not always generate the correct parameters (rarely due to hallucination, but more likely due to message misalignment with required invocation parameters)
• Less deterministic when high precision is required

---

[mpangrazzi]

Looks good! I only feel that some edge cases might be not covered by tests. For example, when request body is missing or when content type is unsupported. Do you think it may worth covering those?

[vblagoje]

Looks good! I only feel that some edge cases might be not covered by tests. For example, when request body is missing or when content type is unsupported. Do you think it may worth covering those?

Yes, good point. Let me see what I can do 🙏

[dfokina]

While I was looking, found a few improvements to the docstrings