feat: Add OpenAPIConnector component, improve OpenAPI integration

[vblagoje]

Why:

Enables invocation of REST endpoints specified in an OpenAPI specification. This component will supersede our current OpenAPIServiceConnector that needs to work in pair with OpenAPIServiceToFunctions to achieve what a new future tool - OpenAPITool will do as one tool in our new tooling architecture.

What:

• Introduced OpenAPIConnector, a new component to invoke REST endpoints from OpenAPI specs.
• Updated __init__.py to include OpenAPIConnector in the exports.
• Added comprehensive unit and integration tests for OpenAPIConnector to ensure its correctness and reliability.

How can it be used:

The OpenAPIConnector can be used to perform API operations defined in an OpenAPI specification. Initialize it with an OpenAPI spec URL, file path, or string, and use it to call endpoints:

from haystack.utils import Secret
from haystack.components.connectors.openapi import OpenAPIConnector

connector = OpenAPIConnector(
    openapi_spec="https://bit.ly/serperdev_openapi",
    credentials=Secret.from_env_var("SERPERDEV_API_KEY")
)
response = connector.run(
    operation_id="search",
    parameters={"q": "Who was Nikola Tesla?"}
)

How did you test it:

Tests cover initialization, serialization, deserialization, and execution of operations to ensure component functionality.

Notes for the reviewer:

• The test_openapi_connector.py file contains both unit and integration tests; some tests require environment variables like SERPERDEV_API_KEY or GITHUB_TOKEN to run integration tests, which are conditionally skipped if not present.
• Pay special attention to the integration with openapi-llm and the potential requirements for the environment setup.

[coveralls]

Pull Request Test Coverage Report for Build 13196768236

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.02%) to 92.674%

---

Totals
Change from base Build 13183889468:	0.02%
Covered Lines:	9057
Relevant Lines:	9773

---

💛 - Coveralls

[vblagoje]

Why are we introducing the OpenAPIConnector and OpenAPITool

• OpenAPIConnector requires you to generate and pass parameters manually. It then invokes the remote REST endpoint.
• OpenAPITool generates invocation parameters automatically from your chat message and directly invokes the remote REST endpoint.

---

OpenAPIConnector

Pros:

• Easy to set up and explicit
• No hallucination of invocation parameters
• More deterministic behavior

Cons:

• Requires manual preparation of REST invocation parameters (coming from other components in the pipeline or run pipeline parameters)
• You need to structure the parameters into a JSON payload yourself

---

OpenAPITool

Pros:

• Quick and easy to set up
• Easy to configure and use
• Just another tool in our new tooling architecture

Cons:

• The LLM might not always generate the correct parameters (rarely due to hallucination, but more likely due to message misalignment with required invocation parameters)
• Less deterministic when high precision is required

---

These two new components will replace existing OpenAPIServiceConnector and OpenAPIServiceToFunctions which will be soon deprecated.

[mpangrazzi]

Looks Good! But should this component be mentioned in docs?

[vblagoje]

Right @mpangrazzi - I'll create a new doc for it 🙏