Update pipeline import to create a new version instead of patching

[bglearning]

Related Issues

• fixes Pipeline overwrite should likely create a new version #285

Proposed Changes?

Previously, in #283, we updated the pipeline import to fetch the latest pipeline version and patch that.

Instead, we want the following behavior:

• First try to fetch the latest version.
• If the pipeline doesn't exist (404), create it instead.
• If the latest version is a draft (is_draft == True), PATCH that version.
• Otherwise, create a new version via POST /pipelines/{name}/versions.

How did you test it?

Changes in unit and integration tests.
And tried out via the import script.

Checklist

• I have updated the referenced issue with new insights and changes
• If this is a code change, I have added unit tests
• I've used the conventional commit specification for my PR title
• I updated the docstrings
• If this is a code change, I added meaningful logs and prepared Datadog visualizations and alerts

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
deepset_cloud_sdk/_service
pipeline_service.py					395
Project Total

This report was generated by python-coverage-comment-action

[abrahamy]

We are assuming the response is a valid response without checking. If we get another status code like, say, 401 or 403 these lines may raise an error.

Instead let's do the following:

import httpx

try:
    version_response = await sef._api.get(...)
    version_response.raise_for_status()
except httpx.HTTPStatusError as e:
    if e.response.status_code != HTTPStatus.NOT_FOUND:
        raise
     # the pipeline does not exist, let's create it.
     return await self._api.create_pipeline(...)

# pass the response
version_body = ...

if is_draft:
    # do draft things
    return

# do non-draft things

[bglearning]

Hi @abrahamy Updated this accordingly.

See comment above.