Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Run imported from CSV cannot be repaired #248

Open
4 tasks done
leonardmq opened this issue Mar 16, 2025 · 1 comment
Open
4 tasks done

[Bug] Run imported from CSV cannot be repaired #248

leonardmq opened this issue Mar 16, 2025 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@leonardmq
Copy link
Contributor

leonardmq commented Mar 16, 2025

Describe the bug
The run repairing flow does not work on CSV-imported task runs. This is because the file_import data source does not have a prompt_id, so it fails to create the repair prompt.

Error shown in the UI is: Unexpected error: Prompt builder 'None' is not a valid prompt builder.

In terms of a fix, not immediately clear where the best place for the user to indicate the prompt type should be - possibly this is a piece of information that they should include in the CSV as a field with possibly a fallback to basic zero shot prompt type if left blank. Alternatively in the CSV import modal, through a dropdown but it does not really work if the samples they import span different types of prompts.

Checks

  • I've read the troubleshooting guide
  • - n/a - I've tried to reproduce the problem using another model, and confirmed it's not an issue specific to the model I've chosen.
  • I've searched the docs for a solution
  • I've searched for existing Github issues/discussions

To Reproduce
Steps to reproduce the behavior:

  1. Go to Dataset
  2. Click on Upload File
  3. Choose a valid file in the modal and click Upload
  4. Click on one of the newly added rows
  5. Rate it 2 stars
  6. Fill in the Repair Instructions text area in Repair Output
  7. Click Attempt Repair
  8. UI shows an error: Unexpected error: Prompt builder 'None' is not a valid prompt builder

Expected behavior
Clicking Attempt Repair on a CSV-imported task run should attempt to repair it.

Screenshots
Image

Error Logs

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 93, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 144, in simple_response
    await self.app(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/app/desktop/studio_server/repair_api.py", line 38, in run_repair
    repair_task_input = RepairTaskRun.build_repair_task_input(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/libs/core/kiln_ai/adapters/repair/repair_task.py", line 67, in build_repair_task_input
    original_prompt = cls._original_prompt(task_run, original_task)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Kiln/libs/core/kiln_ai/adapters/repair/repair_task.py", line 61, in _original_prompt
    raise ValueError(f"Prompt builder '{prompt_id}' is not a valid prompt builder")
ValueError: Prompt builder 'None' is not a valid prompt builder

System Information:

  • OS: MacOS
  • Browser: chrome
  • Kiln app Version: latest main (0.12.1)

Additional context
The error is thrown from here:

raise ValueError(f"Prompt builder '{prompt_id}' is not a valid prompt builder")

@leonardmq leonardmq added the bug Something isn't working label Mar 16, 2025
@scosman
Copy link
Collaborator

scosman commented Mar 17, 2025

Let's default to basic prompt builder if it's missing?

  • Top UI show something like "None (imported)", or just hide that field
  • Repair API defaults to Basic prompt if none provided.

You want to fix or should I?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants