[Question]: How to use it with websockets response

[michalsn]

What happened?

I'm not able to get the response when using WebSockets. I'm using h2oGPT. The Gratio Client in Python works fine, but here I'm struggling to make it work.

In PHP when calling the code:

$result = $client->predict([['instruction_nochat' => 'Who are you?', 'stream_output' => 'True']], apiName: 'submit_nochat_api');

I get the response like this:

object(SergiX44\Gradio\DTO\Output)#239 (4) {
  ["is_generating"]=>
  bool(false)
  ["duration"]=>
  float(0)
  ["average_duration"]=>
  float(0)
  ["data"]=>
  array(0) {
  }
}

When I tried to debug what was actually returned from websockets I got this:

string(19) "{"msg":"send_hash"}"

string(190) "{"msg":"estimation","rank":0,"queue_size":1,"avg_event_process_time":3.7834130906127212,"avg_event_concurrent_process_time":0.05911582954082377,"rank_eta":3.7834130906127212,"queue_eta":0.0}"

string(19) "{"msg":"send_data"}"

string(24) "{"msg":"process_starts"}"

string(365) "{"msg":"process_completed","output":{"error":"An event handler (fn) didn't receive enough input values (needed: 5, got: 1).\nCheck if the event handler calls a Javascript function, and make sure its return value is correct.\nWanted inputs:\n [state, state, state, state, textbox]\nReceived inputs:\n [{'instruction_nochat': 'Who are you?'}]"},"success":false}"

I'm not sure if this is my fault with configuration, or specific h2oGPT implementation which is not compatible with this library in this case. I would really appreciate the hint, to make it work.

How to reproduce the bug

require __DIR__ . '/vendor/autoload.php';

use SergiX44\Gradio\Client;

$client = new Client('https://gpt.h2o.ai');
//$client = new Client('http://localhost:7860');
$result = $client->predict([['instruction_nochat' => 'Who are you?', 'stream_output' => 'True']], apiName: 'submit_nochat_api');

echo '<pre>';
var_dump($result);
echo '</pre>';

Package Version

0.0.2

PHP Version

8.2.11

Which operating systems does with happen with?

macOS

Notes

No response

[sergix44]

Do you have a list of the endpoints supported by that model?
It seems that you not passing enough params 🤔
The websocket mode should be transparent to the end user, if supported by the model it will be used instead of http mode.
I'll take a look and try to reproduce the issue

[michalsn]

Do you have a list of the endpoints supported by that model?

I only have this: https://gpt.h2o.ai/config

It seems that you not passing enough params 🤔

Yes, I know that the error indicates that there are too few params (the config above also states there should be 5 of them), although the same params as an input in the Python client work fine.

Python example:

from gradio_client import Client
import ast
import time

HOST = 'http://localhost:7860'
client = Client(HOST)
api_name = '/submit_nochat_api'
prompt = "Who are you?"
kwargs = dict(instruction_nochat=prompt, stream_output=True)

job = client.submit(str(dict(kwargs)), api_name=api_name)

text_old = ''
while not job.done():
    outputs_list = job.communicator.job.outputs
    if outputs_list:
        res = job.communicator.job.outputs[-1]
        res_dict = ast.literal_eval(res)
        text = res_dict['response']
        new_text = text[len(text_old):]
        if new_text:
            print(new_text, end='', flush=True)
            text_old = text
        time.sleep(0.01)
# handle case if never got streaming response and already done
res_final = job.outputs()
if len(res_final) > 0:
    res = res_final[-1]
    res_dict = ast.literal_eval(res)
    text = res_dict['response']
    new_text = text[len(text_old):]
    print(new_text)

To be fair, I'm very new to using AI models, so it might be that this is my fault... anyway thank you for looking into this. Much appreciate.