Continuous output from remote functions

[psukys]

So far, I've only used a pretty trivial implementation that accepts continuous output from a remote module. Duplicated wrapper to differentiate upon a set parameter self.continuous to not break previous implementations.

Remote module used foo.py:

def test():
    for i in range(1, 10):
        channel.send(i)
    channel.send(None)

if __name__ == '__channelexec__':
    for item in channel:
        eval(item)

Solves #38

[psukys]

This is not straightforward - there needs to be a finalizer and I have doubts about it being on remoto side (rather than remote function explicitly finalizing)

[alfredodeza]

isn't it possible to raise an exception here instead of just getting a None?

[psukys]

I'm not aware (and experienced) with pickling capabilities in this end. What I thought could be an alternative is having some side-channel or trigger that gives us end-of-stream (since we might want to reuse the channel for further communication and so on)

[alfredodeza]

This would basically mean that no function would be allowed to not return anything (which implicitly returns a None) or explicitly return a None unless it wants to really close the channel.

I don't have a use case for this explicitly, but I would prefer an exception. There is no need to pickle anything here though, as you can see the exception is being handled in the next few lines and we are able to poke inside. Something like EOFError('close') would be enough for us to check this?

[psukys]

Sorry for being inactive for quite a while. I saw that there's util.RemoteError, what is its relation to execnet.RemoteError functionality?

[alfredodeza]

the naming is unfortunate, but it is just a helper to try to detect the remote exception name. There is no relation to execent.RemoteError

[psukys]

Also, I'm not sure regarding EOFError from guest side. Having something like this:

def test():
    for i in range(1, 10):
        channel.send(i)
    channel.send(EOFError('close'))

if __name__ == '__channelexec__':
    for item in channel:
        eval(item)

returns me a DumpError as EOFError is not serializable, while:

def test():
    for i in range(1, 10):
        channel.send(i)
    raise EOFError('close')

if __name__ == '__channelexec__':
    for item in channel:
        eval(item)

finishes silently (not caught by execnet?)

[alfredodeza]

I knew this wouldn't be straightforward :( If EOFError is not serializable, I would try with other exceptions. Basically, to find a way where the connection is able to detect there is nothing else coming in, in a non-generic way (e.g. implicit None can't be used).

[psukys]

@alfredodeza so far, what I've come up with is explicitly sending something specific back to (in this scenario None) host. While execnet's documentation isn't very specific. As well, what I managed to communicate in #pytest on IRC, was the aforementioned method.

Not sure how to progress, maybe include method this in README.md and wrap it up?