Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(proto): Add Write relation to support multiple outputs #238

Open
rtpsw opened this issue Jul 3, 2022 · 2 comments
Open

feat(proto): Add Write relation to support multiple outputs #238

rtpsw opened this issue Jul 3, 2022 · 2 comments
Labels
enhancement New feature or request help wanted No one is currently implementing but it seems like a good idea

Comments

@rtpsw
Copy link

rtpsw commented Jul 3, 2022

Currently, a Substrait plan produces a single output corresponding to its root relation. In some cases, multiple outputs are needed, e.g., for multi-table computation as well as for check-point and debug output. The solution proposed here is to add a Write relation, which is a pass-through relation that does not modify the table yet writes it out to some external sink.

@westonpace westonpace added enhancement New feature or request help wanted No one is currently implementing but it seems like a good idea labels Mar 1, 2023
@westonpace
Copy link
Member

I'm going to leave this open as a "generally good idea" but I'm adding the help wanted label as I don't think you're actively working on this?

@rtpsw
Copy link
Author

rtpsw commented Mar 3, 2023

AFAIK, we ended up solving this using a NamedTapRel extension, i.e., outside the scope of Substrait itself. I think this can be closed. @icexelloss?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted No one is currently implementing but it seems like a good idea
Projects
None yet
Development

No branches or pull requests

2 participants