Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce rewrite tools on Rel. #29

Merged
merged 4 commits into from
Jul 20, 2024
Merged

Conversation

scgkiran
Copy link
Contributor

Add GetInputs, Copy, CopyWithExpressionRewrite to Rel

Add GetInputs, Copy, CopyWithExpressionRewrite to Rel
@CLAassistant
Copy link

CLAassistant commented Jul 18, 2024

CLA assistant check
All committers have signed the CLA.

@vbarua
Copy link
Member

vbarua commented Jul 18, 2024

This is very promising! We have something similar to this in substrait-java in the RelCopyOnWriteVisitor and ExpressionCopyOnWriteVisitor which allows us to implement arbitrary tree rewrites quite easily.

Copy link
Contributor

@jacques-n jacques-n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looking good. Few minor proposed changes.

errors.go Show resolved Hide resolved
plan/plan.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
@jacques-n
Copy link
Contributor

...something similar to this in substrait-java...

Lol, why invent when you can just copy useful patterns, right? :D

plan/relations.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
Comment on lines 240 to 250
newExprs, err := v.baseReadRel.copyExpressions(rewriteFunc)
if err != nil {
return nil, err
}
if newExprs == nil {
return nil, nil
}
nt := *v
nt.filter = newExprs[0]
nt.bestEffortFilter = newExprs[1]
return &nt, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than repeating this over and over, it feels like we could easily write a simple generic helper function, something like:

func (b *baseReadRel) updateFilters(filter, bestEffortFilter expr.Expression) {
    b.filter, b.bestEffortFilter = filter, bestEffortFilter
}

type exprRewriter interface {
    copyExpressions(func(expr.Expression) (expr.Expression, error)) ([]expr.Expression, error)
    updateFilters(filter, bestEffortFilter expr.Expression)
}

func copyRewriteHelper[T exprRewriter](r *T, rewriteFunc func(expr.Expression) (expr.Expression, error)) (*T, error) {
    newExprs, err := r.copyExpressions(rewriteFunc)
    if err != nil {
        return nil, err
    }

    if newExprs == nil {
        return r, nil
    }

    nt := *r
    nt.updateFilters(newExprs[0], newExprs[1])
    return &nt, nil
}

...

func (v *VirtualTableReadRel) CopyWithExpressionRewrite(rewriteFunc func(expr.Expression) (expr.Expression, error), newInputs ...Rel) (Rel, error) {
    return copyRewriteHelper(v, rewriteFunc)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hit some issues when I tried this.

  1. func copyRewriteHelper[T exprRewriter](r *T, rewriteFunc RewriteFunc) (*T, error)
    - r is a pointer to interface, complains about unresolved copyReference
    - r is a pointer to interface, so nt := *r won't copy the object?
  2. func copyRewriteHelper[T exprRewriter](r T, rewriteFunc RewriteFunc) (*T, error)
    - nt := *r this throws error invalid indirect of type T. Looks like we cannot copy object using interface type variable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below works, but still some code duplication of is there.

type exprRewriter interface {
	copyExpressions(rewriteFunc RewriteFunc) ([]expr.Expression, error)
	getExpressions() []expr.Expression
	updateFilters(filter []expr.Expression)
	copy() Rel
}

func copyRewriteHelper(n exprRewriter, rewriteFunc RewriteFunc) (Rel, error) {
	newExprs, err := n.copyExpressions(rewriteFunc)
	if err != nil {
		return nil, err
	}
	if slices.Equal(newExprs, n.getExpressions()) {
		return n.(Rel), nil
	}
	nt := n.copy()
	nt.(exprRewriter).updateFilters(newExprs)
	return nt, nil
}

func (n *NamedTableReadRel) copy() Rel {
	ntr := *n
	return &ntr
}

func (n *NamedTableReadRel) CopyWithExpressionRewrite1(rewriteFunc RewriteFunc, _ ...Rel) (Rel, error) {
	return copyRewriteHelper(n, rewriteFunc)
}

plan/relations.go Outdated Show resolved Hide resolved
plan/plan.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
Comment on lines 723 to 727
join := *j
join.left = newInputs[0]
join.right = newInputs[1]
join.expr = exprs[0]
join.postJoinFilter = exprs[1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we do

join := j.Copy(newInputs...)
join.expr, join.postJoinFilter = exprs[0], exprs[1]

instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did this partially. Using j.Copy asks for a type cast to JoinRel.

plan/relations.go Outdated Show resolved Hide resolved
plan/relations.go Outdated Show resolved Hide resolved
@scgkiran scgkiran marked this pull request as ready for review July 19, 2024 05:59
@scgkiran scgkiran changed the title WIP: Introduce rewrite tools on Rel. Introduce rewrite tools on Rel. Jul 19, 2024
Copy link
Contributor

@jacques-n jacques-n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I see most of the feedback that zeroshade provided has been addressed. I think we should merge this. If the remaining items are not sufficient, let's address in a follow-up ticket.

@jacques-n jacques-n merged commit 9ae8754 into substrait-io:main Jul 20, 2024
4 checks passed
@scgkiran scgkiran deleted the SD-7630 branch July 20, 2024 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants