-
Notifications
You must be signed in to change notification settings - Fork 13.8k
[FLINK-38424][planner] Support to parse VECTOR_SEARCH function #27039
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
import java.io.Serializable; | ||
|
||
/** | ||
* A {@link DynamicTableSource} that search rows of an external storage system by one or more |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: search -> searches
* vectors during runtime. | ||
* | ||
* <p>Compared to {@link ScanTableSource}, the source does not have to read the entire table and can | ||
* lazily fetch individual values from a (possibly continuously changing) external table when |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be interesting to compare it to a LookupTableSource also.
PARAM_SEARCH_TABLE, | ||
PARAM_COLUMN_TO_SEARCH, | ||
PARAM_COLUMN_TO_QUERY, | ||
PARAM_TOP_K)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I an see what TOP_K is from googling. It would be useful to add in the documentation describing the parameters with this change - including the default. I wonder if we should add paging, to be able to handle a large number of results or is this not done with vector databases.
What is the purpose of the change
Support to parse VECTOR_SEARCH function.
Brief change log
Verifying this change
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:
@Public(Evolving)
: (yes / no)Documentation