Skip to content

Add the multi2vec vectorizers - #489

Merged
bevzzz merged 10 commits into
v6from
v6-multi2vec
Oct 26, 2026
Merged

bevzzz merged 10 commits into
v6from
v6-multi2vec

Conversation

@g-despot

@g-despot g-despot commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Adds all 9 multimodal vectorizers: cohere, google, jinaai, nvidia, voyageai, twelvelabs, aws, clip and bind. Same conventions as the text2vec modules; builds on #484.

Field weights are plain lists, parallel to the field lists, matching the wire format. multi2vec-cohere was tested end to end against the real Cohere API (text and image objects both vectorized).

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@g-despot
g-despot requested a review from bevzzz October 15, 2026 08:53
Base automatically changed from v6-more-vectorizers to v6 October 25, 2026 14:46
Passing field names and theirs weights is error-prone.
Marshaling a combined field is a little more involved than simply
relying on the struct's format, but hey, we want a good UX
@bevzzz

bevzzz commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Did some refactoring to allow users to specify properties alongside their weights:

TextFields: []modules.WeightedField{
    {Property: "title", Weight: 0.2},
    {Property: "lyrics", Weight: 0.3},
},
ImageFields: []modules.WeightedField{
    {Property: "cover", Weight: 0.5},
},

This way the user doesn't need to be careful about maintaining the order the properties were passed in when giving them weights. Notice that expected values in unit tests are not changed, so the refactor worked :)

Also moved all modules to xxx/modules.go file respectively, instead of having a separate file for each module.

@bevzzz bevzzz self-assigned this Oct 25, 2026
@bevzzz
bevzzz requested review from tsmith023 and removed request for bevzzz October 25, 2026 18:10
Comment thread modules/weighted_field.go Outdated
@bevzzz
bevzzz merged commit 0dd0dfb into v6 Oct 26, 2026
3 checks passed
@bevzzz
bevzzz deleted the v6-multi2vec branch October 26, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants