Skip to content

How to properly exclude language_instruction from statistics computation during finetuning? #163

@myrtheiw

Description

@myrtheiw

I'm using a custom RLDS dataset that includes a language_instruction key (a string) alongside standard fields like action and observation.proprio. During finetuning, I run into this error:

tensorflow.python.framework.errors_impl.UnimplementedError: Cast string to float is not supported

This seems to happen during the dataset statistics computation step, where Octo tries to compute statistics for all fields—including strings.

Is there a recommended way to handle this kind of multimodal input (especially language) that doesn't require normalisation during this phase?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions