[QUESTION] variable tensor shape when pipeline parallelism (pp) #1072
Unanswered
KookHoiKim
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I am working with LLaVA code and i have a question about sequence length when using pipeline parallelism.
In my understanding, tensor shape for recv, send is fixed using args.seq_length in pipeline_parallel/schedules.py.
And if padding tokens make up most of the input, it becomes very inefficient in terms of memory or execution speed.
Is there any way to use variable input length when using pipeline parallel?
Thanks.
Beta Was this translation helpful? Give feedback.
All reactions