Fix kv cache offloading in MRL#2633

sidsingh-nvidia wants to merge 1 commit intoNVIDIA:mainNVIDIA/Megatron-LM:mainfrom

sidsingh-nvidia:siddharth/rl-fix-kv-offloadingsidsingh-nvidia/Megatron-LM:siddharth/rl-fix-kv-offloadingCopy head branch name to clipboard