Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Fixes bug in GAE advantage estimation when gammalmbda is a Tensor #2773

Closed
wants to merge 2 commits into from

Remove terminated second def

5411dd9
Select commit
Loading
Failed to load commit list.
Closed

[Bug] Fixes bug in GAE advantage estimation when gammalmbda is a Tensor #2773

Remove terminated second def
5411dd9
Select commit
Loading
Failed to load commit list.
PyTorch Bot / Dr.CI completed Feb 7, 2025 in 0s

Dr.CI classification results

{"FAILED":[{"workflowId":13203489952,"workflowUniqueId":37755465,"id":36866378281,"runnerName":"i-0aa2f9774d77b18e9","authorEmail":"[email protected]","name":"Generate documentation / build-docs (3.9, 12.4) / linux-job","jobName":"build-docs (3.9, 12.4) / linux-job","conclusion":"failure","completed_at":"2025-02-07T17:58:53.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489952/job/36866378281","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["RuntimeError: Command docker exec -t 088e3110f09602a10e579c220ec7b29d403feee18f4a88ff993700b0ca14d551 /exec failed with exit code 2"],"failure_lines":["RuntimeError: Command docker exec -t 088e3110f09602a10e579c220ec7b29d403feee18f4a88ff993700b0ca14d551 /exec failed with exit code 2"],"failure_context":[],"time":"2025-02-07T17:43:01.000000000Z"},{"workflowId":13203489926,"workflowUniqueId":61790681,"id":36866375157,"runnerName":"i-012ab9bf60c5a0a03","authorEmail":"[email protected]","name":"Continuous Benchmark (PR) / GPU Pytest benchmark","jobName":"GPU Pytest benchmark","conclusion":"failure","completed_at":"2025-02-07T18:15:07.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489926/job/36866375157","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["Workflow failed! Resource not accessible by integration"],"failure_lines":["##[error]Workflow failed! Resource not accessible by integration"],"failure_context":[],"time":"2025-02-07T17:42:57.000000000Z"},{"workflowId":13203489926,"workflowUniqueId":61790681,"id":36866375605,"runnerName":"GitHub Actions 494","authorEmail":"[email protected]","name":"Continuous Benchmark (PR) / CPU Pytest benchmark","jobName":"CPU Pytest benchmark","conclusion":"failure","completed_at":"2025-02-07T18:09:38.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489926/job/36866375605","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["Workflow failed! Resource not accessible by integration"],"failure_lines":["##[error]Workflow failed! Resource not accessible by integration"],"failure_context":[],"time":"2025-02-07T17:42:58.000000000Z"},{"workflowId":13203489945,"workflowUniqueId":61166099,"id":36866376680,"runnerName":"i-056e524644cbff1ad","authorEmail":"[email protected]","name":"RLHF Tests on Linux / unittests (3.9, 12.4) / linux-job","jobName":"unittests (3.9, 12.4) / linux-job","conclusion":"failure","completed_at":"2025-02-07T17:52:31.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489945/job/36866376680","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["RuntimeError: Command docker exec -t 650ad64878f480c83e0d4bc506bf320e5b8272ea8c4765b294491d3690b99cc7 /exec failed with exit code 1"],"failure_lines":["RuntimeError: Command docker exec -t 650ad64878f480c83e0d4bc506bf320e5b8272ea8c4765b294491d3690b99cc7 /exec failed with exit code 1"],"failure_context":[],"time":"2025-02-07T17:42:59.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866378331,"runnerName":"i-0100d8e18a5e8f664","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-cpu (3.10) / linux-job","jobName":"tests-cpu (3.10) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:21:32.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866378331","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:01.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866379592,"runnerName":"i-09a1e44df39e05dab","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-cpu (3.11) / linux-job","jobName":"tests-cpu (3.11) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:20:57.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866379592","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:03.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866380784,"runnerName":"i-0b0123e34830a267f","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-cpu (3.12) / linux-job","jobName":"tests-cpu (3.12) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:22:43.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866380784","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:04.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866381349,"runnerName":"i-0f935db3601be2ebf","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-cpu-oldget (3.12) / linux-job","jobName":"tests-cpu-oldget (3.12) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:22:42.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866381349","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:05.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866382109,"runnerName":"i-00f713033b3dfc9e8","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-cpu (3.9) / linux-job","jobName":"tests-cpu (3.9) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:22:37.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866382109","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:06.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866382485,"runnerName":"i-0c3e576bb124f3fa5","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-gpu (3.11, 12.4) / linux-job","jobName":"tests-gpu (3.11, 12.4) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:46:25.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866382485","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:06.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866383532,"runnerName":"i-06a6eca4b83d678bc","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-optdeps (3.11, 12.4) / linux-job","jobName":"tests-optdeps (3.11, 12.4) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:24:11.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866383532","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:08.000000000Z"},{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866384368,"runnerName":"i-00ef7d74735342d74","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job","jobName":"tests-stable-gpu (3.10, 11.8) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:48:19.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866384368","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0]"],"failure_lines":["FAILED test/test_cost.py::TestValues::test_gae_param_as_tensor[tensor_single_element-tensor-True-dtype1-N2-device0] - AssertionError: Tensor-likes are not close!"],"failure_context":[],"time":"2025-02-07T17:43:09.000000000Z"},{"workflowId":13203489977,"workflowUniqueId":60501564,"id":36866378294,"runnerName":"i-0229cede17f31dd4f","authorEmail":"[email protected]","name":"Habitat Tests on Linux / tests (3.9, 12.4) / linux-job","jobName":"tests (3.9, 12.4) / linux-job","conclusion":"failure","completed_at":"2025-02-07T17:52:50.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489977/job/36866378294","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["RuntimeError: Command docker exec -t 0c82c2f6823b15233bb14e7b850cb23888df084a7f4c3f18ac2025859981ce5d /exec failed with exit code 1"],"failure_lines":["RuntimeError: Command docker exec -t 0c82c2f6823b15233bb14e7b850cb23888df084a7f4c3f18ac2025859981ce5d /exec failed with exit code 1"],"failure_context":[],"time":"2025-02-07T17:43:01.000000000Z"}],"FLAKY":[],"BROKEN_TRUNK":[{"workflowId":13203489950,"workflowUniqueId":82165822,"id":36866384795,"runnerName":"i-0eccc450bed08a9fa","authorEmail":"[email protected]","name":"Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job","jobName":"tests-olddeps (3.8, 11.6) / linux-job","conclusion":"failure","completed_at":"2025-02-07T18:35:32.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489950/job/36866384795","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_transforms.py::TestHash::test_trans_parallel_env_check[str]"],"failure_lines":["FAILED test/test_transforms.py::TestHash::test_trans_parallel_env_check[str] - EOFError"],"failure_context":[],"time":"2025-02-07T17:43:09.000000000Z"},{"workflowId":13203489953,"workflowUniqueId":79519303,"id":36866377696,"runnerName":"i-02c4849cafaf77375","authorEmail":"[email protected]","name":"Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job","jobName":"unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job","conclusion":"failure","completed_at":"2025-02-07T18:19:19.000000000Z","html_url":"https://github.com/pytorch/rl/actions/runs/13203489953/job/36866377696","head_branch":"lf/gae-adv-fix","pr_number":2773,"head_sha":"5411dd933ab4c8ac5d1b9b7493c5b47e2ea0d065","head_sha_timestamp":"2025-02-07T16:02:48Z","failure_captures":["test/test_env.py::TestCustomEnvs::test_llm_hashing_env"],"failure_lines":["FAILED test/test_env.py::TestCustomEnvs::test_llm_hashing_env - AssertionError: The dtypes of the real and fake tensordict don't match for key observation. Got fake=torch.int64 and real=torch.int32."],"failure_context":[],"time":"2025-02-07T17:43:00.000000000Z"}],"UNSTABLE":[]}