Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] break_when_all_done in rollout #2381

Merged
merged 13 commits into from
Aug 12, 2024
Merged

[Feature] break_when_all_done in rollout #2381

merged 13 commits into from
Aug 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Aug 8, 2024 •
edited
Loading

@vmoens
Update
ddb4d01 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 8, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2381

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Cancelled Job, 6 Unrelated Failures

As of commit 48f17a3 with merge base a6310ae (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Aug 7, 2024
Merged
Open
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 8, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 8, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.7430ms 58.8783ms 16.9842 Ops/s 17.2366 Ops/s $\color{#d91a1a}-1.46\%$
test_sync 44.5048ms 36.6403ms 27.2923 Ops/s 27.4434 Ops/s $\color{#d91a1a}-0.55\%$
test_async 61.9854ms 31.0664ms 32.1891 Ops/s 32.1646 Ops/s $\color{#35bf28}+0.08\%$
test_simple 0.4961s 0.4207s 2.3770 Ops/s 2.3969 Ops/s $\color{#d91a1a}-0.83\%$
test_transformed 0.6335s 0.5705s 1.7527 Ops/s 1.7463 Ops/s $\color{#35bf28}+0.37\%$
test_serial 1.3269s 1.2633s 0.7916 Ops/s 0.7904 Ops/s $\color{#35bf28}+0.16\%$
test_parallel 1.1004s 1.0918s 0.9159 Ops/s 0.8864 Ops/s $\color{#35bf28}+3.33\%$
test_step_mdp_speed[True-True-True-True-True] 0.2316ms 25.5698μs 39.1086 KOps/s 40.2593 KOps/s $\color{#d91a1a}-2.86\%$
test_step_mdp_speed[True-True-True-True-False] 37.5390μs 15.0004μs 66.6651 KOps/s 69.7404 KOps/s $\color{#d91a1a}-4.41\%$
test_step_mdp_speed[True-True-True-False-True] 67.3150μs 14.4945μs 68.9917 KOps/s 70.5347 KOps/s $\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-True-False-False] 33.1020μs 8.5950μs 116.3472 KOps/s 121.9724 KOps/s $\color{#d91a1a}-4.61\%$
test_step_mdp_speed[True-True-False-True-True] 88.0240μs 27.0275μs 36.9993 KOps/s 37.8327 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-True-False-True-False] 49.5120μs 16.3653μs 61.1050 KOps/s 63.1362 KOps/s $\color{#d91a1a}-3.22\%$
test_step_mdp_speed[True-True-False-False-True] 71.4210μs 15.9324μs 62.7652 KOps/s 63.8884 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[True-True-False-False-False] 51.5640μs 10.0109μs 99.8912 KOps/s 102.6344 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[True-False-True-True-True] 85.5470μs 28.5971μs 34.9686 KOps/s 35.4901 KOps/s $\color{#d91a1a}-1.47\%$
test_step_mdp_speed[True-False-True-True-False] 43.7520μs 18.0431μs 55.4229 KOps/s 57.0215 KOps/s $\color{#d91a1a}-2.80\%$
test_step_mdp_speed[True-False-True-False-True] 70.5410μs 15.9457μs 62.7129 KOps/s 63.8264 KOps/s $\color{#d91a1a}-1.74\%$
test_step_mdp_speed[True-False-True-False-False] 39.1930μs 10.1130μs 98.8822 KOps/s 101.5170 KOps/s $\color{#d91a1a}-2.60\%$
test_step_mdp_speed[True-False-False-True-True] 80.8710μs 30.0370μs 33.2922 KOps/s 34.0036 KOps/s $\color{#d91a1a}-2.09\%$
test_step_mdp_speed[True-False-False-True-False] 75.9780μs 19.7471μs 50.6403 KOps/s 52.7578 KOps/s $\color{#d91a1a}-4.01\%$
test_step_mdp_speed[True-False-False-False-True] 60.7330μs 17.6386μs 56.6940 KOps/s 58.4353 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[True-False-False-False-False] 0.1083ms 11.5385μs 86.6660 KOps/s 88.7276 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[False-True-True-True-True] 64.8300μs 28.9015μs 34.6002 KOps/s 35.4496 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[False-True-True-True-False] 81.0710μs 18.2196μs 54.8861 KOps/s 57.3380 KOps/s $\color{#d91a1a}-4.28\%$
test_step_mdp_speed[False-True-True-False-True] 74.1180μs 18.8343μs 53.0947 KOps/s 54.3612 KOps/s $\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-True-True-False-False] 67.5250μs 11.4298μs 87.4907 KOps/s 91.2308 KOps/s $\color{#d91a1a}-4.10\%$
test_step_mdp_speed[False-True-False-True-True] 96.0380μs 30.4106μs 32.8833 KOps/s 33.7572 KOps/s $\color{#d91a1a}-2.59\%$
test_step_mdp_speed[False-True-False-True-False] 65.6930μs 19.5706μs 51.0971 KOps/s 52.8565 KOps/s $\color{#d91a1a}-3.33\%$
test_step_mdp_speed[False-True-False-False-True] 62.3460μs 20.2281μs 49.4362 KOps/s 49.4957 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-True-False-False-False] 75.9610μs 12.8654μs 77.7282 KOps/s 80.3019 KOps/s $\color{#d91a1a}-3.21\%$
test_step_mdp_speed[False-False-True-True-True] 3.3183ms 32.2396μs 31.0177 KOps/s 31.7498 KOps/s $\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-False-True-True-False] 46.8670μs 21.1891μs 47.1940 KOps/s 48.4826 KOps/s $\color{#d91a1a}-2.66\%$
test_step_mdp_speed[False-False-True-False-True] 75.8900μs 20.0930μs 49.7686 KOps/s 49.9985 KOps/s $\color{#d91a1a}-0.46\%$
test_step_mdp_speed[False-False-True-False-False] 69.7400μs 13.0124μs 76.8499 KOps/s 79.8916 KOps/s $\color{#d91a1a}-3.81\%$
test_step_mdp_speed[False-False-False-True-True] 89.8170μs 32.7235μs 30.5591 KOps/s 30.7399 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-False-False-True-False] 68.9280μs 23.0895μs 43.3098 KOps/s 45.2456 KOps/s $\color{#d91a1a}-4.28\%$
test_step_mdp_speed[False-False-False-False-True] 62.0960μs 21.4787μs 46.5578 KOps/s 47.7602 KOps/s $\color{#d91a1a}-2.52\%$
test_step_mdp_speed[False-False-False-False-False] 50.5340μs 14.5848μs 68.5645 KOps/s 71.9659 KOps/s $\color{#d91a1a}-4.73\%$
test_values[generalized_advantage_estimate-True-True] 11.2409ms 9.8346ms 101.6820 Ops/s 108.0382 Ops/s $\textbf{\color{#d91a1a}-5.88\%}$
test_values[vec_generalized_advantage_estimate-True-True] 36.0686ms 33.7430ms 29.6357 Ops/s 27.5731 Ops/s $\textbf{\color{#35bf28}+7.48\%}$
test_values[td0_return_estimate-False-False] 0.2317ms 0.1779ms 5.6210 KOps/s 5.7371 KOps/s $\color{#d91a1a}-2.02\%$
test_values[td1_return_estimate-False-False] 24.7379ms 23.8358ms 41.9537 Ops/s 42.3915 Ops/s $\color{#d91a1a}-1.03\%$
test_values[vec_td1_return_estimate-False-False] 35.5574ms 33.7164ms 29.6592 Ops/s 27.6462 Ops/s $\textbf{\color{#35bf28}+7.28\%}$
test_values[td_lambda_return_estimate-True-False] 37.7346ms 34.5894ms 28.9106 Ops/s 28.9085 Ops/s $+0.01\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.3322ms 33.7341ms 29.6436 Ops/s 27.5937 Ops/s $\textbf{\color{#35bf28}+7.43\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 11.9790ms 8.4782ms 117.9498 Ops/s 122.5789 Ops/s $\color{#d91a1a}-3.78\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.1972ms 2.0006ms 499.8465 Ops/s 510.0611 Ops/s $\color{#d91a1a}-2.00\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5397ms 0.3561ms 2.8080 KOps/s 2.7150 KOps/s $\color{#35bf28}+3.43\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 49.4141ms 48.7736ms 20.5029 Ops/s 21.4368 Ops/s $\color{#d91a1a}-4.36\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0978ms 3.0450ms 328.4118 Ops/s 328.6559 Ops/s $\color{#d91a1a}-0.07\%$
test_dqn_speed 1.5451ms 1.2760ms 783.6944 Ops/s 779.6713 Ops/s $\color{#35bf28}+0.52\%$
test_ddpg_speed 3.5700ms 2.6834ms 372.6622 Ops/s 371.7123 Ops/s $\color{#35bf28}+0.26\%$
test_sac_speed 9.6670ms 7.8803ms 126.8980 Ops/s 126.4365 Ops/s $\color{#35bf28}+0.37\%$
test_redq_speed 14.3949ms 12.7003ms 78.7380 Ops/s 76.9786 Ops/s $\color{#35bf28}+2.29\%$
test_redq_deprec_speed 13.5513ms 12.7289ms 78.5614 Ops/s 78.3221 Ops/s $\color{#35bf28}+0.31\%$
test_td3_speed 8.1575ms 7.8197ms 127.8825 Ops/s 126.8210 Ops/s $\color{#35bf28}+0.84\%$
test_cql_speed 36.9829ms 35.5986ms 28.0910 Ops/s 25.2028 Ops/s $\textbf{\color{#35bf28}+11.46\%}$
test_a2c_speed 8.4073ms 7.3555ms 135.9523 Ops/s 137.8137 Ops/s $\color{#d91a1a}-1.35\%$
test_ppo_speed 8.6721ms 7.6947ms 129.9600 Ops/s 131.6075 Ops/s $\color{#d91a1a}-1.25\%$
test_reinforce_speed 8.5428ms 6.6118ms 151.2446 Ops/s 155.3638 Ops/s $\color{#d91a1a}-2.65\%$
test_iql_speed 33.6898ms 31.9250ms 31.3235 Ops/s 30.1915 Ops/s $\color{#35bf28}+3.75\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.0541ms 4.8218ms 207.3927 Ops/s 204.6452 Ops/s $\color{#35bf28}+1.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9459ms 0.4777ms 2.0932 KOps/s 2.1022 KOps/s $\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6603ms 0.4537ms 2.2040 KOps/s 2.2116 KOps/s $\color{#d91a1a}-0.34\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.2613ms 4.8292ms 207.0754 Ops/s 206.8421 Ops/s $\color{#35bf28}+0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0494ms 0.4775ms 2.0941 KOps/s 2.1129 KOps/s $\color{#d91a1a}-0.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6122ms 0.4465ms 2.2394 KOps/s 2.2322 KOps/s $\color{#35bf28}+0.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2678ms 1.6853ms 593.3501 Ops/s 589.4939 Ops/s $\color{#35bf28}+0.65\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2240ms 1.6016ms 624.3785 Ops/s 624.3627 Ops/s $+0.00\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.3658ms 4.9667ms 201.3420 Ops/s 198.8694 Ops/s $\color{#35bf28}+1.24\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1527s 0.7455ms 1.3413 KOps/s 1.6338 KOps/s $\textbf{\color{#d91a1a}-17.90\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7557ms 0.5862ms 1.7059 KOps/s 1.7046 KOps/s $\color{#35bf28}+0.07\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6421ms 4.8410ms 206.5685 Ops/s 204.0453 Ops/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6323ms 0.4798ms 2.0842 KOps/s 2.0894 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.2712ms 0.4596ms 2.1759 KOps/s 2.1846 KOps/s $\color{#d91a1a}-0.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.3313ms 4.7829ms 209.0774 Ops/s 209.0629 Ops/s $+0.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3129ms 0.4757ms 2.1023 KOps/s 2.1164 KOps/s $\color{#d91a1a}-0.67\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6200ms 0.4496ms 2.2240 KOps/s 2.2209 KOps/s $\color{#35bf28}+0.14\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4220ms 4.9967ms 200.1326 Ops/s 202.0326 Ops/s $\color{#d91a1a}-0.94\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6773ms 0.6209ms 1.6105 KOps/s 1.6274 KOps/s $\color{#d91a1a}-1.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7679ms 0.5904ms 1.6938 KOps/s 1.6940 KOps/s $-0.01\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1452s 6.5192ms 153.3937 Ops/s 157.9248 Ops/s $\color{#d91a1a}-2.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1314s 15.3609ms 65.1002 Ops/s 76.7932 Ops/s $\textbf{\color{#d91a1a}-15.23\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5443ms 1.0461ms 955.9192 Ops/s 866.9369 Ops/s $\textbf{\color{#35bf28}+10.26\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1112s 5.8893ms 169.8003 Ops/s 171.5667 Ops/s $\color{#d91a1a}-1.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.0546ms 12.9722ms 77.0878 Ops/s 77.2481 Ops/s $\color{#d91a1a}-0.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.7361ms 1.2024ms 831.6622 Ops/s 909.3754 Ops/s $\textbf{\color{#d91a1a}-8.55\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1279s 6.3043ms 158.6209 Ops/s 119.9888 Ops/s $\textbf{\color{#35bf28}+32.20\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.6273ms 13.2095ms 75.7029 Ops/s 75.9205 Ops/s $\color{#d91a1a}-0.29\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.9863ms 1.2800ms 781.2370 Ops/s 782.5008 Ops/s $\color{#d91a1a}-0.16\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 8, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1069s 0.1057s 9.4644 Ops/s 9.4803 Ops/s $\color{#d91a1a}-0.17\%$
test_sync 95.6763ms 93.7003ms 10.6723 Ops/s 11.2521 Ops/s $\textbf{\color{#d91a1a}-5.15\%}$
test_async 0.1762s 87.0021ms 11.4940 Ops/s 11.9137 Ops/s $\color{#d91a1a}-3.52\%$
test_single_pixels 0.1158s 0.1156s 8.6529 Ops/s 8.7476 Ops/s $\color{#d91a1a}-1.08\%$
test_sync_pixels 81.3247ms 77.1635ms 12.9595 Ops/s 13.1847 Ops/s $\color{#d91a1a}-1.71\%$
test_async_pixels 0.1421s 70.8981ms 14.1048 Ops/s 14.3093 Ops/s $\color{#d91a1a}-1.43\%$
test_simple 0.7520s 0.7499s 1.3335 Ops/s 1.3181 Ops/s $\color{#35bf28}+1.17\%$
test_transformed 1.0784s 1.0059s 0.9941 Ops/s 1.0306 Ops/s $\color{#d91a1a}-3.54\%$
test_serial 2.2173s 2.1450s 0.4662 Ops/s 0.4778 Ops/s $\color{#d91a1a}-2.43\%$
test_parallel 1.9467s 1.8798s 0.5320 Ops/s 0.5348 Ops/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-True-True-True] 99.2110μs 38.2891μs 26.1171 KOps/s 26.8536 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-True-True-True-False] 0.2126ms 21.3580μs 46.8208 KOps/s 48.2575 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[True-True-True-False-True] 37.4100μs 21.4988μs 46.5142 KOps/s 48.6297 KOps/s $\color{#d91a1a}-4.35\%$
test_step_mdp_speed[True-True-True-False-False] 37.1420μs 12.1198μs 82.5094 KOps/s 84.3516 KOps/s $\color{#d91a1a}-2.18\%$
test_step_mdp_speed[True-True-False-True-True] 0.2293ms 40.4613μs 24.7149 KOps/s 25.6623 KOps/s $\color{#d91a1a}-3.69\%$
test_step_mdp_speed[True-True-False-True-False] 0.2105ms 23.5132μs 42.5293 KOps/s 43.9285 KOps/s $\color{#d91a1a}-3.19\%$
test_step_mdp_speed[True-True-False-False-True] 0.2113ms 23.2056μs 43.0931 KOps/s 43.4690 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[True-True-False-False-False] 47.4810μs 14.4208μs 69.3444 KOps/s 72.0189 KOps/s $\color{#d91a1a}-3.71\%$
test_step_mdp_speed[True-False-True-True-True] 0.2355ms 42.8983μs 23.3109 KOps/s 24.5554 KOps/s $\textbf{\color{#d91a1a}-5.07\%}$
test_step_mdp_speed[True-False-True-True-False] 0.2137ms 25.8811μs 38.6383 KOps/s 40.1243 KOps/s $\color{#d91a1a}-3.70\%$
test_step_mdp_speed[True-False-True-False-True] 41.7310μs 23.4101μs 42.7165 KOps/s 44.5409 KOps/s $\color{#d91a1a}-4.10\%$
test_step_mdp_speed[True-False-True-False-False] 31.0400μs 14.3689μs 69.5948 KOps/s 72.0701 KOps/s $\color{#d91a1a}-3.43\%$
test_step_mdp_speed[True-False-False-True-True] 0.2351ms 44.9301μs 22.2568 KOps/s 23.3980 KOps/s $\color{#d91a1a}-4.88\%$
test_step_mdp_speed[True-False-False-True-False] 0.2155ms 28.1568μs 35.5155 KOps/s 36.3233 KOps/s $\color{#d91a1a}-2.22\%$
test_step_mdp_speed[True-False-False-False-True] 0.2109ms 25.6968μs 38.9154 KOps/s 40.5931 KOps/s $\color{#d91a1a}-4.13\%$
test_step_mdp_speed[True-False-False-False-False] 33.0310μs 16.6884μs 59.9219 KOps/s 62.0137 KOps/s $\color{#d91a1a}-3.37\%$
test_step_mdp_speed[False-True-True-True-True] 0.2355ms 42.9036μs 23.3081 KOps/s 24.2837 KOps/s $\color{#d91a1a}-4.02\%$
test_step_mdp_speed[False-True-True-True-False] 0.2104ms 25.8979μs 38.6132 KOps/s 40.0447 KOps/s $\color{#d91a1a}-3.57\%$
test_step_mdp_speed[False-True-True-False-True] 0.2178ms 28.5381μs 35.0408 KOps/s 37.3852 KOps/s $\textbf{\color{#d91a1a}-6.27\%}$
test_step_mdp_speed[False-True-True-False-False] 43.2200μs 16.2618μs 61.4938 KOps/s 63.9256 KOps/s $\color{#d91a1a}-3.80\%$
test_step_mdp_speed[False-True-False-True-True] 0.2346ms 44.9350μs 22.2544 KOps/s 23.0631 KOps/s $\color{#d91a1a}-3.51\%$
test_step_mdp_speed[False-True-False-True-False] 0.2108ms 27.9946μs 35.7211 KOps/s 37.0176 KOps/s $\color{#d91a1a}-3.50\%$
test_step_mdp_speed[False-True-False-False-True] 48.1500μs 30.8874μs 32.3757 KOps/s 34.4553 KOps/s $\textbf{\color{#d91a1a}-6.04\%}$
test_step_mdp_speed[False-True-False-False-False] 39.1200μs 18.4415μs 54.2255 KOps/s 55.8250 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-False-True-True-True] 4.2543ms 47.9122μs 20.8715 KOps/s 21.6348 KOps/s $\color{#d91a1a}-3.53\%$
test_step_mdp_speed[False-False-True-True-False] 0.2257ms 30.6708μs 32.6043 KOps/s 33.7049 KOps/s $\color{#d91a1a}-3.27\%$
test_step_mdp_speed[False-False-True-False-True] 0.2237ms 30.5804μs 32.7007 KOps/s 34.7188 KOps/s $\textbf{\color{#d91a1a}-5.81\%}$
test_step_mdp_speed[False-False-True-False-False] 35.6200μs 18.3487μs 54.4996 KOps/s 55.9200 KOps/s $\color{#d91a1a}-2.54\%$
test_step_mdp_speed[False-False-False-True-True] 0.2361ms 49.2139μs 20.3195 KOps/s 21.3695 KOps/s $\color{#d91a1a}-4.91\%$
test_step_mdp_speed[False-False-False-True-False] 0.2183ms 32.9205μs 30.3762 KOps/s 31.7149 KOps/s $\color{#d91a1a}-4.22\%$
test_step_mdp_speed[False-False-False-False-True] 0.2199ms 32.4539μs 30.8129 KOps/s 32.2390 KOps/s $\color{#d91a1a}-4.42\%$
test_step_mdp_speed[False-False-False-False-False] 37.1100μs 20.5341μs 48.6994 KOps/s 50.2937 KOps/s $\color{#d91a1a}-3.17\%$
test_values[generalized_advantage_estimate-True-True] 24.1367ms 23.6951ms 42.2028 Ops/s 42.7432 Ops/s $\color{#d91a1a}-1.26\%$
test_values[vec_generalized_advantage_estimate-True-True] 91.8186ms 2.7206ms 367.5666 Ops/s 378.8827 Ops/s $\color{#d91a1a}-2.99\%$
test_values[td0_return_estimate-False-False] 98.0110μs 65.7860μs 15.2008 KOps/s 15.5532 KOps/s $\color{#d91a1a}-2.27\%$
test_values[td1_return_estimate-False-False] 54.4877ms 52.9700ms 18.8786 Ops/s 18.8018 Ops/s $\color{#35bf28}+0.41\%$
test_values[vec_td1_return_estimate-False-False] 1.3908ms 1.0666ms 937.5785 Ops/s 936.7189 Ops/s $\color{#35bf28}+0.09\%$
test_values[td_lambda_return_estimate-True-False] 87.4841ms 84.4232ms 11.8451 Ops/s 11.7555 Ops/s $\color{#35bf28}+0.76\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.2893ms 1.0621ms 941.5574 Ops/s 939.4669 Ops/s $\color{#35bf28}+0.22\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.8569ms 23.6357ms 42.3089 Ops/s 42.5682 Ops/s $\color{#d91a1a}-0.61\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9425ms 0.7051ms 1.4183 KOps/s 1.3707 KOps/s $\color{#35bf28}+3.47\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8080ms 0.6507ms 1.5368 KOps/s 1.5417 KOps/s $\color{#d91a1a}-0.32\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6744ms 1.4613ms 684.3094 Ops/s 688.7973 Ops/s $\color{#d91a1a}-0.65\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8348ms 0.6656ms 1.5025 KOps/s 1.5104 KOps/s $\color{#d91a1a}-0.53\%$
test_dqn_speed 7.2574ms 1.4157ms 706.3444 Ops/s 717.5732 Ops/s $\color{#d91a1a}-1.56\%$
test_ddpg_speed 3.3377ms 2.7881ms 358.6628 Ops/s 361.5277 Ops/s $\color{#d91a1a}-0.79\%$
test_sac_speed 0.1024s 8.7706ms 114.0170 Ops/s 124.4754 Ops/s $\textbf{\color{#d91a1a}-8.40\%}$
test_redq_speed 12.3885ms 10.3679ms 96.4516 Ops/s 97.4370 Ops/s $\color{#d91a1a}-1.01\%$
test_redq_deprec_speed 11.6291ms 11.1494ms 89.6909 Ops/s 92.2209 Ops/s $\color{#d91a1a}-2.74\%$
test_td3_speed 8.0905ms 7.9108ms 126.4100 Ops/s 125.8326 Ops/s $\color{#35bf28}+0.46\%$
test_cql_speed 26.6470ms 25.4021ms 39.3668 Ops/s 39.0521 Ops/s $\color{#35bf28}+0.81\%$
test_a2c_speed 7.3724ms 5.6363ms 177.4212 Ops/s 181.3295 Ops/s $\color{#d91a1a}-2.16\%$
test_ppo_speed 6.4899ms 5.9754ms 167.3530 Ops/s 170.5706 Ops/s $\color{#d91a1a}-1.89\%$
test_reinforce_speed 4.8598ms 4.5954ms 217.6091 Ops/s 224.8325 Ops/s $\color{#d91a1a}-3.21\%$
test_iql_speed 20.6710ms 19.9530ms 50.1178 Ops/s 51.2163 Ops/s $\color{#d91a1a}-2.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8423ms 6.6676ms 149.9795 Ops/s 152.8117 Ops/s $\color{#d91a1a}-1.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1594ms 0.5355ms 1.8676 KOps/s 1.8768 KOps/s $\color{#d91a1a}-0.49\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7443ms 0.5197ms 1.9241 KOps/s 1.9689 KOps/s $\color{#d91a1a}-2.28\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1038ms 6.5678ms 152.2591 Ops/s 154.4318 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8817ms 0.5225ms 1.9138 KOps/s 1.9334 KOps/s $\color{#d91a1a}-1.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7265ms 0.5021ms 1.9918 KOps/s 1.9904 KOps/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1764ms 2.0018ms 499.5538 Ops/s 503.8754 Ops/s $\color{#d91a1a}-0.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 5.5900ms 1.9223ms 520.1968 Ops/s 523.6544 Ops/s $\color{#d91a1a}-0.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0919ms 6.8036ms 146.9818 Ops/s 149.2865 Ops/s $\color{#d91a1a}-1.54\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1261s 0.7903ms 1.2653 KOps/s 1.4287 KOps/s $\textbf{\color{#d91a1a}-11.44\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9022ms 0.6570ms 1.5220 KOps/s 1.5138 KOps/s $\color{#35bf28}+0.54\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.8212ms 6.6535ms 150.2965 Ops/s 153.1034 Ops/s $\color{#d91a1a}-1.83\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.5659ms 0.5252ms 1.9039 KOps/s 1.8869 KOps/s $\color{#35bf28}+0.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7243ms 0.5061ms 1.9759 KOps/s 1.9507 KOps/s $\color{#35bf28}+1.29\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.9892ms 6.5943ms 151.6472 Ops/s 154.6346 Ops/s $\color{#d91a1a}-1.93\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6955ms 0.5174ms 1.9329 KOps/s 1.9221 KOps/s $\color{#35bf28}+0.56\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7063ms 0.5005ms 1.9980 KOps/s 1.9664 KOps/s $\color{#35bf28}+1.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.0139ms 6.8070ms 146.9067 Ops/s 148.8566 Ops/s $\color{#d91a1a}-1.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1988ms 0.6818ms 1.4667 KOps/s 1.4887 KOps/s $\color{#d91a1a}-1.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.5687ms 0.6999ms 1.4289 KOps/s 1.5133 KOps/s $\textbf{\color{#d91a1a}-5.58\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1309s 7.6219ms 131.2005 Ops/s 126.9305 Ops/s $\color{#35bf28}+3.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 18.6884ms 16.0425ms 62.3344 Ops/s 63.5617 Ops/s $\color{#d91a1a}-1.93\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.5484ms 1.3100ms 763.3365 Ops/s 780.7574 Ops/s $\color{#d91a1a}-2.23\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1255s 7.5512ms 132.4301 Ops/s 99.4057 Ops/s $\textbf{\color{#35bf28}+33.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1388s 18.3631ms 54.4570 Ops/s 63.9700 Ops/s $\textbf{\color{#d91a1a}-14.87\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.4813ms 1.4068ms 710.8478 Ops/s 725.1645 Ops/s $\color{#d91a1a}-1.97\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1263s 7.7679ms 128.7343 Ops/s 128.6633 Ops/s $\color{#35bf28}+0.06\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.5486ms 16.0292ms 62.3862 Ops/s 63.5771 Ops/s $\color{#d91a1a}-1.87\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.6713ms 1.5734ms 635.5462 Ops/s 708.2484 Ops/s $\textbf{\color{#d91a1a}-10.27\%}$

@vmoens vmoens added the enhancement New feature or request label Aug 8, 2024
@vmoens vmoens linked an issue Aug 8, 2024 that may be closed by this pull request
[Feature Request] break_when_all_done in env.rollout() #2355
Open
@vmoens
Update
766cfbc 
[ghstack-poisoned]
@vmoens
Update
732d13b 
[ghstack-poisoned]
@vmoens
Update
74a72ba 
[ghstack-poisoned]
@vmoens
Update
b439608 
[ghstack-poisoned]
@vmoens
Update
15e9a9a 
[ghstack-poisoned]
@vmoens
Update
3fa2184 
[ghstack-poisoned]
@vmoens
Update
8075f6f 
[ghstack-poisoned]
@vmoens
Update
d2b3847 
[ghstack-poisoned]
@vmoens
Update
f1e3c75 
[ghstack-poisoned]
@vmoens
Update
87eb60a 
[ghstack-poisoned]
@vmoens
Update
1a5a246 
[ghstack-poisoned]
@vmoens
Update
48f17a3 
[ghstack-poisoned]
@vmoens vmoens merged commit 48f17a3 into gh/vmoens/11/base Aug 12, 2024
57 of 68 checks passed
vmoens added a commit that referenced this pull request Aug 12, 2024
@vmoens
[Feature] break_when_all_done in rollout
bb0ddb5 
ghstack-source-id: 103fd4f3ba8eb8d6e916b6921ab14f95c920f3b5
Pull Request resolved: #2381
@vmoens vmoens deleted the gh/vmoens/11/head branch August 12, 2024 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Feature Request] break_when_all_done in env.rollout()
2 participants
@vmoens @facebook-github-bot