[DO NOT CLOSE] Call for contributions

[vmoens]

This issue is a list of contributions requests from the community.

How to use this list

If you're willing to contribute to the library, have a look at the list below and don't hesitate to pick up a task.
If you need guidance, refer to @vmoens for more information!
Once you pick up a task, assign the related issue to yourself, to make sure that no other collaborator is working on the same task at the same time (or create an issue if there isn't one already).

If you want to add an item to this list, start by raising an issue and mention that you think it would be appropriate to put it in the "call for contributions" stack.

If it's your first contribution, warm up with an issue marked with https://github.com/github/docs/labels/Good%20first%20issue label (and claim the issue so no one else does!)

New algorithms

New algorithms can be coded either in a free form or using torchrl's trainer class.
In any case, we would ask to the user to use hydra for the configuration, and to limit the number of extra dependencies as much as can be.

• PILCO (no open issue yet)
• Image augmentation is all you need [Feature Request] Image Augmentation Is All You Need benchmark #32
• TQC algorithm ([Feature Request] Implement TQC for the example algorithms #1623)
• A3C [Feature Request] Add A3C #1755

## New environment libraries

• Safety gym compatibility
• Procgen
• Genesis

New modules and features

• Reward-to-go [Feature Request] Reward-to-go #16
• On-the-fly adaptation of alpha and beta in PRB [Feature Request] Support for alpha and beta parameters' schedule in torchrl.data.PrioritizedReplayBuffer #1575
• Raise exception when sampling from empty replay buffer ([Feature Request] Replay Buffer features #994)
• Add an option to "squash" the observation dictionary in register_gym when there is only one observation (ie, not return a dict but a simple tensor)
• Add a num_envs option in DMControlEnv to create a parallel env in just one call, e.g. DMControlEnv(name, task, num_envs=4) would run 4 parallel envs of the name, task dmc env.

Datasets

• MIMIC MIMIC Dataset for Batch RL [Feature Request] #1679
• SG2EGSet
• TDMPC2
• DexGraspNet A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
• Ego4d
• BridgeDataV2
• RoboVQA
• RL Unplugged
• d3rlpy

BugFixes

• Colabs don't render in the doc
• Raise an meaningful exception when one-hot specs are reshaped with a shape that doesn't match the last dim for all transforms that incur a change of shape [BUG] FlattenObservation transform with OneHotDiscreteTensorSpec #1904

Deprecation calls

• Softly deprecate NormalParamWrapper in favor of NormalParamExtractor
• Deprecate wrappers

  rl/torchrl/modules/tensordict_module/exploration.py

  Line 252 in 69d44f5

  class AdditiveGaussianWrapper(TensorDictModuleWrapper):

  and

  rl/torchrl/modules/tensordict_module/exploration.py

  Line 385 in 69d44f5

  class OrnsteinUhlenbeckProcessWrapper(TensorDictModuleWrapper):

  in favor of simple modules like

  rl/torchrl/modules/tensordict_module/exploration.py

  Line 31 in 69d44f5

  class EGreedyModule(TensorDictModuleBase):

Solved issues

• SMAC (Starcraft Multi-agent challenge) -> [WIP][Feature] Add support of SMAC environment #810
• PettingZoo
• A2C algorithm [Feature Request] A2C #17 -> solved as of [Doc, Test] Add A2C script test and doc #702
• TD3 algorithm [Feature Request] TD3 #18 -> [Algorithm] Td3 #684
• Decision transformers [Feature Request] Algorithm: Implement Decision Transformer #15
• DQN Atari (https://offline-rl.github.io/). A D4RL wrapper can already be found here, which can be a good source of inspiration. [Feature] Atari DQN dataset #1815
• Gen DGRL (WebShop and ProcGen) => Gen DGRL Dataset[Feature Request] #1678
• Roboset [Feature] Roboset datasets #1743
• V-D4RL (issue [Feature Request] V-D4RL datasets for offline RL. #1674) => [Feature] V-D4RL #1756
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models [Feature] Open-X Embodiement datasets #1751
• Minari [Feature] Minari datasets #1721

Thanks for contributing to TorchRL!