Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is possible to filter experience from the episode whose length is longer than a specified value to add into replay_buffer? #1999

Closed
4 tasks done
CornfileChase opened this issue Sep 3, 2024 · 1 comment
Labels
question Further information is requested

Comments

@CornfileChase
Copy link

❓ Question

In the self-build custom environment, I suppose that some state-action pair that from very short episode is not helpful to the training process, thus I want to filter the experience from these episodes and do not push into the replay_buffer, can anyone tell me how to implement this function? thanks!

Checklist

@CornfileChase CornfileChase added the question Further information is requested label Sep 3, 2024
@araffin araffin added Maintainers on vacation Maintainers are on vacation so they can recharge their batteries, we will be back soon ;) and removed Maintainers on vacation Maintainers are on vacation so they can recharge their batteries, we will be back soon ;) labels Sep 3, 2024
@araffin
Copy link
Member

araffin commented Sep 10, 2024

I suppose that some state-action pair that from very short episode is not helpful to the training process

why not?
failures are important to learn what not to do.

I want to filter the experience from these episodes and do not push into the replay_buffer,

If you want to do that, you would need to fork SB3 or subclass SB3 algorithms.

@araffin araffin closed this as completed Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants