Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resampling options for multi-Pathfinder #3242

Closed
2 tasks
avehtari opened this issue Nov 16, 2023 · 5 comments
Closed
2 tasks

Add resampling options for multi-Pathfinder #3242

avehtari opened this issue Nov 16, 2023 · 5 comments

Comments

@avehtari
Copy link
Collaborator

  1. Currently, multi-Pathfinder always resamples. When the approximation is far from the true target and especially in case of high dimensional posteriors, one weight can dominate and resampling returns only copies of a single draw. Furthermore, in edge cases, Pareto-smoothing or computation of weights can fail and produce completely unexpected results. To allow 1) further diagnostics in such problematic cases and 2) have more unique draws to initialize MCMC to improve MCMC convergence diagnostics
  • add boolean option resample with default value TRUE, and value FALSE disabling the resampling
  1. For cases where we don't care about doing further diagnostics, but want a minimum number of unique draws, allow resampling without replacement (and then set num_draws to the number of unique draws needed, e.g. for initializing MCMC)
  • add boolean option with_replacement with default value TRUE, and value FALSE using without replacement resampling
@WardBrian
Copy link
Member

What is returned when resample is false?

@avehtari
Copy link
Collaborator Author

When resample is false, return the draws from the normal approximations (skip lines 173-186 and index with consecutive numbers on line 188 of multi.hpp)

@WardBrian
Copy link
Member

I still don't think I understand how this is different from setting save_single_paths to true and using those draws

@avehtari
Copy link
Collaborator Author

  • I wasn't aware of save_single_paths option (after looking the doc, I think it's not well documented (one part gives an impression of that it saves the paths and not the draws), and also it's not currently working in CmdStanR (Pathfinder doc / behavior issues cmdstanr#878 (comment))
  • save_single_paths saves also the paths, which I'm not interested in, and to load the draws from the individual csv's would require additional code from the user or method in CmdStanR, so it would not be as clean solution

@WardBrian
Copy link
Member

#3249

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants