Implement a minimizer for INLA #513
Conversation
|
Currently there's a few outstanding TODOs. These are just issues getting quality-of-life features to work with pytensor - the actual algorithm itself works fine. Please find the TODOs listed as comments in the code, and use the code in |
pymc_extras/inference/laplace.py
Outdated
| use_jac: bool = True, | ||
| use_hess: bool = False, # TODO Tbh we can probably just remove this arg and pass True to the minimizer all the time, but if this is the case, it will throw a warning when the hessian doesn't need to be computed for a particular optimisation routine. | ||
| optimizer_kwargs: dict | None = None, | ||
| ) -> list[TensorLike]: |
There was a problem hiding this comment.
Signature is wrong, this function returns numpy arrays.
But why are you compiling a pytensor function and evaluating it? This seems to just be doing find_map? I imagine you wanted a symbolic mode not the numerical (evaluated) one?
There was a problem hiding this comment.
I felt that for the purposes of INLA, we only ever needed numerical values of mode and hess, but in truth, simply returning the compiled function probably makes this more versatile as well as eliminating a need for x0 and args (although these will likely need to be obtained somewhere later). I'll refactor it to return the compiled function.
There was a problem hiding this comment.
I've refactored it to only return the compiled function.
pymc_extras/inference/laplace.py
Outdated
| model: pm.Model | None = None, | ||
| method: minimize_method = "BFGS", | ||
| use_jac: bool = True, | ||
| use_hess: bool = False, # TODO Tbh we can probably just remove this arg and pass True to the minimizer all the time, but if this is the case, it will throw a warning when the hessian doesn't need to be computed for a particular optimisation routine. |
There was a problem hiding this comment.
I'm not really sure why these are options here. Presumably, the minimization method itself knows what it needs and it's redundant to specify use_jac or use_hess here at all.
| sigma_mu = rng.random() | ||
|
|
||
| coords = {"city": ["A", "B", "C"], "obs_idx": np.arange(n)} | ||
| with pm.Model(coords=coords) as model: |
There was a problem hiding this comment.
I would try to make this test in pytensor directly if possible.
|
@ricardoV94 @jessegrabowski The unittests currently seem to be failing because the current release of pytensor doesn't have optimize in it yet. Would it be possible to make a point release to so we can merge this? |
Addresses #342.
This PR should add:
find_modeContingent on pymc-devs/pytensor#1182, as it uses
pytensor.tensor.optimize.minimizeto find the mode (and hessian at that point).