Issue/873 embedded laplace #874
Conversation
| The Laplace approximation is especially useful if $p(\theta)$ is | ||
| multivariate normal and $p(y \mid \phi, \theta)$ is | ||
| log-concave. Stan's embedded Laplace approximation is restricted to | ||
| have multivariate normal prior $p(\theta)$ and ... likelihood |
There was a problem hiding this comment.
add here the restrictions for the likelihood
There was a problem hiding this comment.
There are two kinds of restrictions:
- what the user can do without breaking Stan, i.e. the operations in the likelihood need to support higher-order autodiff.
- what the user should do to insure the approximation is reliable.
I'll assume you have the first in mind.
There was a problem hiding this comment.
Yes, I was thinking the first as a restriction. For the second one we can say which kind of likelihood are more likely to work, ie, log concave and maybe near log concave
|
I made some edits to use the statistical terms correctly. In the end of first section, it would be good to tell the constraints on the likelihood function and I left there three dots. |
|
@charlesm93 I started to fill in some of the boilerplate we have in our functions reference. Those comments and things are actually useful for building the index page |
WardBrian
force-pushed
the
issue/873-embeddedLaplace
branch
from
4db967a to
7a07abd
Compare
|
@WardBrian In the doc, what are the |
|
The unnormalized versions, which correspond to propto=true in the C++. For these functions they may be equivalent, but for technical reasons they still need to exist. If they don’t do anything we could remove the documentation, but it would be less consistent with others then |
| <!-- real; laplace_marginal; (function ll_function, tuple(...), vector theta0, function K_function, tuple(...)); --> | ||
| \index{{\tt \bfseries laplace\_marginal }!{\tt (function ll\_function, tuple(...), vector theta0, function K\_function, tuple(...)): real}|hyperpage} | ||
|
|
||
| `real` **`laplace_marginal`**`(function ll_function, tuple(...), vector theta0, function K_function, tuple(...))`<br>\newline |
There was a problem hiding this comment.
Can we call theta0 theta_init? I just think it sounds more clear
| * `hessian_block_size`: the size of the blocks, assuming the Hessian | ||
| $\partial \log p(y \mid \theta, phi) \ \partial \theta$ is block-diagonal. | ||
| The structure of the Hessian is determined by the dependence structure of $y$ | ||
| on $\theta$. By default, the Hessian is treated as diagonal |
There was a problem hiding this comment.
We should note where that if Hessian block size is not 1 or N then theta needs to be divisible by the Hessian block size
| * `solver`: choice of Newton solver. The optimizer used to compute the | ||
| Laplace approximation does one of three matrix decompositions to compute a | ||
| Newton step. The problem determines which decomposition is numerical stable. | ||
| By default (`solver=1`), the solver makes a Cholesky decomposition of the |
| ``` | ||
| matrix K_function(...) | ||
| ``` | ||
| There is no type restrictions for the variadic arguments. The variables $\phi$ |
There was a problem hiding this comment.
I think moreso from the section it's not clear how this is related to the k function
| The only restriction is that this function returns a positive-definite matrix | ||
| with size $n \times n$ where $n$ is the size of $\theta$. The signature is: | ||
| ``` | ||
| matrix K_function(...) |
There was a problem hiding this comment.
Can we call this covariance_function?
|
does the pdf build? I had to go through the file and change the |
| In the above procedure, neither the marginal posterior nor the conditional posterior | ||
| are typically available in closed form and so they must be approximated. | ||
| The marginal posterior can be written as $p(\phi \mid y) \propto p(y \mid \phi) p(\phi)$, | ||
| where $p(y \mid \phi) = \int p(y \mid \phi, \theta) p(\theta) \text{d}\theta$ $ |
…ev/docs into issue/873-embeddedLaplace
…arginal likelihood.
Submission Checklist
<<{ since VERSION }>>Summary
Documentation for suite of functions for the embedded Laplace approximation. Starting a PR to allow easy file comparison and will fill in the details soon.
Copyright and Licensing
Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company): (Figuring this out)
By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses: