Sampling multivariate normal with low-rank covariance
$begingroup$
In the context of sparse Gaussian Processes we get an approximation for an $N$ by $N$ positive definite covariance matrix that is of rank $M$:
$Sigma = Lambda + VV^T$
where $Lambda$ is diagonal and $V$ is an $N$ by $M$ matrix.
This is very useful for finding the inverse (via the Woodbury formula), but what I need is to sample from the resulting multivariate normal. This usually requires a Cholesky or an eigendecomposition, but I cannot find a way to exploit the low-rank structure of $Sigma$ in those cases.
So, my question is whether there is a way to compute Cholesky, Eigen/SVD, or to sample from the normal distribution without explicitly forming the $N$ by $N$ matrix?
linear-algebra matrices symmetric-matrices
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
$endgroup$
add a comment |
$begingroup$
In the context of sparse Gaussian Processes we get an approximation for an $N$ by $N$ positive definite covariance matrix that is of rank $M$:
$Sigma = Lambda + VV^T$
where $Lambda$ is diagonal and $V$ is an $N$ by $M$ matrix.
This is very useful for finding the inverse (via the Woodbury formula), but what I need is to sample from the resulting multivariate normal. This usually requires a Cholesky or an eigendecomposition, but I cannot find a way to exploit the low-rank structure of $Sigma$ in those cases.
So, my question is whether there is a way to compute Cholesky, Eigen/SVD, or to sample from the normal distribution without explicitly forming the $N$ by $N$ matrix?
linear-algebra matrices symmetric-matrices
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
$endgroup$
add a comment |
$begingroup$
In the context of sparse Gaussian Processes we get an approximation for an $N$ by $N$ positive definite covariance matrix that is of rank $M$:
$Sigma = Lambda + VV^T$
where $Lambda$ is diagonal and $V$ is an $N$ by $M$ matrix.
This is very useful for finding the inverse (via the Woodbury formula), but what I need is to sample from the resulting multivariate normal. This usually requires a Cholesky or an eigendecomposition, but I cannot find a way to exploit the low-rank structure of $Sigma$ in those cases.
So, my question is whether there is a way to compute Cholesky, Eigen/SVD, or to sample from the normal distribution without explicitly forming the $N$ by $N$ matrix?
linear-algebra matrices symmetric-matrices
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
$endgroup$
In the context of sparse Gaussian Processes we get an approximation for an $N$ by $N$ positive definite covariance matrix that is of rank $M$:
$Sigma = Lambda + VV^T$
where $Lambda$ is diagonal and $V$ is an $N$ by $M$ matrix.
This is very useful for finding the inverse (via the Woodbury formula), but what I need is to sample from the resulting multivariate normal. This usually requires a Cholesky or an eigendecomposition, but I cannot find a way to exploit the low-rank structure of $Sigma$ in those cases.
So, my question is whether there is a way to compute Cholesky, Eigen/SVD, or to sample from the normal distribution without explicitly forming the $N$ by $N$ matrix?
linear-algebra matrices symmetric-matrices
linear-algebra matrices symmetric-matrices
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
asked Jul 12 '18 at 10:30
Sergiy ProtsivSergiy Protsiv
32
32
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
No need to make Cholesky decomposition to sample from a multivariate Gaussian with covariance $Sigma = Lambda + V^top V$.
Let the shapes of $Lambda = text{diag}(lambda)$ be (N, N) and $V$ be (N, K).
First, we need samples from i.i.d. zero-mean one-variance normal with size (N, ) and (K, ). Let these samples be $epsilon_lambda$ and $epsilon_V$.
The sample we want can be computed from
$Lambda^{1/2} epsilon_lambda + V epsilon_V$.
The variance of the first term is $Lambda$ and that for the second is $V^T V$.
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
$endgroup$
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2848517%2fsampling-multivariate-normal-with-low-rank-covariance%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
No need to make Cholesky decomposition to sample from a multivariate Gaussian with covariance $Sigma = Lambda + V^top V$.
Let the shapes of $Lambda = text{diag}(lambda)$ be (N, N) and $V$ be (N, K).
First, we need samples from i.i.d. zero-mean one-variance normal with size (N, ) and (K, ). Let these samples be $epsilon_lambda$ and $epsilon_V$.
The sample we want can be computed from
$Lambda^{1/2} epsilon_lambda + V epsilon_V$.
The variance of the first term is $Lambda$ and that for the second is $V^T V$.
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
$endgroup$
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
add a comment |
$begingroup$
No need to make Cholesky decomposition to sample from a multivariate Gaussian with covariance $Sigma = Lambda + V^top V$.
Let the shapes of $Lambda = text{diag}(lambda)$ be (N, N) and $V$ be (N, K).
First, we need samples from i.i.d. zero-mean one-variance normal with size (N, ) and (K, ). Let these samples be $epsilon_lambda$ and $epsilon_V$.
The sample we want can be computed from
$Lambda^{1/2} epsilon_lambda + V epsilon_V$.
The variance of the first term is $Lambda$ and that for the second is $V^T V$.
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
$endgroup$
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
add a comment |
$begingroup$
No need to make Cholesky decomposition to sample from a multivariate Gaussian with covariance $Sigma = Lambda + V^top V$.
Let the shapes of $Lambda = text{diag}(lambda)$ be (N, N) and $V$ be (N, K).
First, we need samples from i.i.d. zero-mean one-variance normal with size (N, ) and (K, ). Let these samples be $epsilon_lambda$ and $epsilon_V$.
The sample we want can be computed from
$Lambda^{1/2} epsilon_lambda + V epsilon_V$.
The variance of the first term is $Lambda$ and that for the second is $V^T V$.
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
$endgroup$
No need to make Cholesky decomposition to sample from a multivariate Gaussian with covariance $Sigma = Lambda + V^top V$.
Let the shapes of $Lambda = text{diag}(lambda)$ be (N, N) and $V$ be (N, K).
First, we need samples from i.i.d. zero-mean one-variance normal with size (N, ) and (K, ). Let these samples be $epsilon_lambda$ and $epsilon_V$.
The sample we want can be computed from
$Lambda^{1/2} epsilon_lambda + V epsilon_V$.
The variance of the first term is $Lambda$ and that for the second is $V^T V$.
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
answered Dec 8 '18 at 17:48
Keisuke FUJIIKeisuke FUJII
1261
1261
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
add a comment |
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
$begingroup$
Thanks a lot! Sometimes I forget about all of the nice properties of Gaussians.
$endgroup$
– Sergiy Protsiv
Dec 9 '18 at 19:48
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2848517%2fsampling-multivariate-normal-with-low-rank-covariance%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
