Computing the log-likelihood term
I´m currently reading the paper "The Little Engine that Could
Regularization by Denoising (RED)" by Yaniv Romano and Micheal Elad. In the beginning they come up with solving the Maximum aposteriori Probability (MAP) for an given (measured) image $y$ and a unknown image $x$. So one tries to solve ${argmax}_x P(xvert y)$ which is equivalent to solving ${argmin}_x -log(P(yvert x)) -log(P(x))$. The term $l(y,x)=-log(P(yvert x))$ is also know as the log-likelihood term. Then there is written that $l(y,x)=frac{1}{2sigma^2}vertvert Hx-yvert{vert}_2^2$ if $y=Hx+e$, where $e$ is a gaussian noise with variance $sigma^2$, and $H$ is a linear Operator. This is what I don´t get. Can anybody help me to compute the log-likeliehood term $l(y,x)$ for the case that $y=Hx+e$?
Thanking you in anticipation,
Christian
bayes-theorem empirical-bayes
asked Nov 29 at 15:37
Chris S.
103
add a comment |
I´m currently reading the paper "The Little Engine that Could
Regularization by Denoising (RED)" by Yaniv Romano and Micheal Elad. In the beginning they come up with solving the Maximum aposteriori Probability (MAP) for an given (measured) image $y$ and a unknown image $x$. So one tries to solve ${argmax}_x P(xvert y)$ which is equivalent to solving ${argmin}_x -log(P(yvert x)) -log(P(x))$. The term $l(y,x)=-log(P(yvert x))$ is also know as the log-likelihood term. Then there is written that $l(y,x)=frac{1}{2sigma^2}vertvert Hx-yvert{vert}_2^2$ if $y=Hx+e$, where $e$ is a gaussian noise with variance $sigma^2$, and $H$ is a linear Operator. This is what I don´t get. Can anybody help me to compute the log-likeliehood term $l(y,x)$ for the case that $y=Hx+e$?
Thanking you in anticipation,
Christian
bayes-theorem empirical-bayes
asked Nov 29 at 15:37
Chris S.
103
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04
add a comment |
I´m currently reading the paper "The Little Engine that Could
Regularization by Denoising (RED)" by Yaniv Romano and Micheal Elad. In the beginning they come up with solving the Maximum aposteriori Probability (MAP) for an given (measured) image $y$ and a unknown image $x$. So one tries to solve ${argmax}_x P(xvert y)$ which is equivalent to solving ${argmin}_x -log(P(yvert x)) -log(P(x))$. The term $l(y,x)=-log(P(yvert x))$ is also know as the log-likelihood term. Then there is written that $l(y,x)=frac{1}{2sigma^2}vertvert Hx-yvert{vert}_2^2$ if $y=Hx+e$, where $e$ is a gaussian noise with variance $sigma^2$, and $H$ is a linear Operator. This is what I don´t get. Can anybody help me to compute the log-likeliehood term $l(y,x)$ for the case that $y=Hx+e$?
Thanking you in anticipation,
Christian
bayes-theorem empirical-bayes
asked Nov 29 at 15:37
Chris S.
103
I´m currently reading the paper "The Little Engine that Could
Regularization by Denoising (RED)" by Yaniv Romano and Micheal Elad. In the beginning they come up with solving the Maximum aposteriori Probability (MAP) for an given (measured) image $y$ and a unknown image $x$. So one tries to solve ${argmax}_x P(xvert y)$ which is equivalent to solving ${argmin}_x -log(P(yvert x)) -log(P(x))$. The term $l(y,x)=-log(P(yvert x))$ is also know as the log-likelihood term. Then there is written that $l(y,x)=frac{1}{2sigma^2}vertvert Hx-yvert{vert}_2^2$ if $y=Hx+e$, where $e$ is a gaussian noise with variance $sigma^2$, and $H$ is a linear Operator. This is what I don´t get. Can anybody help me to compute the log-likeliehood term $l(y,x)$ for the case that $y=Hx+e$?
Thanking you in anticipation,
Christian
bayes-theorem empirical-bayes
bayes-theorem empirical-bayes
asked Nov 29 at 15:37
Chris S.
103
asked Nov 29 at 15:37
Chris S.
103
asked Nov 29 at 15:37
Chris S.
103
asked Nov 29 at 15:37
Chris S.
103
asked Nov 29 at 15:37
Chris S.
103
103
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04
add a comment |
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04
add a comment |
1 Answer
1
active
oldest
votes
The prior assumption is that $y=Hx + e$, where $e$ follows a gaussian distribution with mean $0$ and covariance $sigma^2I$. Let me restate this using the notation $p(e) = mathcal{N}(e|0, sigma^2I)$.
This assumption implies that $p(y|x)=mathcal{N}(y|Hx, sigma^2)$ (note that we are considering a conditional distribution where $x$ is given, so we may see $x$ as deterministic).
Therefore, assuming the vectors $x$ and $y$ are $k$-dimensional,
begin{align}
-log p(y|x) &= -log left(frac{1}{sigma sqrt{(2pi)^k}} expleft(-frac{1}{2}(y-Hx)'sigma^{-2}I(y-Hx) right) right) \
&= frac{1}{2sigma^2}||y-Hx||^2 + text{const.}
end{align}
Assuming $sigma^2$ is fixed, the "const." is a constant and, therefore, does not affect the arg min.
answered Nov 29 at 17:15
D...
213113
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3018791%2fcomputing-the-log-likelihood-term%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The prior assumption is that $y=Hx + e$, where $e$ follows a gaussian distribution with mean $0$ and covariance $sigma^2I$. Let me restate this using the notation $p(e) = mathcal{N}(e|0, sigma^2I)$.
This assumption implies that $p(y|x)=mathcal{N}(y|Hx, sigma^2)$ (note that we are considering a conditional distribution where $x$ is given, so we may see $x$ as deterministic).
Therefore, assuming the vectors $x$ and $y$ are $k$-dimensional,
begin{align}
-log p(y|x) &= -log left(frac{1}{sigma sqrt{(2pi)^k}} expleft(-frac{1}{2}(y-Hx)'sigma^{-2}I(y-Hx) right) right) \
&= frac{1}{2sigma^2}||y-Hx||^2 + text{const.}
end{align}
Assuming $sigma^2$ is fixed, the "const." is a constant and, therefore, does not affect the arg min.
answered Nov 29 at 17:15
D...
213113
add a comment |
The prior assumption is that $y=Hx + e$, where $e$ follows a gaussian distribution with mean $0$ and covariance $sigma^2I$. Let me restate this using the notation $p(e) = mathcal{N}(e|0, sigma^2I)$.
This assumption implies that $p(y|x)=mathcal{N}(y|Hx, sigma^2)$ (note that we are considering a conditional distribution where $x$ is given, so we may see $x$ as deterministic).
Therefore, assuming the vectors $x$ and $y$ are $k$-dimensional,
begin{align}
-log p(y|x) &= -log left(frac{1}{sigma sqrt{(2pi)^k}} expleft(-frac{1}{2}(y-Hx)'sigma^{-2}I(y-Hx) right) right) \
&= frac{1}{2sigma^2}||y-Hx||^2 + text{const.}
end{align}
Assuming $sigma^2$ is fixed, the "const." is a constant and, therefore, does not affect the arg min.
answered Nov 29 at 17:15
D...
213113
add a comment |
The prior assumption is that $y=Hx + e$, where $e$ follows a gaussian distribution with mean $0$ and covariance $sigma^2I$. Let me restate this using the notation $p(e) = mathcal{N}(e|0, sigma^2I)$.
This assumption implies that $p(y|x)=mathcal{N}(y|Hx, sigma^2)$ (note that we are considering a conditional distribution where $x$ is given, so we may see $x$ as deterministic).
Therefore, assuming the vectors $x$ and $y$ are $k$-dimensional,
begin{align}
-log p(y|x) &= -log left(frac{1}{sigma sqrt{(2pi)^k}} expleft(-frac{1}{2}(y-Hx)'sigma^{-2}I(y-Hx) right) right) \
&= frac{1}{2sigma^2}||y-Hx||^2 + text{const.}
end{align}
Assuming $sigma^2$ is fixed, the "const." is a constant and, therefore, does not affect the arg min.
answered Nov 29 at 17:15
D...
213113
The prior assumption is that $y=Hx + e$, where $e$ follows a gaussian distribution with mean $0$ and covariance $sigma^2I$. Let me restate this using the notation $p(e) = mathcal{N}(e|0, sigma^2I)$.
This assumption implies that $p(y|x)=mathcal{N}(y|Hx, sigma^2)$ (note that we are considering a conditional distribution where $x$ is given, so we may see $x$ as deterministic).
Therefore, assuming the vectors $x$ and $y$ are $k$-dimensional,
begin{align}
-log p(y|x) &= -log left(frac{1}{sigma sqrt{(2pi)^k}} expleft(-frac{1}{2}(y-Hx)'sigma^{-2}I(y-Hx) right) right) \
&= frac{1}{2sigma^2}||y-Hx||^2 + text{const.}
end{align}
Assuming $sigma^2$ is fixed, the "const." is a constant and, therefore, does not affect the arg min.
answered Nov 29 at 17:15
D...
213113
edited Nov 30 at 10:31
answered Nov 29 at 17:15
D...
213113
answered Nov 29 at 17:15
D...
213113
answered Nov 29 at 17:15
D...
213113
213113
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3018791%2fcomputing-the-log-likelihood-term%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The book has to make some assumption about the distribution $P(y|x)$, like for instance assuming it's a gaussian or something like that.
– D...
Nov 29 at 15:49
We only know that $e$ has a gaussian distribution. Do we really need further assumption? The result is also dependent on $x$ and $y$.
– Chris S.
Nov 29 at 16:21
Sorry, I misunderstood the problem statement initially. The book is correct. I'll show you why in a few moments.
– D...
Nov 29 at 17:04