Why SVD is not unique but the Moore-Penrose pseudo inverse is unique?
$begingroup$
I feel confused about the uniqueness of the Moore-Penrose inverse generated from SVD.
For any matrix $A$, if $X$ satisfied $$AXA=A, XAX=X, (AX)^mathrm{T}=AX, (XA)^mathrm{T}=XA $$then $X$ is called the Moore-Penrose inverse of $A$.
If $A$ has the SVD(singular value decomposition)$$A=Pleft[begin{matrix}Lambda_r&0\0&0end{matrix}right]Q^mathrm{T}$$
then it is easy to prove that$$A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$$ is a Moore-Penrose inverse.
If $X$ and $Y$ are both Moore-Penrose inverse of $A$, from the equation$$X=XAX=X(AX)^mathrm{T}=XX^mathrm{T}A^mathrm{T}=XX^mathrm{T}(AYA)^mathrm{T}=X(AX)^mathrm{T}(AY)^mathrm{T}=(XAX)AY=XAY=(XA)^mathrm{T}YAY=A^mathrm{T}X^mathrm{T}A^mathrm{T}Y^mathrm{T}Y=A^mathrm{T}Y^mathrm{T}Y=(YA)^mathrm{T}Y=YAY=Y$$
we can see that the Moore-Penrose inverse is unique.
However, the Moore-Penrose inverse depends on the SVD and SVD is not unique. How to explain it?
linear-algebra matrices matrix-decomposition svd pseudoinverse
asked Dec 31 '18 at 12:05
breggbregg
235
$endgroup$
|
show 1 more comment
$begingroup$
I feel confused about the uniqueness of the Moore-Penrose inverse generated from SVD.
For any matrix $A$, if $X$ satisfied $$AXA=A, XAX=X, (AX)^mathrm{T}=AX, (XA)^mathrm{T}=XA $$then $X$ is called the Moore-Penrose inverse of $A$.
If $A$ has the SVD(singular value decomposition)$$A=Pleft[begin{matrix}Lambda_r&0\0&0end{matrix}right]Q^mathrm{T}$$
then it is easy to prove that$$A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$$ is a Moore-Penrose inverse.
If $X$ and $Y$ are both Moore-Penrose inverse of $A$, from the equation$$X=XAX=X(AX)^mathrm{T}=XX^mathrm{T}A^mathrm{T}=XX^mathrm{T}(AYA)^mathrm{T}=X(AX)^mathrm{T}(AY)^mathrm{T}=(XAX)AY=XAY=(XA)^mathrm{T}YAY=A^mathrm{T}X^mathrm{T}A^mathrm{T}Y^mathrm{T}Y=A^mathrm{T}Y^mathrm{T}Y=(YA)^mathrm{T}Y=YAY=Y$$
we can see that the Moore-Penrose inverse is unique.
However, the Moore-Penrose inverse depends on the SVD and SVD is not unique. How to explain it?
linear-algebra matrices matrix-decomposition svd pseudoinverse
asked Dec 31 '18 at 12:05
breggbregg
235
$endgroup$
1
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
1
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
1
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18
|
show 1 more comment
$begingroup$
I feel confused about the uniqueness of the Moore-Penrose inverse generated from SVD.
For any matrix $A$, if $X$ satisfied $$AXA=A, XAX=X, (AX)^mathrm{T}=AX, (XA)^mathrm{T}=XA $$then $X$ is called the Moore-Penrose inverse of $A$.
If $A$ has the SVD(singular value decomposition)$$A=Pleft[begin{matrix}Lambda_r&0\0&0end{matrix}right]Q^mathrm{T}$$
then it is easy to prove that$$A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$$ is a Moore-Penrose inverse.
If $X$ and $Y$ are both Moore-Penrose inverse of $A$, from the equation$$X=XAX=X(AX)^mathrm{T}=XX^mathrm{T}A^mathrm{T}=XX^mathrm{T}(AYA)^mathrm{T}=X(AX)^mathrm{T}(AY)^mathrm{T}=(XAX)AY=XAY=(XA)^mathrm{T}YAY=A^mathrm{T}X^mathrm{T}A^mathrm{T}Y^mathrm{T}Y=A^mathrm{T}Y^mathrm{T}Y=(YA)^mathrm{T}Y=YAY=Y$$
we can see that the Moore-Penrose inverse is unique.
However, the Moore-Penrose inverse depends on the SVD and SVD is not unique. How to explain it?
linear-algebra matrices matrix-decomposition svd pseudoinverse
asked Dec 31 '18 at 12:05
breggbregg
235
$endgroup$
I feel confused about the uniqueness of the Moore-Penrose inverse generated from SVD.
For any matrix $A$, if $X$ satisfied $$AXA=A, XAX=X, (AX)^mathrm{T}=AX, (XA)^mathrm{T}=XA $$then $X$ is called the Moore-Penrose inverse of $A$.
If $A$ has the SVD(singular value decomposition)$$A=Pleft[begin{matrix}Lambda_r&0\0&0end{matrix}right]Q^mathrm{T}$$
then it is easy to prove that$$A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$$ is a Moore-Penrose inverse.
If $X$ and $Y$ are both Moore-Penrose inverse of $A$, from the equation$$X=XAX=X(AX)^mathrm{T}=XX^mathrm{T}A^mathrm{T}=XX^mathrm{T}(AYA)^mathrm{T}=X(AX)^mathrm{T}(AY)^mathrm{T}=(XAX)AY=XAY=(XA)^mathrm{T}YAY=A^mathrm{T}X^mathrm{T}A^mathrm{T}Y^mathrm{T}Y=A^mathrm{T}Y^mathrm{T}Y=(YA)^mathrm{T}Y=YAY=Y$$
we can see that the Moore-Penrose inverse is unique.
However, the Moore-Penrose inverse depends on the SVD and SVD is not unique. How to explain it?
linear-algebra matrices matrix-decomposition svd pseudoinverse
linear-algebra matrices matrix-decomposition svd pseudoinverse
asked Dec 31 '18 at 12:05
breggbregg
235
asked Dec 31 '18 at 12:05
breggbregg
235
edited Dec 31 '18 at 12:20
bregg
asked Dec 31 '18 at 12:05
breggbregg
235
asked Dec 31 '18 at 12:05
breggbregg
235
asked Dec 31 '18 at 12:05
breggbregg
235
235
1
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
1
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
1
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18
|
show 1 more comment
1
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
1
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
1
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18
1
1
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
1
1
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
1
1
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18
|
show 1 more comment
1 Answer
1
active
oldest
votes
$begingroup$
The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have
$$
Lambda_r = pmatrix{s_1 I_{k_1} \ & ddots \ && s_{m-1} I_{k_{m-1}}}, quad Sigma = pmatrix{Lambda_r \ & 0_{k_m}}
$$
Then $A = PSigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form
$$
U = pmatrix{U^{(1)}\ & ddots \ && U^{(m)}}
$$
where $U^{(j)}$ is (orthogonal and) of size $k_j times k_j$.
With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have
$$
[Q_0U] pmatrix{Lambda_r^{-1} \ & 0} [P_0U]^T = Q_0 pmatrix{Lambda_r^{-1} \ & 0} P_0
$$
It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $Lambda_r^{-1}$ and block-matrix multiplication.
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3057644%2fwhy-svd-is-not-unique-but-the-moore-penrose-pseudo-inverse-is-unique%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have
$$
Lambda_r = pmatrix{s_1 I_{k_1} \ & ddots \ && s_{m-1} I_{k_{m-1}}}, quad Sigma = pmatrix{Lambda_r \ & 0_{k_m}}
$$
Then $A = PSigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form
$$
U = pmatrix{U^{(1)}\ & ddots \ && U^{(m)}}
$$
where $U^{(j)}$ is (orthogonal and) of size $k_j times k_j$.
With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have
$$
[Q_0U] pmatrix{Lambda_r^{-1} \ & 0} [P_0U]^T = Q_0 pmatrix{Lambda_r^{-1} \ & 0} P_0
$$
It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $Lambda_r^{-1}$ and block-matrix multiplication.
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
$endgroup$
add a comment |
$begingroup$
The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have
$$
Lambda_r = pmatrix{s_1 I_{k_1} \ & ddots \ && s_{m-1} I_{k_{m-1}}}, quad Sigma = pmatrix{Lambda_r \ & 0_{k_m}}
$$
Then $A = PSigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form
$$
U = pmatrix{U^{(1)}\ & ddots \ && U^{(m)}}
$$
where $U^{(j)}$ is (orthogonal and) of size $k_j times k_j$.
With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have
$$
[Q_0U] pmatrix{Lambda_r^{-1} \ & 0} [P_0U]^T = Q_0 pmatrix{Lambda_r^{-1} \ & 0} P_0
$$
It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $Lambda_r^{-1}$ and block-matrix multiplication.
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
$endgroup$
add a comment |
$begingroup$
The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have
$$
Lambda_r = pmatrix{s_1 I_{k_1} \ & ddots \ && s_{m-1} I_{k_{m-1}}}, quad Sigma = pmatrix{Lambda_r \ & 0_{k_m}}
$$
Then $A = PSigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form
$$
U = pmatrix{U^{(1)}\ & ddots \ && U^{(m)}}
$$
where $U^{(j)}$ is (orthogonal and) of size $k_j times k_j$.
With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have
$$
[Q_0U] pmatrix{Lambda_r^{-1} \ & 0} [P_0U]^T = Q_0 pmatrix{Lambda_r^{-1} \ & 0} P_0
$$
It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $Lambda_r^{-1}$ and block-matrix multiplication.
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
$endgroup$
The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have
$$
Lambda_r = pmatrix{s_1 I_{k_1} \ & ddots \ && s_{m-1} I_{k_{m-1}}}, quad Sigma = pmatrix{Lambda_r \ & 0_{k_m}}
$$
Then $A = PSigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form
$$
U = pmatrix{U^{(1)}\ & ddots \ && U^{(m)}}
$$
where $U^{(j)}$ is (orthogonal and) of size $k_j times k_j$.
With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have
$$
[Q_0U] pmatrix{Lambda_r^{-1} \ & 0} [P_0U]^T = Q_0 pmatrix{Lambda_r^{-1} \ & 0} P_0
$$
It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $Lambda_r^{-1}$ and block-matrix multiplication.
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
answered Dec 31 '18 at 19:35
OmnomnomnomOmnomnomnom
129k792185
129k792185
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3057644%2fwhy-svd-is-not-unique-but-the-moore-penrose-pseudo-inverse-is-unique%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = hat b$, where $hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique.
$endgroup$
– littleO
Dec 31 '18 at 12:14
1
$begingroup$
@littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique.
$endgroup$
– bregg
Dec 31 '18 at 12:28
1
$begingroup$
Just because $Q,Lambda_R,P$ are not unique does not imply that the product $A^+ = Qleft[begin{matrix}Lambda_r^{-1}&0\0&0end{matrix}right]P^mathrm{T}$ is not unique
$endgroup$
– user617446
Dec 31 '18 at 14:04
$begingroup$
The reason is similar to the fact that $1=2cdot2^{-1}=3cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$.
$endgroup$
– egreg
Dec 31 '18 at 15:38
$begingroup$
@egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula
$endgroup$
– Omnomnomnom
Dec 31 '18 at 19:18