Line Of Best Fit With Perpendicular Error
$begingroup$
The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?
geometry statistics regression
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
$endgroup$
add a comment |
$begingroup$
The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?
geometry statistics regression
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
$endgroup$
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01
add a comment |
$begingroup$
The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?
geometry statistics regression
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
$endgroup$
The standard statistical formula for the least squares error gives us a line that minimises the sum of the vertical distances of the sample points to the line. Suppose that I wanted to find the equation of a line that minimises the sum of the perpendicular distance of the points to the line, is there a way of analytically solving this problem?
geometry statistics regression
geometry statistics regression
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
asked Dec 20 '18 at 10:17
Elie BergmanElie Bergman
1,993815
1,993815
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01
add a comment |
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:
Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.
Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html
Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis
A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.
LATTER ADDITION
Numerical example with the principal component method :
Comparison with the above least mean square offset method :
The results of both methods are exactly the same, as it is analytically expected.
Graphical representation :
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
$endgroup$
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
add a comment |
$begingroup$
Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.
So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$
This will automatically handle all your roubles ( if you add some tea at the start ).
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3047373%2fline-of-best-fit-with-perpendicular-error%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:
Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.
Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html
Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis
A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.
LATTER ADDITION
Numerical example with the principal component method :
Comparison with the above least mean square offset method :
The results of both methods are exactly the same, as it is analytically expected.
Graphical representation :
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
$endgroup$
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
add a comment |
$begingroup$
This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:
Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.
Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html
Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis
A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.
LATTER ADDITION
Numerical example with the principal component method :
Comparison with the above least mean square offset method :
The results of both methods are exactly the same, as it is analytically expected.
Graphical representation :
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
$endgroup$
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
add a comment |
$begingroup$
This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:
Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.
Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html
Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis
A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.
LATTER ADDITION
Numerical example with the principal component method :
Comparison with the above least mean square offset method :
The results of both methods are exactly the same, as it is analytically expected.
Graphical representation :
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
$endgroup$
This is well-known. See https://fr.scribd.com/doc/14819165/Regressions-coniques-quadriques-circulaire-spherique . Figure below:
Note : For the choice of sign, compute $sum_{k=1}^n(ax_k+b-y_k)^2$ in both cases and keep the smaller.
Also, see : http://mathworld.wolfram.com/LeastSquaresFittingPerpendicularOffsets.html
Moreover, it is of interest to look at the related Principal Component Analysis method. https://en.wikipedia.org/wiki/Principal_component_analysis
A numerical example of the principal component regression is given in page 12 of this paper : https://fr.scribd.com/doc/31477970/Regressions-et-trajectoires-3D . This example is in 3D. But is is very easy to see how proceed on the same manner in 2D. which is even simpler.
LATTER ADDITION
Numerical example with the principal component method :
Comparison with the above least mean square offset method :
The results of both methods are exactly the same, as it is analytically expected.
Graphical representation :
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
edited Dec 22 '18 at 7:13
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
answered Dec 20 '18 at 10:38
JJacquelinJJacquelin
44.2k21853
44.2k21853
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
add a comment |
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Jean ! This problem of orthogonal distances is very important to me. Does your method work "easily" for a parabola (just to stay simple) ?
$endgroup$
– Claude Leibovici
Dec 22 '18 at 6:31
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Hi Claude ! Unfortunately the least mean square perpendicular offset leads to higher order polynomial equations in case of parabolic instead of linear fit. I will look at this more carefully. But I think that probably numerical iterative method is inevitable.
$endgroup$
– JJacquelin
Dec 22 '18 at 7:30
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
Yes, I know and I expect it. But, I don't have any problems with polynomials if this is just a way to get an easy solution. Think about the problem of an hyperbola; there is one point in error close to the asymptote. It would be great if you could work on quadratic, cubic ... fits. Cheers & Merry Xmas if we don't meet again before.
$endgroup$
– Claude Leibovici
Dec 22 '18 at 7:46
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
@Claude Leibovici. I will not have enough time at short term, even for the case of parabola only. I think that it should be a good idea to open a new question on the forum about this wide subject, with an example of data to make it more concrete. Chers and Merry Xmas.
$endgroup$
– JJacquelin
Dec 22 '18 at 8:53
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
$begingroup$
Keep the problem in mind for a parabola even $y=a x^2$. It would interesting. I shall work on my side. Cheers :-)
$endgroup$
– Claude Leibovici
Dec 22 '18 at 8:56
add a comment |
$begingroup$
Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.
So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$
This will automatically handle all your roubles ( if you add some tea at the start ).
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
$endgroup$
add a comment |
$begingroup$
Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.
So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$
This will automatically handle all your roubles ( if you add some tea at the start ).
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
$endgroup$
add a comment |
$begingroup$
Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.
So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$
This will automatically handle all your roubles ( if you add some tea at the start ).
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
$endgroup$
Just build a function of higher in-dimensionality than whatever you tried to fit firstly. And instead of fitting to values you fit towards a level-set.
So instead of fitting to $$f(x)=kx+m, f(x_k) = y_k$$
We fit to:
$$f(x,y) = ax+by+c, f(x_k,y_k) = 0$$
This will automatically handle all your roubles ( if you add some tea at the start ).
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
answered Dec 21 '18 at 22:33
mathreadlermathreadler
15k72263
15k72263
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3047373%2fline-of-best-fit-with-perpendicular-error%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Make a higher dimensional level set instead.
$endgroup$
– mathreadler
Dec 21 '18 at 22:01