Derivative of function with 2 summations and powers
up vote
0
down vote
favorite
I'm trying to compute the gradient of a complex loss function but I'm stuck on calculating the derivative of part of the expression,namely:
with respect to $w_i$
$$sum_{n=1}^N(y^n -w_0 - sum_{j=1}^p A_{nj}w_j)^2$$
A - design matrix;
w - vector; y - vector; $w_0$ - first element of w
Edit
Where it said k it should have been n - now fixed
derivatives partial-derivative
add a comment |
up vote
0
down vote
favorite
I'm trying to compute the gradient of a complex loss function but I'm stuck on calculating the derivative of part of the expression,namely:
with respect to $w_i$
$$sum_{n=1}^N(y^n -w_0 - sum_{j=1}^p A_{nj}w_j)^2$$
A - design matrix;
w - vector; y - vector; $w_0$ - first element of w
Edit
Where it said k it should have been n - now fixed
derivatives partial-derivative
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to compute the gradient of a complex loss function but I'm stuck on calculating the derivative of part of the expression,namely:
with respect to $w_i$
$$sum_{n=1}^N(y^n -w_0 - sum_{j=1}^p A_{nj}w_j)^2$$
A - design matrix;
w - vector; y - vector; $w_0$ - first element of w
Edit
Where it said k it should have been n - now fixed
derivatives partial-derivative
I'm trying to compute the gradient of a complex loss function but I'm stuck on calculating the derivative of part of the expression,namely:
with respect to $w_i$
$$sum_{n=1}^N(y^n -w_0 - sum_{j=1}^p A_{nj}w_j)^2$$
A - design matrix;
w - vector; y - vector; $w_0$ - first element of w
Edit
Where it said k it should have been n - now fixed
derivatives partial-derivative
derivatives partial-derivative
edited Nov 25 at 11:19
asked Nov 25 at 10:16
Mr Wondeful
174
174
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14
add a comment |
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
accepted
For $iin{1,ldots,p}$:
$$begin{align*}&phantom{=} , frac{partial}{partial w_i}sum_{n=1}^Nleft[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nfrac{partial}{partial w_i}left[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)frac{partial}{partial w_i}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( -sum_{j=1}^p A_{nj}frac{partial w_j}{partial w_i}right)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( - A_{ni}right)right]\
&=-2sum_{n=1}^Nleft[A_{ni}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]
end{align*}$$
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
For $iin{1,ldots,p}$:
$$begin{align*}&phantom{=} , frac{partial}{partial w_i}sum_{n=1}^Nleft[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nfrac{partial}{partial w_i}left[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)frac{partial}{partial w_i}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( -sum_{j=1}^p A_{nj}frac{partial w_j}{partial w_i}right)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( - A_{ni}right)right]\
&=-2sum_{n=1}^Nleft[A_{ni}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]
end{align*}$$
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
add a comment |
up vote
0
down vote
accepted
For $iin{1,ldots,p}$:
$$begin{align*}&phantom{=} , frac{partial}{partial w_i}sum_{n=1}^Nleft[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nfrac{partial}{partial w_i}left[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)frac{partial}{partial w_i}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( -sum_{j=1}^p A_{nj}frac{partial w_j}{partial w_i}right)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( - A_{ni}right)right]\
&=-2sum_{n=1}^Nleft[A_{ni}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]
end{align*}$$
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
For $iin{1,ldots,p}$:
$$begin{align*}&phantom{=} , frac{partial}{partial w_i}sum_{n=1}^Nleft[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nfrac{partial}{partial w_i}left[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)frac{partial}{partial w_i}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( -sum_{j=1}^p A_{nj}frac{partial w_j}{partial w_i}right)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( - A_{ni}right)right]\
&=-2sum_{n=1}^Nleft[A_{ni}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]
end{align*}$$
For $iin{1,ldots,p}$:
$$begin{align*}&phantom{=} , frac{partial}{partial w_i}sum_{n=1}^Nleft[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nfrac{partial}{partial w_i}left[left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)^2right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)frac{partial}{partial w_i}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( -sum_{j=1}^p A_{nj}frac{partial w_j}{partial w_i}right)right]\
&=sum_{n=1}^Nleft[2left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)left( - A_{ni}right)right]\
&=-2sum_{n=1}^Nleft[A_{ni}left(y^n -w_0 - sum_{j=1}^p A_{nj}w_jright)right]
end{align*}$$
edited Nov 25 at 11:25
answered Nov 25 at 11:13
smcc
4,297517
4,297517
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
add a comment |
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
I'm so sorry I just noticed my mistake where it sais k it should be n, will substituting it everywhere be sufficient?
– Mr Wondeful
Nov 25 at 11:20
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
Yes, it does not matter whether the upper limit of the sum is called $N$ or $K$.
– smcc
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
I meant "k" as the index, sorry
– Mr Wondeful
Nov 25 at 11:22
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
Changing $k$ to $j$ in $A_{kj}$ just changes the last lines. I have updated my answer.
– smcc
Nov 25 at 11:26
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3012658%2fderivative-of-function-with-2-summations-and-powers%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
How can $y$ be a vector? Should it be $y_n$ rather than $y^n$?
– smcc
Nov 25 at 11:14