Continuous Markov chain - expected time
$begingroup$
I have been studying problems from some of my lecture notes and was given this question
A continuous time Markov Chain $X_t$ with state space ${1,2,3}$ has infinitesimal generator
$$A= begin{bmatrix}-6 & 2 & 4\2 &-5 &3 \2 & 3& -5end{bmatrix}$$
If the chain starts in state 2, what is expected total time spent in state 2 before the chain first enters state 3
In the solution the answer is given as $$frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}(frac{2}{15})^{2}+.......= frac{frac{1}{5}}{1-frac{2}{15}}$$
I know from my lectures notes that
The expected time at state 2 is: $(frac{1}{5})$
The chain jumps to state 1 with probability $(frac{2}{5})$ and state 3 with probability $(frac{3}{5})$
and I know chain jumps from 1 to 2 with probability $(frac{1}{3})$
so the probability that the chain return to state 2 in two steps, starting at 2 is thus $(frac{2}{5})(frac{1}{3})=(frac{2}{15})$
So after finding these probabilities I can understand the right hand side of the equation given in the solution but im confused about how it becomes $$frac{frac{1}{5}}{1-frac{2}{15}}$$
I have been looking through my notes and haven't been able to find anything. Any help would be great
probability statistics markov-chains
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
$endgroup$
add a comment |
$begingroup$
I have been studying problems from some of my lecture notes and was given this question
A continuous time Markov Chain $X_t$ with state space ${1,2,3}$ has infinitesimal generator
$$A= begin{bmatrix}-6 & 2 & 4\2 &-5 &3 \2 & 3& -5end{bmatrix}$$
If the chain starts in state 2, what is expected total time spent in state 2 before the chain first enters state 3
In the solution the answer is given as $$frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}(frac{2}{15})^{2}+.......= frac{frac{1}{5}}{1-frac{2}{15}}$$
I know from my lectures notes that
The expected time at state 2 is: $(frac{1}{5})$
The chain jumps to state 1 with probability $(frac{2}{5})$ and state 3 with probability $(frac{3}{5})$
and I know chain jumps from 1 to 2 with probability $(frac{1}{3})$
so the probability that the chain return to state 2 in two steps, starting at 2 is thus $(frac{2}{5})(frac{1}{3})=(frac{2}{15})$
So after finding these probabilities I can understand the right hand side of the equation given in the solution but im confused about how it becomes $$frac{frac{1}{5}}{1-frac{2}{15}}$$
I have been looking through my notes and haven't been able to find anything. Any help would be great
probability statistics markov-chains
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
$endgroup$
2
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15
add a comment |
$begingroup$
I have been studying problems from some of my lecture notes and was given this question
A continuous time Markov Chain $X_t$ with state space ${1,2,3}$ has infinitesimal generator
$$A= begin{bmatrix}-6 & 2 & 4\2 &-5 &3 \2 & 3& -5end{bmatrix}$$
If the chain starts in state 2, what is expected total time spent in state 2 before the chain first enters state 3
In the solution the answer is given as $$frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}(frac{2}{15})^{2}+.......= frac{frac{1}{5}}{1-frac{2}{15}}$$
I know from my lectures notes that
The expected time at state 2 is: $(frac{1}{5})$
The chain jumps to state 1 with probability $(frac{2}{5})$ and state 3 with probability $(frac{3}{5})$
and I know chain jumps from 1 to 2 with probability $(frac{1}{3})$
so the probability that the chain return to state 2 in two steps, starting at 2 is thus $(frac{2}{5})(frac{1}{3})=(frac{2}{15})$
So after finding these probabilities I can understand the right hand side of the equation given in the solution but im confused about how it becomes $$frac{frac{1}{5}}{1-frac{2}{15}}$$
I have been looking through my notes and haven't been able to find anything. Any help would be great
probability statistics markov-chains
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
$endgroup$
I have been studying problems from some of my lecture notes and was given this question
A continuous time Markov Chain $X_t$ with state space ${1,2,3}$ has infinitesimal generator
$$A= begin{bmatrix}-6 & 2 & 4\2 &-5 &3 \2 & 3& -5end{bmatrix}$$
If the chain starts in state 2, what is expected total time spent in state 2 before the chain first enters state 3
In the solution the answer is given as $$frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}(frac{2}{15})^{2}+.......= frac{frac{1}{5}}{1-frac{2}{15}}$$
I know from my lectures notes that
The expected time at state 2 is: $(frac{1}{5})$
The chain jumps to state 1 with probability $(frac{2}{5})$ and state 3 with probability $(frac{3}{5})$
and I know chain jumps from 1 to 2 with probability $(frac{1}{3})$
so the probability that the chain return to state 2 in two steps, starting at 2 is thus $(frac{2}{5})(frac{1}{3})=(frac{2}{15})$
So after finding these probabilities I can understand the right hand side of the equation given in the solution but im confused about how it becomes $$frac{frac{1}{5}}{1-frac{2}{15}}$$
I have been looking through my notes and haven't been able to find anything. Any help would be great
probability statistics markov-chains
probability statistics markov-chains
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
edited Dec 13 '18 at 22:32
Tianlalu
3,08621038
3,08621038
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
asked Dec 13 '18 at 18:12
Rito LoweRito Lowe
516
516
2
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15
add a comment |
2
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15
2
2
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
It is a geometric series. In this particular case you could say:
$$S = frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+cdots$$
multiplying by $frac2{15}$ gives
$$frac2{15}S = frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+frac{1}{5}left(frac{2}{15}right)^{3}+cdots$$
and subtracting gives
$$left(1-frac2{15}right)S = frac{1}{5}$$
so
$$S = frac{frac{1}{5}}{1-frac2{15}}$$
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3038398%2fcontinuous-markov-chain-expected-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
It is a geometric series. In this particular case you could say:
$$S = frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+cdots$$
multiplying by $frac2{15}$ gives
$$frac2{15}S = frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+frac{1}{5}left(frac{2}{15}right)^{3}+cdots$$
and subtracting gives
$$left(1-frac2{15}right)S = frac{1}{5}$$
so
$$S = frac{frac{1}{5}}{1-frac2{15}}$$
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
$endgroup$
add a comment |
$begingroup$
It is a geometric series. In this particular case you could say:
$$S = frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+cdots$$
multiplying by $frac2{15}$ gives
$$frac2{15}S = frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+frac{1}{5}left(frac{2}{15}right)^{3}+cdots$$
and subtracting gives
$$left(1-frac2{15}right)S = frac{1}{5}$$
so
$$S = frac{frac{1}{5}}{1-frac2{15}}$$
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
$endgroup$
add a comment |
$begingroup$
It is a geometric series. In this particular case you could say:
$$S = frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+cdots$$
multiplying by $frac2{15}$ gives
$$frac2{15}S = frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+frac{1}{5}left(frac{2}{15}right)^{3}+cdots$$
and subtracting gives
$$left(1-frac2{15}right)S = frac{1}{5}$$
so
$$S = frac{frac{1}{5}}{1-frac2{15}}$$
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
$endgroup$
It is a geometric series. In this particular case you could say:
$$S = frac{1}{5}+frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+cdots$$
multiplying by $frac2{15}$ gives
$$frac2{15}S = frac{1}{5}frac{2}{15}+frac{1}{5}left(frac{2}{15}right)^{2}+frac{1}{5}left(frac{2}{15}right)^{3}+cdots$$
and subtracting gives
$$left(1-frac2{15}right)S = frac{1}{5}$$
so
$$S = frac{frac{1}{5}}{1-frac2{15}}$$
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
answered Dec 14 '18 at 9:21
HenryHenry
100k480165
100k480165
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3038398%2fcontinuous-markov-chain-expected-time%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
$begingroup$
$$sum_{n=0}^infty ax^n=frac a{1-x}qquad (a=1/5, x=2/15)$$
$endgroup$
– Did
Dec 13 '18 at 18:15