$newcommand{set}[1]{{#1}}$
$newcommand{mc}{mathcal}$
$newcommand{P}{mathbb P}$
$newcommand{N}{mathbb N}$
$newcommand{E}{mathbb E}$
$newcommand{mr}{mathscr}$

I am looking for the proof of the theorem in Markov chain theory which roughly states that a recurrent Markov chain admit an essentially unique invariant measure (See the theorem at the end for the precise statement).

The theorem is stated here (Theroem 10) but no proof or a reference is provided.

Definitions

Transition Probability Kernel

Let $S$ be a set and $mathcal S$ be a $sigma$-algebra on $S$.
A transition probability kernel on $S$ is a map $K:Stimes mathcal Sto [0, 1]$ such that
$bullet$ $K(x, cdot)$ is a probability measure on $(S, mathcal S)$ for each $xin S$.
$bullet$ The map $K(cdot, A)$ is a measurable function from $S$ to $[0, 1]$ for each $Ain mathcal S$.

We will refer to a measurable space $(S, mc S)$ equipped with a transition probability kernel as a Markov chain.

One can think of $K(x, A)$ as the probability of jumping from $x$ to $A$ in one step.
This viewpoint naturally given rise, for each $ngeq 1$, to a transition probability kernel $K^n:Stimes mathcal Sto [0, 1]$ as follows:

Let $mr P(S)$ denote the set of all the probability measures in $S$.
Define a map $K_sharp:mr P(S)to mr P(S)$ as $(K_sharpmu)(A)=int_S K_x(A) dmu(x)$ for all $muin mr P(S)$ and all $Ain mc S$.
Define $K^n$ as $Kcirc (K_sharp)^{n-1}$.
We can thus think of $K^n(x, A)$ as the probability of jumping from $x$ to $A$ in $n$ steps.
We refer to $K^n$ as the $n$-step transition kernel (arising out of $K$).

Forward Trajectories

Let $K$ be a transition probability kernel on $S$.
For each $ngeq 0$, let $Omega_n=prod_{i=0}^n S$ and equip it with the product $sigma$-algebra.
Write $P_x^0$ to denote the Dirac measure $delta_x$ on $Omega_0=S$.

We define a measure $P_x^1$ on $Omega_1$ as follows:
$$
P_x^1(E_0times E_1) = int_{E_0} P_y(E_1) dP_x^0(y)
$$
for all $E_0, E_1in mc S$.
Intuitively, $P_x^1(E_0times E_1)$ is the probability that the Markov chain ``follows the trajectory $E_0times E_1$" provided it starts at $x$.
It is easy to verify that $P_x^1$ is indeed a finitely additive measure on the algebra of measurable rectangles in $Omega_1$ and thus by the Caratheodory extension theorem extends uniquely to a measure on the Borel $sigma$-algebra of $Omega_1$.

More generally, once $P_x^n$ is constructed, we can construct $P_x^{n+1}$ by declaring
$$
P_x^{n+1}(E_0times cdots times E_{n+1}) = int_{E_0times cdots times E_n} P_{omega_n}(E_{n+1}) dP_x^n(omega)
$$
for all $E_0 , ldots, E_{n+1}in mc S$.
Again, intuitively, $P_x^{n+1}(E_0times cdots times E_{n+1})$ is the probability that the Markov chain follows the trajectory $E_0times cdots times E_{n+1}$ provided it starts at $x$.
Note that the push-forward of $P_x^{n}$ to $Omega_{n-1}$ by the projection map $Omega_nto Omega_{n-1}$ which kills the last coordinate is same as $P_x^{n-1}$.
So we have a unique probability measure $P_x$ on the product space $Omega=prod_{n=0}^infty S$ such that the push-forward of $P_x$ to $Omega_n$ under the projection map $Omegato Omega_n$ which kills all coordinates beyond $n$ is $P_x^n$.
See Figure, where all the arrows are projection maps.
The set $Omega$ equipped with the product $sigma$-algebra and the measure $P_x$ can be thought of as the limit of the system $ cdots to Omega_2toOmega_1toOmega_0$.

Irreducibility and Recurrence

Given a non-zero measure $phi$ on $S$, we say that $K$ is $phi$-irreducible if for all $Ain mathcal S$ with $phi(A)>0$ and all $xin S$ we have that $K^n(x, A)>0$ for some $ngeq 1$.

For $Ain mathcal S$, define a map $eta_A:Omegato N_0cup{infty}$ as $eta_A(omega)=|set{nin N_0: omega_nin A}|$, where $omega_n$ is the $n$-coordinate of $omega$.
We say that a $phi$-irreducible chain is recurrent if $E_{P_x}[eta_A]=infty$ for all $xin S$ whenever $phi(A)>0$.

Invariant Measure

Given a transition probability kernel $K$ on a state space $S$, we say that a measure $mu$ on $mc S$ is invariant if $mu(A)=int_S K_x(A) dmu(x)$ for all $Ain mc S$.

The Theorem

Theorem 10 here states the following.

Theorem 1.
A recurrent Markov chain has a unique invariant measure (up to a multiplicative constant).

However, no proof or a reference is provided.