Ytukyg

Given a real-symmetric (or Hermitian), positive definite matrix $A$, it is well known that: $$lambda_{min}leqdfrac{(x,Ax)}{(x,x)}. tag{1}$$

This is a direct consequence of the min-max theorem and also easily proved by the fact that such an $A$ has orthonormal eigenbasis. But is there any way to prove this without invoking the Spectral theorem or SVD decomposition or anything that is similarly powerful?

The best I could do was:
$$dfrac{(x,Ax)}{(x,x)}geq lambdacdotdfrac{(x,v)^2}{(v,v)(x,x)} tag{2}$$
where $lambda$ and $v$ are an arbitrary eigenpair of $A$, which is weaker than $(1)$ by Cauchy-Schwartz.

Let $m = inf_{xne 0}frac{langle Ax,xrangle}{|x|^2} = inf_{|x|=1}langle Ax,xrangle$. We claim that $lambda_{text{min}} le m$.

It suffices to show that $m$ is indeed an eigenvalue for $A$.

For that, consider a positive definite matrix $B ge 0$ and recall that $$|B| = sup_{|x| = 1} |langle Bx,xrangle| = sup_{|x| = 1} langle Bx,xrangle$$

Pick a sequence $(x_n)_n$ of vectors on the unit sphere such that $|B| = lim_{ntoinfty} langle Bx_n, x_nrangle$. We have
$$|Bx_n - |B|x_n|^2 = |Bx_n|^2 + |B|^2 - 2|B|langle Bx_n, x_nrangle le 2|B|(|B| - langle Bx_n, x_nrangle ) xrightarrow{ntoinfty} 0$$

so $lim_{ntoinfty} |Bx_n - |B|x_n| = 0$. We conclude that $B - |B|I$ is not bounded from below so it cannot be invertible. Hence $|B|$ is an eigenvalue of $B$.

Now consider $B = |A|I - A ge 0$. We have $$|B| = sup_{|x| = 1} langle Bx,xrangle = |A| - inf_{|x| = 1} langle Ax,xrangle = |A| - m$$

Above we showed that $|B| = |A|-m$ is an eigenvalue of $B = |A|I - A$ so $m$ is an eigenvalue of $A$.

To see this, notice that for any eigenvalue $lambda$ of $A$ holds $lambda ge m$. Indeed, consider $lambda = m-varepsilon$ for some $varepsilon > 0$.

Then for any vector $x$ we have $$langle (A-lambda I)x,xrangle = langle Ax,xrangle - lambdalangle x,xrangle ge (m-lambda)|x|^2 = varepsilon|x|^2$$

so $$|(A-lambda I)x||x| ge |langle (A-lambda I)x,xrangle|ge varepsilon|x|^2$$

Hence $A-lambda I$ is bounded from below and thus injective. Therefore $lambda$ cannot be an eigenvalue.

$$text{Let} R(X) = frac{langle X, AX rangle}{langle X, X rangle},$$

which (by setting $U=X/|X|$, i.e. restricting our attention to the unit sphere)is the same as

$$R(U)=langle U, AU rangle.$$

There is a classical result

Proposition : For any $U$ belonging to the unit sphere, $R(U)$ is a barycenter (normlized weighted mean) with positive coefficients of the eigenvalues of $A$.

Corollary $R(U)$ is necessary inside the range $[lambda_{min},lambda_{max}]$.

As I haven't seen it explained in a simple, non allusive way, I prove it again :

Proof : consider an eigendecomposition :

$$A=P^TDP text{with} D=diag(lambda_1,...lambda_n)$$

Then : $$R(U)=langle U, P^TDPU rangle =langle underbrace{PU}_V, DPU rangle=langle V, DV rangle tag{1}$$

Letting $v_1,v_2,... v_n$ be the coordinates of $V$, (recall that
$v_1^2+v_2^2+...+v_n^2=1$ because $U$ belongs to the unit sphere), we can write :

$$R(U)=v_1lambda_1v_1+ v_2lambda_2v_2+cdots + v_nlambda_nv_n$$

i.e.,

$$R(U)=v_1^2lambda_1+ v_2^2lambda_2+cdots + v_n^2lambda_n$$

which is the looked for weighted average of the $lambda_k$ by positive weights $v_k^2$ whose sum is $1$.