Ytukyg

For the answer, we can get support from two claims:
Claim 1: If $A$ is real and symmetric, then it has real eigenvalues. Proof here
Claim 2: If $lambda$ is an eigenvalue of $A$, then $lambda^2$ is an Eigenvalue of $A^2$. Proof here

So by those two claims we can say that if $lambda$ is an eigenvalue of $A$, then the corresponding eigenvalue $lambda^2$ of $A^2$ is nonnegative. But my problem is how can we guarantee that it will cover all the possible eigenvalues of $A^2$?

Begin with the diagonalization
$$
A=PDP^{-1}
$$
where $P$ is the matrix of eigenvectors and $D$ is the diagonal matrix with eigenvalues on the main diagonal. Then
$$
A^2=(PDP^{-1})(PDP^{-1})=PD^2P^{-1}
$$
from which the claim follows.

A symmetric matrix is diagonalizable, $D=PAP^{-1}$ where $A$ is symmetric and $D$ diagonal. $D^2=PA^2P^{-1}$, this implies that the eigenvalues of $A^2$ are the square of the eigenvalues of $A$.

More than this is true: if $B$ is any real matrix, then the eigenvalues of $B^TB$ are real and nonnegative.

The fact that the eigenvalues are real follows from your claim 1.

Suppose $lambda$ is an eigenvalue with eigenvector $v$; then
$$
lambda(v^Tv)=v^T(lambda v)=v^TB^TBv=(Bv)^T(Bv)ge0
$$
so $lambdage0$.

In your case $A$ is symmetric, so $A^2=A^TA$.

With diagonalization you can indeed prove that the eigenvalues of $A^2$ are the squares of the eigenvalues of $A$, but it is not necessary for proving the statement you have.

Suppose $J$ is a Jordan form of $A$, then it should be clear that the diagonal elements of $J^2$ are the squares of the diagonal elements of $J$. In other words, the eigenvalues of
$A^2$ are exactly the squares of the eigenvalues of $A$. This is true for any
square matrix, and is true much more generally (that is, the eigenvalues of the function of a matrix are the functions of the eigenvalues of the matrix).

Hence the eigenvalues are all of the form $x^2$ where $x$ is real.

Let $lambda = a+ibin mathbb{C}setminus mathbb{R}$. For any $xinmathbb{C}^n$ we have

begin{align}
|(A^2-lambda I)x|^2|x|^2 &ge left|langle (A^2-lambda I)x,x rangleright|^2 \
&= left|langle A^2x,xrangle - lambdalangle x,xrangleright|^2\
&= left|left(|Ax|^2 - a|x|^2right) + ib|x|^2right|^2\
&= left(|Ax|^2 - a|x|^2right)^2 + |b|^2|x|^2\
&ge |b|^2|x|^4
end{align}
so $|(A^2-lambda I)x| ge |b||x|$. Since $bne 0$, we see that $A^2-lambda I$ is bounded from below and hence injective. Therefore $lambda$ is not an eigenvalue of $A^2$.

Now let $lambda < 0$. For any $xinmathbb{C}^n$ we have

$$|(A^2-lambda I)x||x| ge langle (A^2-lambda I)x,xrangle = langle A^2x,xrangle - lambdalangle x,xrangle = |Ax|^2 - lambda |x|^2 ge -lambda |x|^2$$
so again $A^2-lambda I$ is bounded from below and hence injective. Therefore $lambda$ is not an eigenvalue of $A^2$.

We conclude that the eigenvalues of $A^2$ are contained in $[0, +inftyrangle$.