Ytukyg

$lim_{xto -infty}sqrt{x^2+5x+3}+x = lim_{xto -infty}(-x)+x$

Apparently, the 2nd step is illegal here.

In this step, you dropped $5x+3$ to go from $sqrt{x^2}$ to $-x$, but what makes you think you can just leave out the terms $5x+3$...? Surely you agree that $x^2+5x+3 ne x^2$.

But in the 3th step I used $x^2-x^2=0$, how is that legal?

How could that not be legal? The difference of two equal real numbers is $0$, of course!

The same goes for:

Also, in the 2nd step I implicitly used:
$-xsqrt{x^2+5x+3}+xsqrt{x^2+5x+3}=0$

Addition after comment: the idea of dominating terms (in a polynomial) is fine, but you cannot choose to apply the limit "locally" and leave the other terms unchanged.

To clarify, this is fine:
$$lim_{x to -infty}sqrt{x^2+5x+3}=lim_{x to -infty}sqrt{x^2}=+infty$$
but that's not what we have here; we have:
$$lim_{x to -infty}left(sqrt{x^2+5x+3}color{blue}{+x}right)$$
and we cannot take the limit of the terms separately
$$lim_{x to -infty}left(sqrt{x^2+5x+3}color{blue}{+x}right)color{red}{ne}lim_{x to -infty}sqrt{x^2+5x+3}+lim_{x to -infty} x$$
because both limits need to exist in order for this step to be valid.

The first step you took simply wasn’t correct.

$$lim_{x to -infty}sqrt{x^2+5x+3}+x color{red}{neq lim_{x to infty}-x+x}$$

You shouldn’t split a limit if the individual limits don’t exist or are undefined. Going for $infty-infty$ is certainly incorrect as it’s an indeterminate form, so you should always avoid it.

As for your question, you’re cancelling out $x^2$ and $-x^2$. That’s simple algebra and has nothing to do with whether $x$ is approaching an infinite value or not. It always applies for all additive inverses.

You may set $x = -frac{1}{t}$ and consider the limit for $stackrel{trightarrow 0+}{longrightarrow}$:
$$begin{eqnarray*} sqrt{x^2+5x+3}+x
& stackrel{x = -frac{1}{t}}{=} & frac{sqrt{1-5t +3t^2} - 1}{t} \
& stackrel{trightarrow 0+}{longrightarrow} & f'(0) = -frac{5}{2}mbox{ for } f(t) = sqrt{1-5t +3t^2}
end{eqnarray*}$$

The first step is illegal, not the second. This is because $sqrt{x^2 + 5x + 3} notequiv -x$.

Also addressing your comment:

The idea behind $lim_{x to -infty} sqrt{x^2 + 5x + 3} = dots = lim_{x to -infty} −x$ was, that $x^2$ dominantes $5x+3$. I also thought if that argument is wrong here, if $lim_{x to -infty} x − x = 0$ is legal and everything of the same "type"; then I'm okay again. Then I have to think about why the domination argument doesn't work as I thought it does here.

You are right in thinking that $x^2$ dominates $5x+3$, so that $color{blue}{sqrt{x^2 + 5x + 3}} sim color{red}{-x}$ for large negative $x$. Indeed there is no problem when computing their ratio:

$$lim_{x to -infty} frac{color{blue}{sqrt{x^2 + 5x + 3}}}{color{red}{-x}} = 1$$

However, because you are computing the difference between $color{blue}{sqrt{x^2 + 5x + 3}}$ and $color{red}{-x}$, such an approximation is not fine enough. You need more terms of the series expansion:

$$
begin{align*}
color{blue}{sqrt{x^2 + 5x + 3}}
&= -x sqrt{1 + frac{5}{x} + frac{3}{x^2}} \
&sim -x left(1 + frac{1}{2} cdot frac{5}{x} + Oleft(frac{1}{x^2}right) right) \
&= color{red}{-x} - frac{5}{2} + Oleft(frac{1}{x}right)
end{align*}
$$

Then may you conclude that the required limit is $-5/2$.

By analogy, an approximation such as $color{blue}{x + 1} sim color{red}{x}$ is perfectly fine for computing the ratio limit $lim_{x to -infty} (color{blue}{x + 1}) / (color{red}{x})$. But if you are computing the difference limit, then you may not discard the $1$ in $color{blue}{x + 1}$.

Possibly the biggest problem in your first paragraph is that $sqrt{text{whatever}}$ is non-negative when we are working over the reals. Can you exhibit any choice of $x$ in the real numbers giving a negative value of $sqrt{x^2 + 5 x + 3}$?

I will flesh out much of what would be considered a series of valid steps you may take in order to get something similar to the first step performed in the initial limit calculation. I will note that the second step that was taken is valid because we perform all of the algebra in the limit expression first, then we take a limit of what results.

The following line of reasoning shows that whenever you have $sqrt{x^2+beta x+alpha}$ as a term (added or substracted, not multiplied or divided) in a limit, it may be replaced freely by $|x+frac{beta}{2}|$ inside the limit (of course slight adjustments must be made for it to be shown in general).

Note that for every $b>frac{5}{2}$,

$$lim_{xto-infty}sqrt{x^2+2bx+b^2}+x lelim_{xto-infty}sqrt{x^2+5x+3}+x.$$

To see why this is the case, here is a series of inequalities. First pick a $b>frac{5}{2}$. Then pick $x$ so that $xlefrac{3-b^2}{2b-5}$.

Then
$$(2b-5)xle 3-b^2,$$ $$2bx+b^2le 5x+3,$$
$$x^2+2bx+b^2le x^2+5x+3,$$
$$sqrt{x^2+2bx+b^2}lesqrt{x^2+5x+3},$$
$$sqrt{x^2+2bx+b^2}+xlesqrt{x^2+5x+3}+x,$$
$$lim_{xto-infty}sqrt{x^2+2bx+b^2}+x lelim_{xto-infty}sqrt{x^2+5x+3}+x.$$

The intuition for this comes largely from working backward. So thus the claim is justified.
So now we can say that

$$lim_{bto frac{5}{2}^-}lim_{xto-infty}sqrt{x^2+2bx+b^2}+x lelim_{xto-infty}sqrt{x^2+5x+3}+x,$$
$$lim_{bto frac{5}{2}^-}|x+b|+x=lim_{bto frac{5}{2}^-}-x-b+x=lim_{bto frac{5}{2}^-}-b=-frac{5}{2}lelim_{xto-infty}sqrt{x^2+5x+3}+x.$$

Now since $3<frac{25}{4}$, we have that.

$$lim_{xto-infty}sqrt{x^2+5x+3}+xle lim_{xto-infty}sqrt{x^2+5x+frac{25}{4}}+x=lim_{xto-infty}|x+frac{5}{2}|+x=lim_{xto-infty}-x-frac{5}{2}+x=lim_{xto-infty}-frac{5}{2}=-frac{5}{2}.$$

So $$lim_{xto-infty}sqrt{x^2+5x+3}+x=-frac{5}{2}.$$