The “Setting” here that appears to independently move your text around at will is the Paragraph Composer. The Paragraph Composer is designed to calculate the optimal line breaking points all over a single paragraph, taking numerous factors into account — just to mention a few: tracking, number and position of word breaks, the optimal/desired word and letter spacing as defined in Spacing, and the deviations in space size between successive lines. For left-aligned text, such as yours here, it avoids a large gap at the end immediately followed by a very small one — it evens that out as well.
So when you think “All I have to do is move this one single word one line further to git it perfectly right”, InDesign does its calculations and notices that would make the spacing on that line visibly different from the lines before and after, and it re-calculates the entire paragraph to even things out again, leaving you to wonder what you should do to “git it right”.
Three things will help you here.
1. Don't think you know better than InDesign … Its spacing is “mathematically perfect”. Manual intervention on a single line will make it less than mathematically perfect, so ID will shrug, and adjust anyway.
2. If you do want some words to stay together, use No Break to just keep those two words together — nothing more, nothing less. ID will shuffle the paragraph around until it finds a way to make it so.
3. To gain or loose a single line in a paragraph, don't try to mess with individual “lines” (which you got cornered by). Select the entire paragraph and apply your tracking to it, in very small increments until it fits. Your suggested tracking of -20 and more makes me shiver — surely that will be visible in your output!? I never go beyond +/-10 (unless there is a real emergency going on, such as having to copy-fit a single page in between otherwise finished pages — and then I sulk the rest of the day).
Is that all you can do? No. The power and independence of the Paragraph Composer are impressive, but if you really like to micro-manage your text, and you don't mind sub-optimal linebreaks and spacing, you can always switch back to the Single Line Composer. This, in essence, behaves exactly like paragraph filling has been doing since WordPerfect 3.0 — and Quark XPress as well –, it fills up a line until it's stuffed full, then goes on to the next one.
For justified paragraphs I would strongly advise against the Single Line Composer (you can compare the results for yourself), but I must admit for left-aligned text you could try and see whether you can spot the quality difference between this and the Paragraph Composer.
To find how to use the Single Line Composer, look it up in the Online Help in “Text Composition”.