Sign of the Shadow

[Elmo]

A sign of the shadow falling on Numenor was when instead of bringing gifts to the men of Middle Earth they became their masters and created an Endorian Empire.

What is the difference between these colonial exploits and Gondor's up to the mid-point of the third age? When Gondor conquered great swathes of Harad and Rhun. Some might say that the 'wild men' attacked Gondor unlike Numenor first so it was just retaliation but when you conquer a people you just create more enmity and create more enemies. People who might not be so usaully so inclined, might join the cause against Gondor, to drive the foreign imperialists out of their land.

What I don't get is how why, when Numenor conquers foreign lands, its the sign of the shadow, when Gondor does it, there seems to be no reprimand from the author. Is this unintentional and Gondor's foreign conquest show that the Land of Stone has indeed fallen under the shadow in some way. If Gondor had kept getting stronger and there was no decay would it, to paraphrase Melkor, had the whole world for its belly?

(In the case of the Rohirrim and the northmen, weren't they distant kin to the Gondorians and that is the reason of the friendship. Maybe because the folk of Harad and Rhun were fair game because they weren't)