Gandalf before the War of the Ring

[Imbefana]

Just as the title says, I was wondering what Gandalf did exactly between 1000 T.A. when he arrived in the Grey Havens and the War of the Ring. In the Silmarillion it says

Long they journeyed far and wide among Elves and Men... Mithrandir was closest in counsel with Elrond and the Elves. He wandered far in the North and West and made never in any land any lasting abode...

I know he also drove Sauron out of Dol Guldur (twice) and was a member in the White Council, I suppose I'm just wondering why he didn't play a bigger part in the world of Men when he seemed to do so extensively right before and during the War of the Ring.

For example, was Aragorn the only heir of Isildur that befriended the wizard? Did one of the other Cheiftains of the Dunedain have that sort of companionship with him, or was Aragorn a special case?

I was just thinking how much Gandalf affected (in a good way of course) the lives of Men and Hobbits in the trilogy, and it seems odd he didn’t do so when he first came to Middle Earth. What do you guys think?