The Difference of the Species

[Nukapei]

I've been wondering -- how do people in Middle-earth tell which species a stranger belongs to? I'm speaking mainly of the difference between Elves and Men -- the rest (Dwarves, Hobbits, Ents, Orcs, etc) are pretty easy to tell apart. But what of the children of Ilúvatar? Is it perhaps cultural (Silvan Elves wear these clothes and their hair this way, whereas Men from Gondor this way)? Or physical? Or perhaps something subtle, that someone not used to looking for it would never be able to tell? I know about Elves having ears the shape of leaves, but you can't always see a person's ear -- either distance of hair can obstruct your view! I was just wondering, since the characters in the book seem to be able to tell the difference right away, whereas I don't know if a person from real Earth could.

And, if it's cultural, could that have played a part in Arwen's falling in love with Aragorn (because Galadriel dressed him as an "elf-lord", so if it was cultural, she might have mistaken him for an Elf!)