The fun part is that other languages have, in the past, enforced the distinction with better rigour.
Classical Greek, for example, distinguishes between ανθροπος, “man” in the sense of “human being”, and ανηρ, “man” in the sense of “male person”. Likewise, Latin distinguishes between homo, “man” in the sense of “human being”, and vir, “man” in the sense of “male person”.
English not only became lazy with respect to declension over time, but with respect to other linguistic constructs as well.