Well, Kinsey's research into male sexuality suggests otherwise: 46% of males reacted sexually to both sexes, and 10% were exclusively homosexual. That's a majority. Given the extent of official homophobia at the time, it may be that a large number of the remainder simply repressed or disavowed any sexual response to other men.
As for why certain desires are considered perverse, I still think you're barking up the wrong tree. Homosexuality obviously wasn't considered an illness according to the DSM classifications because of what the majority could or could not imagine feeling, if Kinsey's findings were accurate. The roots of such contemporary categories are surely to be found in doctrines of 'hygeine' arising from 18th and 19th Century medicine, the biologising of what had previously been seen as 'moral' states and their reinterpretation as infections; the development of race theory, the accounts of masculinity that arose within it, and concepts of race 'health'; and the gendered identities produced through colonial domination.