<< Does anyone have any concrete explanation for why the
U.S. continually shoots itself in the foot supporting
an ally that seems to serve no strategic purpose? >>
There are about two million Jews in the US, almost all of whom vote and many of whom contribute; they are a lot less rabidly Zionist than the Israel lobby--polls showed that there was majority support for a two-state solution long before that was speakable in public--but, as with the gun lobby here, there is a substantial minority of hard-core militantly pro-Zionist characters who are extremely well organized. The milder and less reactionary majority cares less and isn't organized. You know the old slogan, Money talks, but Organization Delivers?
In "national security" terms, Israel operates as a rather unruly and hard-to-rein in proxy for the US, formerly against the USSR, and still against radical Arab nationalism--it draws lightening and attention from what the US and the rest of the West is doing there (extracting cheap oil) while serving as a sort of regional cop. Noam Chomsky' The Fateful Triangle is very good on this. I heard there was an updated edition a few years ago.
--jks