I also found those figures surprisingly low when I first read them, but then I thought they may be not that much off. There might be pockets of sexually hyperactive groups, but they are small and thus do not weigh much in calculating the mean for the entire nation. Second is the question of definition - many people do not consider oral or commercial sex as having a "sex partner." I have a vague recollection that the authors mentioned that problem in their methodological write-up but I am not sure what they actually did about it (my copy of the book is missing, so I cannot easily check it). I also suspect that the study suffered from the "sin and motherhood" syndrome (i.e. Rs giving conventionally expected answers to survey questions addressing controversial, problematic or personal issues).
But the bottom line is that if we discount commercial sex, the number of "legitimate" sex partners for most folks is not that great for a very simple reason - outside colleges and bohemian circles, social life in the US is not very intensive by international standards, and confined for the most part to formal settings (work) Assuming that the average US-er works from 9 to 5 and then commutes back home for an hour -- he gets home around 6 o'clock which does not leave him much time for socializing and, by implication, for having sexual encounters except commercially arranged quickies.
Wojtek