Complex question, there is no simple answer to that, but I will try to make a long story short (relatively, that is :)). Based on my understanding of literature, factors that affect delinquent behavior (which includes but is not limited to crime) are:
- level of social integration / alienation - stability of community - legitimacy of violent or delinquent behavior - individual stakes in the community - peer network types - their norms values and interaction patterns - situational factors - individual social/coping skills.
Social integration refers to the degree an individual is a part of a larger community (i.e. feels connected to it, is subject to community controls, etc.). Too little of it produces alienation and reduces the stake individual has in the community, and that encourages delinquency in various forms (e.g. egoistic pursuit of self-interest disregarding others, or alienation- or fear- induced aggression). Likewise, too much of it also produces delinquency, albeit of a different kind (e.g. mob violence, authoritarian aggression, suicide).
Another fact is social mobility - transient communities (a lot of people moving in an out) reduces individual stakes in community and reduces social integration, with the effects as described above.
Legitimacy of violent or delinquent act - people do what they see as a legitimate way of accomplishing their goals. Legitimate acts have higher incidence than illegitimate. If, say, wife beating is seen as a legitimate way of resolving domestic disputes, violence against women will be high. Likewise, if aggression is projected as a legitimate way of conflict resolution (e.g. the government waging aggressive wars), you will see increase of aggressive behavior in every day life. Legitimacy can be bestowed either formally (e.g. by government or other public institution endorsement) or informally (as folkways, "this is what everyone else does," "this is our tradition" etc.
Individual stakes in the community - people who have higher stakes in the community i.e. feel a part of it are less likely to commit delinquent act against other members of that community - given of course the perceived legitimacy of an act. Individual stakes, in turn, are affected by mobility (people moving in and out) and general level of social integration.
Peer network - this refers to small group and networks of which individuals are a part withing larger community contexts. Basically, low level of social integration gives more prominence to such networks and groups, especially those that may adopt and legitimize delinquent norms and expectations (e.g. street gangs, fraternities, etc.).
Situational factors refer to the interpersonal interactions types that are likely to lead to delinquent or violent outcomes e.g. contracting sex with a street walker. Competition between sports fan groups, etc.
Individual social and problem solving skills - people with low social skills and low ability to deal with contingencies, ambiguities, uncertainty and unfamiliar people and situations are more prone to authoritarian aggression and delinquent behavior. Those skills are often affected by upbringing and relations to the community - people brought up in authoritarian families and alienated communities may have lower social and coping skills, - however I also think that there is a significant, albeit not yet quantified biological factor (ie. brain chemistry).
I think that the US society scores high on several of these factors:
- relatively low level of social integration, albeit that varies from place to place; what we have is different groups living parallel lives that have little in common;
- relatively high legitimacy of aggressive and violent pursuit of self-interest - both formal (the US government is at nearly permanent aggressive war against someone since WW2) and less formal (Wild west mythology, self-made individualism, etc.)
- high level of social mobility i.e. frequent changes of residence - more frequent than in most other countries, which reduces social integration and individual stakes in community.
- peer network types - a relatively high incidence of various peer networks that legitimize violent and aggressive behavior - these include not just street gangs, but fraternities, lobbying groups, corporate culture etc.
- situational factors are, by definition, situational and difficult to fit in general categories, but the low level of social integration and the emphasis on "market-type transactions" i.e. short term, on the spot, strictly quid pro quo is more likely to produce situations that may encourage delinquent behavior.
- social skills - again difficult to fit into general categories, but dysfunctional families and communities, and low social integration and the culture of self-serving individualism do have a negative effect on the development of such skills.
Since the United States tend to be somewhat higher on these factors than most other developed countries - it also tends to have a somewhat higher level of delinquent behavior.
Whether that delinquent behavior translates into higher incarceration rates, however, is a very different story. The US has a much higher absolute capacity to incarcerate since it controls more resources, but that capacity is used rather selectively. Street crimes are more likely to result in incarceration than white collar crimes. Therefore, comparing incarceration rates or for that matter reported crime rates (as opposed to crime victimization rates) among countries is like comparing apples and oranges. Ideally we should compare victimization rates, but few countries outside the US can afford doing crime victimization studies. Besides, such studies are affected by legitimacy issues e.g. people are less likely to report victimization that is not "legitimate" (e.g. rape in many cultures) or if the violence they suffered is seen as "legitimate" (the "she asked for it" tripe).
So to summarize I would venture to say that:
- US tend to have higher delinquency rates (including but not limited to crime) due to lower levels of social integration, low stakes in community and high legitimation of individualism and self-serving aggression in the US culture
- US has a rather high level of incarceration which is likely a result of not just higher delinquency rates, but the material capacity to incarcerate. Given that capacity and general delinquency level, I think that the incarceration is actually low in the US due to the prosecutorial discretion not to go after white collar crimes. If white collar criminals were prosecuted with the same vigor as street criminals, the US prison population would probably double.
- However, incarceration rates are not the best indicator of delinquency in a comparative perspective, because it is affected by a host of other factors, such as the material capacity to incarcerate, strictness of the law, prosecutorial discretion, the fairness of the judicial system - which may work both ways i.e. a corrupt system may lead either to higher rates or lower rates due to bribery or inefficiency.
Wojtek