Scholarly Debate Over Guns and Crime Rekindles as States Debate Legalization By DAVID GLENN
As state legislatures in Ohio and Missouri prepare to vote this month on legalizing the carrying of concealed weapons, an exceptionally heated scholarly debate has erupted over the relationship between crime rates and handgun laws. In the April issue of the Stanford Law Review, which was published on Friday, two law professors assert that a certain body of gun-policy research, which has frequently been cited by gun-legalization proponents, is marred by a pattern of careless statistical coding errors.
The two legal scholars, Ian Ayres of Yale University and John J. Donohue III of Stanford University, accuse their opponents of "repeatedly bringing erroneous data into the public debate."
"There are many factors that people might weigh in deciding whether to pass these laws or not," says Mr. Donohue. "But this idea that there is a statistical basis for thinking that these laws will reduce crime simply is not true. It's appalling to me that some legislators are voting for these laws thinking that that is true, because it just isn't so."
Mr. Donohue's opponents strongly defend their research, and deny the allegations of carelessness. In an essay that also appears in the law review, Florenz Plassmann, an assistant professor of economics at the State University of New York at Binghamton, and John Whitley, a lecturer in economics at the University of Adelaide, in Australia, charge that Mr. Ayres and Mr. Donohue "have simply misread their own results."
Mr. Plassmann and Mr. Whitley's paper was originally co-written by John R. Lott Jr., a senior research scholar at the American Enterprise Institute. Mr. Lott withdrew his name from the paper on April 12 because, he says, the law-review editors were unfairly permitting Mr. Ayres and Mr. Donohue to make a minor late correction to their original manuscript, while not offering him and his colleagues similar latitude.
Mr. Ayres and Mr. Donohue have suggested, in the pages of the law review and elsewhere, that Mr. Lott may have withdrawn his name because of embarrassment over the alleged statistical errors.
Mr. Lott strenuously denies this. The only issue, he says, was the dispute over the fairness of the law review's policy. "We had an agreement with the students that was very clear," he says. "I wasn't going to be a part of it, even if my young co-authors wanted to get the publication" for their CV's. Mr. Lott provided The Chronicle with an e-mail exchange he had with the editors of the law review, which tends to support his account.
Mr. Lott also points out that because the claim of coding errors appears in a law review, it has not been subject to review by third-party scholars, as would have been the case in a peer-reviewed economics journal.
The scholarly dispute concerns the "more guns, less crime" thesis -- so named after a 1998 book of that title by Mr. Lott. In that book and in subsequent articles, Mr. Lott and his co-authors have argued that states that permit citizens to carry concealed handguns have lower rates of violent crime, in part because potential criminals are deterred by the fear of armed resistance.
That body of research, which relies on complex statistical analyses designed to tease out the effects of handgun laws from the many other factors that cause crime rates to vary across states, has often been cited by gun-rights proponents in state legislatures.
In the years since Mr. Lott's first publication, at least six scholars have published studies that tend to confirm his findings, while at least four other studies have tended to cast doubt on his findings. Mr. Donohue noted in an interview that Mr. Lott's research has convinced his peers of at least one point: No scholars now claim that legalizing concealed weapons causes a major increase in crime. Even Mr. Donohue's analysis, which is highly critical of Mr. Lott's, finds only "modest pernicious effects," in his words.
Mr. Lott's 1997 paper on gun policy was, "to that point, the most important piece of empirical research that has ever been done in the social sciences," says Jeffrey S. Parker, a professor of law at George Mason University. "I doubt that even Ayres and Donohue would dispute that point."
Mr. Ayres and Mr. Donohue's critique of Mr. Lott's scholarship runs as follows: The models used by Mr. Lott and his co-authors have not taken sufficient account of the broad differences between states that permit the concealed carrying of guns and those that do not.
In particular, Mr. Ayres and Mr. Donohue suggest that the spike in murders associated with the crack-cocaine epidemic of the late 1980s was concentrated in urban areas in states with restrictive gun laws, while states that permitted people to carry concealed weapons in the 1980s tended to be relatively rural and unaffected by drug violence. That imbalance, Mr. Ayres and Mr. Donohue say, has given the right-to-carry states an artificial boost in studies by Mr. Lott and his allies.
In their reply, Mr. Plassmann and Mr. Whitley argue that their opponents' own data, when properly read, demonstrate immediate state-level benefits from the legalization of concealed weapons. They also present new county-level data for the period 1977-2000, which they say further supports the more-guns, less-crime thesis, whether one uses their opponents' preferred statistical techniques or their own.
It is here, in this new 1977-2000 data set, that Mr. Ayres and Mr. Donohue claim to have identified a serious set of coding errors. Mr. Plassmann and Mr. Whitley failed to assign dummy variables (which researchers use as place holders, to stand for meaningful variables that they may have neglected to identify) for the states of Alaska, Massachusetts, and Pennsylvania for certain years in their calculations. Correcting those errors, Mr. Ayres and Mr. Donohue write, "completely reverse[s]" the paper's conclusions and "restore[s] the conclusion that concealed-carry laws were associated with increases in crime (or no effect) for all crime categories."
Mr. Ayres and Mr. Donohue liken this incident to a similar recent dispute, in which David B. Mustard, an assistant professor of economics at the University of Georgia who collaborated with Mr. Lott on his earliest study of gun policy, was asked to revise his contribution to a Brookings Institution forum on handguns after the discovery of alleged coding errors. "For the second time," they write, "Lott and co-authors have put into the public domain flawed regression results that happen to support their thesis, even though their results disappear when corrected."
"There's really nothing they can say," Mr. Donohue says of his opponents. "It's sort of like we caught them failing to carry the one."
Mr. Lott replies that the alleged coding errors are irrelevant to the larger debate. "Whether one believes the regressions in the Plassmann and Whitley piece or not, just looking at Ayres and Donohue's own results -- you can't look at the graphs that Plassmann and Whitley have of Ayres and Donohue's results and not see a significant drop in violent crime."
"The basic results are not fragile," Mr. Whitley writes in an e-mail message. "Minor errors in coding would not undermine them (and an entire literature)." Mr. Whitley says that he could not reply to the charges in detail because he had not yet had time to carefully review Mr. Ayres and Mr. Donohue's comments. Because the allegations appear in a law review rather than a peer-reviewed academic journal, no third-party scholars have reviewed the claim of coding errors.
Six tables that derive from the same allegedly miscoded data set appear in Mr. Lott's new book, The Bias Against Guns: Why Almost Everything You've Heard About Gun Control Is Wrong (Regnery, 2003). James Lindgren, a professor of law at Northwestern University, says, "If Donohue and Ayres's account is as it appears -- and I'm not in a position to judge that -- then Lott should withdraw the book for revision."
Mr. Lindgren adds that he believes it extremely unlikely that any coding errors were the result of a conscious intent to distort the study's findings. He notes that Mr. Lott has not only shared his data sets with other scholars, but has made them generally available to the public on his Web site <http://www.johnlott.org/>. "You tend not to do that if you've intentionally miscoded your variables," he says.
In 2004, a panel of the National Academy of Sciences is scheduled to release a lengthy evaluation of the strengths and weaknesses of econometric studies of handgun laws and crime rates, including Mr. Lott's.
The Stanford Law Review article <http://lawreview.stanford.edu/content/vol55/4/Ayres_Donohue_article.pdf> by Mr. Ayres and Mr. Donohue, the response <http://lawreview.stanford.edu/content/vol55/4/Plassmann_Whitley.pdf> by Mr. Plassmann and Mr. Whitley, and a rebuttal <http://lawreview.stanford.edu/content/vol55/4/Ayres_Donohue_comment.pdf> by Mr. Ayres and Mr. Donohue are available at the law review's Web site.
-----
Chronicle of Higher Education - February 14, 2003
Scholar's Most Vigorous Defender Turns Out to Be Himself, Pseudonymously By DAVID GLENN
EVERYONE NEEDS A BEST FRIEND: "Last year I worked extremely hard," John R. Lott Jr. wrote last month in an e-mail reply to a review of his research ethics. Among other things, Mr. Lott, a resident scholar at the American Enterprise Institute for Public Policy Research, was busy completing a new book, The Bias Against Guns, due from Regnery Publishing in March. The hectic schedule had kept him from many things, he said, including keeping up on Internet debates on gun control. "I am not a member of the firearms discussion groups and I have not been following them."
Not quite true. Last year, the author, who is best known for the book More Guns, Less Crime (University of Chicago Press, 1998), did find time to be a regular voice in gun-control debates in such Usenet groups as talk.politics.guns, alt.crime, and misc.fitness.weights.
But he didn't do any of this under his own name. Using the pseudonym "Mary Rosh" -- derived from the names of his sons Maxim, Ryan, Roger, and Sherwin -- he vigorously defended the integrity and scholarship of John R. Lott Jr.
"He has probably published more research in refereed journals than almost anyone his age," proclaimed Ms. Rosh in May 2002. Six months earlier she wrote, "I had him for a Ph.D.-level empirical methods class when he taught at the Wharton School at the University of Pennsylvania back in the early 1990s, well before he gained national attention, and I have to say that he was the best professor that I ever had."
Mr. Lott's deception was uncovered in late January by Julian Sanchez, a staff writer at the Cato Institute who maintains a Weblog on libertarian politics. Mr. Sanchez noticed parallels between Ms. Rosh's language and Mr. Lott's, and then scrutinized the Internet-protocol codes attached to their e-mail addresses.
Mr. Lott seems to have used the pseudonym primarily as a means to let off steam. ("YOU ARE AMAZINGLY DISHONEST," Ms. Rosh wrote to a critic of Mr. Lott in 2001. "HAVE YOU ABSOLUTELY NO SHAME?") In an e-mail message to the Chronicle, Mr. Lott says, "I used my own name for a while, but many times it was difficult to get past who I was and to discuss the issues."
On at least one occasion, however, Ms. Rosh appears to have been employed to do a bit of espionage. On November 1, 2002, someone known as "Alpha Male" wrote on the Usenet group talk.politics.guns: "I wouldn't 'do' anything to John Lott besides reject him for publication, which I did. The reason I had to do that is not because I have a thing against guns. ... John Lott's research design did not support his wild claims."
Three days later, Ms. Rosh replied: "Are you saying that you're an academic and that you have actually refereed a paper of Lott's that was submitted to a journal? Was it a real academic journal? I have some familiarity with journals. Which one was it and what piece of his did you referee?" Mr. Lott says that those questions were "purely rhetorical" and that he was simply trying to taunt "Alpha Male" into confessing that he is not an academic and had never actually reviewed any of Mr. Lott's papers. ("Alpha Male" did not return an e-mail asking for comment.)
***
All of this was unearthed in the context of much more serious, but unproved, allegations about Mr. Lott's research. Otis Dudley Duncan, a professor emeritus of sociology at the University of California at Santa Barbara, has long questioned a statement on Page 3 of More Guns, Less Crime: "98 percent of the time that people use guns defensively, they merely have to brandish a weapon to break off an attack."
Most studies have found that people who brandish their weapons in self-defense actually fire them more than 20 percent of the time, not 2 percent. For two years, in op-ed essays and other publications, Mr. Lott attributed his 98-percent figure to "polls by the Los Angeles Times, Gallup, and Peter Hart Research Associates." But those polls asked only about how often people have "used" their guns defensively, and do not reveal anything about how often guns are actually fired during such defensive uses. (And in any case, the polls don't contain 98-percent figures of any sort.) In 1999, however, he told Mr. Duncan that the 98-percent figure actually came from his own research -- from a telephone survey he conducted in 1997. The second edition of Mr. Lott's book, published in 2000, amends the sentence so that it begins: "If a national survey that I conducted is correct ..."
Asked to produce data from that 1997 survey, Mr. Lott has told his critics that all traces of it were lost when his hard drive crashed that summer. His former colleague David B. Mustard, now an associate professor of economics at the University of Georgia, confirms that many of their data were lost in that hard-drive crash.
James Lindgren, a law professor at Northwestern University, decided last September to settle the question of the survey's existence once and for all. Mr. Lindgren, who has also been one of the most prominent critics of Michael Bellesiles, the former Emory University historian who resigned in October in the wake of allegations that he had committed fraud in his book Arming America, conducted an audit of the dispute and released a report in mid-January, several days before the "Mary Rosh" revelations.
Mr. Lindgren's conclusion: "I am not confident at all, one way or the other, whether the 1997 study was ever done," he says. "Lott has changed his account of what he told me [in an initial September conversation] in several respects, which I find troubling."
"While I recognized that it is extremely easy to lose data in a computer crash," Mr. Lindgren's report begins, "I had not anticipated that Lott would claim to have done a large national survey without discussing the sampling design with anyone, leaving any financial or other records of the study, or remembering anyone who had worked on it."
Mr. Lindgren hopes that the students who Mr. Lott claims worked on the study -- undergraduates at the University of Chicago, where he then taught -- will come forward to confirm his tale. (Mr. Lott has placed an advertisement in Chicago's alumni magazine asking those former students, whose names he says he forgets, to contact him.)
Mr. Lott says he has since replicated his 1997 survey and found similar results. The later survey will be discussed in his forthcoming book.
As for Ms. Rosh, Mr. Lott has now retired her -- but her career will live forever in Google's Usenet archives. Beyond the gun-control arena, she took part in arguments about television networks' early decision to declare Al Gore the winner of Florida's electoral votes -- a decision that Mr. Lott has publicly condemned. In a July 2001 exchange with someone identified as James Simpson, Ms. Rosh denounced a Florida analysis conducted by Philip Klinkner, of Hamilton College. "You are extremely emotional on the issue of Lott and Klinkner and have made all sorts of charges," she wrote. "Why are you so emotional? Are you Klinkner?"