Church, Lies, and Polling Data
by Andrew Walsh
Americans are a religious people who attend worship services two, three, and even four times more frequently than Western Europeans. That's been an article of sociological and journalistic faith for 50 years.
It has been built on a mighty foundation of polling data. Since the late 1930s, the Gallup Organization has been asking pollees if they "happened to attend" church or synagogue in the past seven days. Invariably, about 40 percent respond that they have done so. Long running surveys like the General Social Survey of the National Opinion Research Center at the University of Chicago, the Harris polls, and the polling of the Barna Research Group in California have tended to support the 40 percent figure.
This number is so commonplace that when polling data on church attendance is released, as it is several times each year, American journalists usually relegate it to news notes or use it as a springboard to other stories. "Nearly all surveys of American churchgoing habits show that roughly 40 percent of Americans attend church once a week," Ari Goldman reported in a typical piece, a 1991 New York Times story that went on to show that the growth of some churches occurs at the cost of others.
But there's a problem with the 40 percent mantra: It may grossly exaggerate how often Americans actually attend religious worship. For much of the 1990s, contending groups of sociologists have competed, usually outside the view of journalists, to revise and improve their measurements. And many of them have concluded that the "fact" of high church attendance is built on a foundation of lies-or, more generously, "overrepresentations."
The underlying issue is that for the past half-century Gallup and other polling enterprises have relied on self-reporting to determine attendance at worship. Unfortunately, "there's a well-known tendency for individuals in self-report surveys to exaggerate what they perceive to be socially desirable behavior," Mark Chaves, now at the University of Arizona, told Richard Chapman of the Chicago Sun-Times in December of 1994. No behavior is more "socially desirable" than church attendance, but pollsters have rarely addressed this shortcoming in their press releases.
In 1993, Chaves, Kirk Hadaway of the United Church of Christ, and Penny Marler of Samford University ignited the debate about "overrepresentation" by reporting the results of a study of church attendance by Protestants in Ashtabula County, Ohio, and in 18 Roman Catholic dioceses around the country. Instead of using telephone polling, the researchers counted heads at services and in parking lots, and checked with pastors. They then estimated that 20 percent rather than 40 percent of Protestants, and 28 percent rather than 50 percent of Catholics, attend church weekly. The study, "What the Polls Don't Show: A Closer Look at U.S. Church Attendance," appeared in the American Sociological Review.
Scholars whose work has relied on the polling data were particularly unhappy with the challenge. The Rev. Andrew Greeley-Catholic priest, novelist, columnist, and sociologist-was incensed by the study, denouncing it in a Religion News Service column as "a sloppy piece of work" that naively extrapolated regional findings into national ones. Hadaway, Chaves, and Marler responded that they weren't attacking the validity of poll data, merely pointing out its sharp limitations. "Americans misreport how often they vote, how much they give to charity, and how frequently they use illegal drugs. People are not entirely accurate in their self-reports about other areas as well," Hadaway wrote in the magazine Christian Century. "Males exaggerate their number of sexual partners, university workers are not very honest about reporting how many photocopies they make. Actual attendance at museums, symphonies and operas does not match survey results. We should not expect religious behavior to be immune to such misreporting."
After the first wave of scholarly discussion, Hadaway et al. returned to Ashtabula County to measure Catholic attendance. They counted heads at all of the regularly scheduled masses in the county-38 in 13 parishes-over a several-month period. Based on the count, they projected an average weekly attendance of 24 percent of the Catholic population (a figure not far out of line with numbers reported by many Midwestern Catholic dioceses based on their own head counts). They then polled a scientifically valid sample of Ashtabula County residents by telephone. Fifty-one percent of Roman Catholic respondents said they had attended church during the past week.
Thus the "overstatement gap" snapped into focus. In the United States, the difference between attendance levels of 20 and 40 percent is immense-a swing of at least 50 million people. Institutional religion, far from being stable and vital in the United States, might well be weakening under the cover of misleading poll data. Any way one looks at it, there was a substantial religion news story to cover.
And in the fall of 1993, the academic dispute over levels of church attendance did gain some coverage in the American newspapers. "A high-level debate, spiced with a dash of bare-knuckle language, has erupted in some academic circles over claims by three scholars that church attendance in the United States, as established by decades of telephone polls, has been heavily inflated," reported a September 18, 1993 story in the Los Angeles Times. "If the revised rate proves true, it would have significant implications for assumptions about American religious practice that have long undergirded articles in popular and academic journals."
Reporting that "academic sniping" had broken out, the Atlanta Constitution quoted Greeley attacking the study and another sociologist, Jay Demerath of the University of Massachusetts, praising it for bringing out the fact that Americans inflate their religiosity in traditional polls. "Gallup and other pollsters are aware of this," Demerath said. "It's kind of a dirty little secret."
But despite the potential significance of the dispute over church attendance, few news organizations covered the story vigorously. Most of the published reports were drawn from wire services. Only a few bylined stories discussed the issue in 1993 and 1994.
During the middle of the decade, journalists did give increasing attention to pollsters whose analyses of church attendance were more complex than Gallup's-especially to the work of George Barna, an outspoken evangelical Protestant. Coverage of Barna's more volatile and nuanced surveys increased dramatically in this period, although journalists still ignored the overrepresentation issue. The Barna polls tended to portray falling rates of church attendance (although from levels closer to 50 percent in the early 1980s to the middle-high 30 percent range).
More concretely, Barna suggested that significant changes in belief and practice had been obscured by "macro-level" statistics such as Gallup's. In 1996 and 1997, the Orlando Sentinel, the Los Angeles Times, the Kansas City Star, the Washington Post, the San Diego Union, the Chicago Sun-Times, the Denver Post, and the Houston Chronicle all carried substantial Barna-based stories suggesting declines in religious practice, significant regional variations in religious practice, a slowdown in the growth of evangelical Protestantism, and shifting popular understanding of the religious content of many questions pollsters ask.
In 1997, Bill Broadway of the Washington Post quoted Barna as saying that Gallup's statistics are valid but fail to "show the subcurrents of change." For example, Barna said he believed that in 1947, the date of a landmark Gallup poll, "the vast majority of people believed in a God described in the Bible." By the 1990s, he claimed, deeper probing revealed that one third of those who tell pollsters they believe in God do not believe in the Biblical God, embracing instead a 'higher consciousness,' or a sense of the divine derived from Eastern religions, or simply 'many gods.'"
Meanwhile, beneath the radar of most journalists, the sociological mill was grinding away. Researchers at the University of Michigan's Institute for Social Research and others began to explore the claims made by Chaves, Hadaway, and Marler. In general, these studies found strong support for the "overrepresentation thesis."
Follow-up studies have usually focused on who is overrepresenting their church attendance to pollsters. Perhaps counter-intuitively, scholars now suggest that it is the most committed believers who overstate their attendance, not those who seldom or never attend services.
Some poll respondents stretch the boundaries of truth because they feel that they usually attend weekly worship, even if they didn't happen to do so the week the pollster called. "Follow-up questions about what people meant by 'attending church' revealed that a few were counting things other than attending worship-such as going to weddings, funerals, committee meetings, Sunday school and choir practice," Hadaway wrote in the Christian Century. "One individual in Ashtabula County even said his attendance consisted of mowing the church lawn on the previous Saturday."
Many of those who overreport, it seems, attend church once or twice a month but don't rate themselves as less committed. "Regular church attendance is increasingly difficult, even for those committed to it," Hadaway wrote. "Sunday morning is no longer 'sacred' time: job responsibilities, sports leagues, family outings, housework, and many other things get in the way of traveling to a church building for worship at a scheduled time."
After several quiet years in the media, the debate about church attendance resurfaced this year. In February, Marler, Hadaway, and Chaves published an aggressive article, "Overreporting Church Attendance in America: Evidence that Demands the Same Verdict," in the American Sociological Review. It asserted the nationwide validity of their estimate of average weekly church attendance at 20 percent. "We believe that too much trust has been placed in survey data and not enough attention given to membership records, patterns of giving, and even the incredulity of local church pastors when they hear that 40 percent of Americans attend church during an average week," Hadaway told the Minneapolis Star Tribune.
Other scholars are still not persuaded. "There's a claim that surveys lead to overreporting of church attendance, which seems to be correct. The question is by how much," Thomas Smith of the National Opinion Research Center told David Briggs of the Cleveland Plain Dealer in May. "We haven't nailed down how much Americans exaggerate."
February's forum in the American Sociological Review included reactions from many prominent sociologists of religion, most of whom expressed skepticis about the 20 percent figure and the research methods used. Theodore Caplow of the University of Virginia, for example, said he thinks the "overrepresentation" thesis is "not proven."
As estimates proliferate, there does seem to be "more consensus around a figure of 30 percent than there is on a figure of 20 percent," Smith told the Plain Dealer. (This would project weekly attendance at worship at about 75 million nationwide.)
In May, two new voices entered the debate with new data and a new methodology. A Washington Post article by Richard Morin reported that research by Stanley Presser of the University of Maryland and Linda Stinson of the U.S. Bureau of Labor Statistics indicated that church attendance has been falling significantly and that the number of Americans who lie about their church-going habits is increasing.
Working with three sets of time-use diaries produced by Americans participating in social scientific research projects in the mid-1960s, 1970s, and 1990s, Stimson and Presser "determined that the percentage of Americans who attended church the previous week plummeted from 42 percent in 1965 to 26 percent in 1994." They argue that their study of time-use diaries avoids many of the problems of respondent "social desirability bias" because "those in the diary study were asked only to account for how they spent their time, and not whether they went to church." Indeed, the 1992-94 diaries were produced in a U.S. Environmental Protection Agency-sponsored study designed to determine how often individuals were exposed to harmful substances.
The Spring 1998 debate triggered substantial articles in, among others, the Plain Dealer, the Dallas Morning News, the Hartford Courant, and the Christian Science Monitor, but few journalists explored the local angles of the story or considered the implications of a substantially diminished and diminishing sector of institutional religion.
Despite the complexities of the discussion, it does matter whether weekly attendance at worship is 20 or 30 or 40 percent of the population. Here are several of the questions that need to be sorted out.
If attendance is at the 20 percent level, we are due for a thorough reassessment of long held views about the strength of organized religion in the United States. If the emperor has no clothes, or is more scantily clad than thought heretofore, that's not only important to know for its own sake but may also be a sign that many other perceptions are distorted by an exaggerated emphasis on polling data.
If attendance turns out to be 30 percent or more, the entire debate may be a tempest in an academic teapot. But even such a figure would call for vigorous efforts to determine whether participation rates are changing or whether we have simply developed more accurate measures.
One emerging school of thought holds that much of the aggregate decrease in participation rates may be caused largely by changes in the practices of Catholics, who make up the largest American religious group. There's considerable evidence that Catholic rates of participation have been moving rapidly downward toward Protestant norms. If so, the "Protestantization" of American Catholicism since the mid-1960s could be the most undercovered religion story around.
Stories about church attendance ultimately relate to how ordinary Americans spend their time and energy-a major preoccupation of American journalists in recent years. If the new studies prove correct, they might indicate that committed church members feel that it's impossible to attend services weekly even though they would like to. That would suggest an attenuation of institutional commitment that aligns in an interesting way with patterns of declining voter participation and less activity in other realms of the voluntary sector.
The persistence of claims of 40 percent or higher church-attendance rates may suggest that even committed believers are drifting off into a private or detached realm of religious observance. But if so, why do Americans continue to tell pollsters their attendance levels are high when direct social pressures to conform and attend worship are far weaker in most parts of the country than they used to be? Is this insistence on exaggerating participation a sign of the strength or weakness of religion in America?
However and whenever these and related questions are resolved, journalists would do well to break one habit right away. To this day, many news outlets continue to report the latest release of polling data with no reference to the dispute over its value. It's almost as if reporters aren't reading their own clip files.
On May 31, 1997, for example, the Washington Post carried a story by religion writer Bill Broadway under the headline: "Poll Finds America 'as Churched as Ever'; Belief in God, Afterlife Have Changed Little Since 1947, but Faithful Sample More Forms of Spirituality." The story quoted George H. Gallup, Jr. of the Princeton Religion Research Center saying that church attendance was stable "despite the church-hopping and spiritual experimentation common among American faithful."
Unqualified Gallup poll data was still cited high up in arguments about the persistent and even increasing role of religion in American life. "Religion is breaking out all over," Bill Moyers wrote in USA TODAY on October 13, 1996. "Public confidence in both organized religion and the clergy has been renewed. Church attendance has held steady and in some instances-among teens for example-is up slightly."
On June 28, 1997, a story reported by Dan Lothian on NBC's Nightly News carried the headline: "Nationwide Increase in Church Attendance Creates a Building Boom for Contractors and Banks." In this case, the significant indicators of growth were bank loans and construction contracts for new synagogues, Catholic churches, and (especially) suburban megachurches. Charles Arn, identified simply as "a church growth researcher," asserted: "Across the country, we are seeing an increase in worship attendance."
Don't bet on it.
Andrew Walsh is managing editor of Religion in the News and visiting assistant professor of religion at Trinity College.
Center for the Study of Religion in Public Life, Trinity College, Hartford, CT
<http://www.trincoll.edu/depts/csrpl/RIN%20Vol.1No.2/Church_lies_polli ng.htm> *****
Yoshie