News Balance Rhetoric: The Fraser Institute's Political Appropriation of Content Analysis

Robert A. Hackett (Simon Fraser University)

William O. Gilsdorf (Concordia University)

Philip Savage (Canadian Broadcasting Corporation)

Abstract: The Fraser Institute's bulletin On Balance, which offers monthly content analyses of news, is critically examined. Methodological, conceptual and epistemological problems are discussed, as well as a rhetorical mode of reporting results which implies a politicized interpretation of news.

Résumé: On fait la critique du bulletin de l'Institut Fraser, On Balance, qui offre des analyses mensuelles du contenu des nouvelles. On discute les problémes de méthodologie, de conceptualisation, et d'épistémologie, et même que le mode rhétorique de rapporter les résultats qui implique une interprétation politique des nouvelles.


Increasingly, the mass media are recognized not simply as observers and reflectors of political life, but as themselves political players and definers of reality. It is not surprising then, that political advocacy groups are devoting increasingly more attention and resources not only to conveying their messages through the media, but to scrutinizing and pressuring the media directly. Such organizations hope thereby to gain more access and public profile, delegitimize opponents, displace rivals, and/or shift the very terms of debate. Indeed, in the U.S. at least, some advocacy groups (such as the right-wing Accuracy in Media, or AIM, and the left-liberal Fairness and Accuracy in Reporting, or FAIR) have arisen specifically for this purpose. Advocacy groups attempting to influence U.S. network television programming have had differing degrees of success, varying with the goals and strategies of each group in relation to the imperatives of the network TV system, as well as external factors beyond each group's control -- changes in the political climate, shifts in television industry practices and priorities. Advocacy groups have had at least some impact on prime-time programming (Montgomery, 1989, p. 217). In that respect, it is unfortunate that media scholars have paid relatively little attention to the claims, strategies and effectiveness of pressure groups which make the media a primary focus of their activities.

The tactics employed by such groups have ranged from letter-writing campaigns and public criticism through regular newsletters and other publications, to more "hard ball" tactics, such as advertiser boycotts, law suits, appeals to government authorities including broadcasting regulatory agencies and even (in the case of wealthy pressure groups) attempted ownership takeovers (see, e.g., Montgomery, 1989, p. 10). Amongst the repertoire of such tactics is one less drastic and probably more typical -- the ongoing monitoring, analyzing and publicizing of media performance. This article concerns one such monitoring effort -- the monthly On Balance newsletter (hereafter cited asOB) produced by the National Media Archive, a division of the Fraser Institute.

Our analysis is based upon a reading of the first fourteen issues of OB, published between October 1988 and January 1990. Our critique focusses first on OB's understanding and application of its preferred methodological technique, content analysis. We argue not only that there are specific problems in application, but more importantly, OB makes claims for the straightforward objectivity of content analysis which cannot in principle be sustained; that is because the technique is dependent upon the construction of categories, which is necessarily an interpretive process. We then identify some fundamental conceptual and epistemological problems of the OB research. In particular, OB seems to share with some other media research an uncritical acceptance of the problematic notion of "balance," a positivist assumption that the news can straightforwardly reflect the "real," and a tendency to analyze the meanings and "content" of news without examining or theorizing the conditions of its production and reception. Finally, we examine OB's rhetorical mode of reporting results, a mode which sometimes appears to lay an inferential basis for a politicized interpretation of the news, one which often accords with the Fraser Institute's own ideological perspective.

Background: The Origins of On Balance

OB owes its existence to the Fraser Institute, a "think tank" founded in 1975 by right-wing economist Michael Walker, with backing from the giant forestry corporation, MacMillan Bloedel. Headquartered in Vancouver, the Institute had by the mid-1980s developed an annual budget of over $1 million, and a staff of eighteen, thanks to the backing of over 400 corporations and prominent Canadian conservatives like Peter Pocklington and Conrad Black (Fraser Institute, 1989). The Institute is openly pro-free enterprise and generally critical of trade unions and government economic intervention. In the words of its own annual report (1989, p. 8), its role is to "redirect attention to the use of competitive markets as the best mechanism for responding to change."

OB itself is produced by the National Media Archive (NMA), a division of the Fraser Institute founded in 1987 with a guaranteed five years of funding. With an annual budget of $200,000 from the Institute and from clients who subscribe to the newsletter, the NMA videorecords and transcribes major news and public affairs programs from the two main national English-language TV networks, CBC and CTV. In that respect, the NMA offers a valuable research service, not readily available elsewhere in Canada. On the other hand, unlike public archives, the NMA does not attempt to maintain professional archival standards of indexing and preservation. Instead, the NMA has made a name for itself primarily by conducting its own in-house analyses, and publicizing them through OB, which employs a popularly accessible writing style and is widely distributed to members of Canada's journalistic and business elites. In the first issue of OB (October 1988, p. 7), the NMA described the newsletter's role:

Each issue of On Balance discusses a different public policy issue and analyses it in terms of the positions represented, the people interviewed, the affiliation of the interviewees and the objectivity of the reporters involved in the story construction. This analysis is done using a standard communication research tool known as content analysis.

The first issue of OB, released during the 1988 federal election campaign, focussed on the highly topical subject of the free trade debate. OB claimed that media coverage, particularly on CBC television, was imbalanced against the free trade agreement. Highlighting the OB study, the Fraser Institute itself issued a press release (October 13, 1989) entitled "CBC provides unbalanced coverage of free trade," stating that "the news and public affairs programming of the [CBC] network has taken a position against the trade deal."

In the midst of an election campaign revolving around the free trade issue, such a claim was particularly newsworthy. TheOB study was publicized, largely uncritically, by a number of columnists, particularly those predisposed to accept the NMA's analysis. Supportive press commentary was penned, for example, by Dave Smith (Vancouver Sun, October 18, 1988), Douglas Fisher (Toronto Sun, October 19, 1988), and right-wing Alberta newsmagazine publisher Ted Byfield (Financial Post, November 7, 1988), amongst others. Over two years later, in a harshly critical presentation to the Canadian Radio-Television and Telecommunications Commission accusing the CBC of being anti-free trade, soon-to-be-appointed CBC director John Crispo called theOB study "definitive" (Michael Valpy, Globe and Mail, April 4, 1991).

Other organizations have likewise uncritically accepted and amplified the OB research. Thus, the Business Council of British Columbia (BCBC) released a special bulletin in January 1989, asking "how balanced is reporting of industrial relations issues?" It then answered this question on the basis of the December 1988 edition of OB:

All parties got bad press but some parties got more bad press than others. The [OB] report concludes that as a result of the enhanced attention paid to labour sources, management got the most unfavourable coverage. In the Globe and Mail, corporations received well over twice as much negative as positive coverage. CBC provided critical coverage of management nine times out of ten. (Business Council of British Columbia, 1989, p. 2)

The BCBC concluded its bulletin with a warning to management to be more pro-active in its relations with the media, based on the NMA's "proof" of inadequate media treatment of management.

The contentious and politically charged nature of OB's conclusions, combined with their apparent resonance within business and media circles, make it especially important to subject the OB research program to critical scrutiny. We begin with a basic underpinning of OB's credibility -- the claim thatOB makes for the objectivity of its content analyses of news. There are two problematic aspects of this claim: first, OB's particular application of content analysis; and second, its general understanding of, and its claims for, the technique itself.

Content Analysis: Misapplying the Technique?

To be clear on our own position, two of the authors of this paper have themselves used the technique of content analysis extensively in previous research (e.g., Gilsdorf, 1979, 1980, and 1981, p. 55; Hackett, 1983, and 1991, Chap. 9). Employed carefully and self-consciously, it can be a useful means of summarizing or describing selected aspects of a large body of material. Especially when combined with other methodologies and analytical frameworks, it can also be an important tool in interpreting the potential meanings of media texts.

On the other hand, we must recognize that content analysis is limited to and by the choices researchers make about categories, and by the degree to which they select and treat the material systematically. Like other efforts to quantify and "scientize" human behavioural and communication processes, content analysis tends to mask the degree of interpretation present and to disguise the inevitable influence created by the intervention of the researcher. That is why it is important to apply the technique in a self-reflexive way, and to indicate how the researcher's assumptions might influence the results. Moreover, done in isolation, content analysis provides only limited knowledge about the coding and decoding process -- that is, about the preferred meanings embedded in the text, and the interpretive process of perceivers. As in all research that considers the coding process or analyzes text, great care must be taken with respect to inferences about audience interpretations or "effects": these cannot simply be "read off" from quantifiable descriptions of the "manifest content."

From our reading of the first year's editions of OB, there appear to be both interesting results and some serious problems with the use of the content analysis method. These problems include certain apparent contradictions in reported findings, a lack of clarity in key areas, an unwillingness to recognize the degree of interpretation present at several levels, and a failure to utilize content analysis in association with other analytical frameworks and methodologies.

Some of the problems can be summarized simply as a lack of rigour and consistency in conducting the research and reporting its findings. Take, for example, the question of sampling. In the edition on free trade, OB excluded both the Globe and Mail's Report on Business, and its editorials, arguing in the latter case that "no equivalent could be found on television programmes" (OB, October 1988, p. 3). Such exclusions almost certainly skewed the results away from finding media coverage supportive of free trade: The Report on Business emphasizes economic concerns and speaks to a business constituency, and the Globe had editorially endorsed free trade. Later OB studies, however, do include the Report on Business, with a significant impact on the reported results, and they make no mention that newspaper editorials are excluded from the sample. Possibly OB researchers have been simply inconsistent in their sampling decisions; alternatively, they may have sought to maximize the probability of finding "imbalanced" coverage.

A similar inconsistency in the sampled universe was evident with regard to the exclusion or inclusion of CBC programmes, and in the definition of the scope of political issues, which shapes the inclusion or exclusion of stories from samples. Thus, the OB study of health care coverage (February 1989, p. 3) excluded stories on specific illnesses or ethical issues from the sample, and included only those reports relating to "the health care system in general." The reported finding that CBC network news does not give health care "prominent attention" (p. 1) in the news may well be in part an artifact of that sampling decision.

Moreover, from the information supplied in the OB reports, it is difficult for the reader to verify the accuracy and reliability of the coding process, even though OB (January 1989) has itself criticized media coverage of opinion polls for vagueness and insufficient methodological explanation. The reader was assured in the first issue of OB that a "high level of inter-coder reliability was obtained," yet no percentage figure was supplied, nor was the type of reliability test identified. Moreover, there was no clear indication in the report itself of the consistent application of coding rules.

Similarly, the discrete coding unit, which appears to be the "statement," is not clearly explained. For instance, to take the example of a reporter's narrative quoted in OB's health care issue (February 1989, p. 4), are five consecutive sentences from the same speaker elaborating a single theme coded as one statement or five? The quantitative results presented by OB, and their implications for perceived "balance" in news reporting, are highly dependent on the (inadequately explained) definition of a unit of analysis. Indeed, the selection of a "statement" extracted from a verbal transcript is far from being an obvious choice as the basic unit in the editorial decision-making process of television news, or in its construction of meanings. Other potential units of analysis which have been employed in other research have ranged from individual words, up to the entire news item or even newscast (Lichty and Bailey, 1978, p. 116), as well as the theme, and the visual shot or scene -- surely an essential dimension in television news. Yet, without further explanation or theorization to justify the "statement" as a unit of analysis,OB (October 1988, p. 1) makes this astonishing claim:

To ensure objectivity each news story is broken down to its constituent statements...By analysing each news item in terms of its integral statements, it is possible to perform completely objective analysis of the positions presented in the sense that any group of randomly selected researchers would come to the same conclusions if exposed to the statements that had been analyzed.

In addition to the inadequate explanation or justification of key methodological choices, the presentation of findings (however interesting they may sometimes be) is sometimes inaccurate, vague or misleading. Sometimes there are apparent contradictions. In one case, a pie chart contradicts the percentage reported in the text (OB, February 1989, p. 3). The OB edition on labour (December 1988, p. 4) decried the preponderance of union over management representatives as interviewees, yet in the specific instance of coverage of the nurses' strike (OB, February 1989, p. 5) doctors and government officials were interviewed far more often than were nurses or union representatives, especially on CBC. Surely this finding ought explicitly to qualify the implication from the earlier study that news favours unionized workers. Sometimes, reported results are simply confusing. In the same OB issue on health care (p. 4), OB first states that the CBC only twice mentioned user pay as a revenue option, then suggests that the "CBC provided 9 in 10 statements in support of government funded" health care, and finally states that "What is significant is that CBC simply did not address the means by which health care in Canada is to be paid for."

Other flaws in OB's reporting of results are evident. The proportion of statements which are "neutral" rather than "favourable" or "unfavourable" on an issue is often excluded or underemphasized, particularly when results are visually highlighted in charts and graphs, and especially in the early issues of OB. Thus, the finding that the vast majority of statements made by reporters on free trade were neutral statements of fact (in the case of CBC, over three-quarters) did not preclude OB (June 1989, p. 3) from asserting that "Story content was two to one against the deal." Moreover, percentages are often reported without indicating the actual total of statements on which they are based (e.g., OB, November 1988, p. 2). If the N's are quite low, then conclusions about "balance" in coverage ought to be appropriately qualified. Percentage-based conclusions are especially troubling in light ofOB's tendency (November 1988, p. 2) to report conclusions based on assertions like, "Of all statements which could be classified..." How many statements were thus excluded from categorization, and on what basis? Finally, findings are over-generalized to characterize whole media organizations on the basis of the particular programme studied; thus, the Fraser Institute press release of October 13, 1988, concerning the OB study of free trade reportage by the National and Journal, was entitled "CBC provides unbalanced coverage of free trade."

OB's application and interpretation of content analysis lacks rigour in other ways as well. There are attributions of the effects of media coverage that cannot be supported by the method employed, namely, content analysis. Thus, OB asserts in one issue (November 1988) that "The way in which Free Trade has been reported by the media is having a serious impact on the way Canadians view the deal, political parties, and the party leaders." Likewise, OB sometimes apparently imputes motives on the basis of content analysis; for instance, OB (February 1989, p. 4) claimed that CBC was "provoked" to cover health issues by the Alberta nurses' strike. Such claims should at least be qualified. The determinants and impact of news cannot simply be "read off" from its apparent content, and indeed no communication research method enables us to understand the very complex process of news production and reception as easily as the OB reports sometimes imply.

A further dual difficulty in OB's application of content analysis is the failure either to formulate theoretically relevant hypotheses for testing, or to position its work in relation to other research on news production and content. These omissions lead OB to repeat commonplace observations or well-established findings in a tone of revelation -- that news privileges events over issues, the negative over the positive, entertainment over information, and so on. More crucially, they also allow OB to avoid dealing with previous research which would contradict or qualify its previous conclusions. One example concerns patterns of regional coverage in Canadian news. Thus OB (June 1982, p. 2) asserted that "news from Ontario dominates the coverage, with nearly two-thirds of CBC coverage focusing on Ontario." This conclusion was based, however, on an assumption that news from the National Capital Region should be categorized as Ontarian. To consider all news stories originating from Parliament, the Supreme Court, or other federal institutions as "focusing on Ontario" is debatable, especially when other published studies specifically address this question. Lorimer's 1984 research on CBC radio found that a "considerable proportion of Ontario sources were accounted for by the presence of the nation's capital in Ontario," and that when these were factored out, CBC coverage "almost exactly follow[s] the relative populations of the provinces" (Lorimer and McNulty, 1987, pp. 265-271). Other research (e.g., Kiefl, 1978) also qualifies theOB conclusions about the regional profile of news.

Content Analysis: Naturalizing the Categories

While the abovementioned problems of rigour, inference, reliability and consistency are bothersome, even more troubling is the unreflexive way in which OB applies its categories of analysis. The findings generated by even the most systematic and quantitative content analyses of news are dependent upon the analytical categories employed, which must be appropriate to the object of enquiry. The derivation of such categories, as well as the coding process itself, are unavoidably acts of interpretation. Even the tradition which gave birth to content analysis as a method of communication research, American behaviouralist social science, recognized long ago, as Bernard Berelson put it, that "content analysis stands or falls by its categories" (cited in Olsti, 1968).

OB implicitly recognizes that categories and coding are interpretive processes whose validity is not internally guaranteed, since it takes pains to point out in each issue that the content analysis coders are selected for their political diversity. Such diversity, however, is no guarantee of methodological value-neutrality or objectivity in any absolute sense, since even in a well-conducted content analysis, values and judgements are unavoidably embedded in the very coding protocol, the categories and rules employed. The subjective preferences of individual coders are a secondary problem.

Such considerations, however, do not deter OB from sometimes bluntly asserting that its content analyses are straightforwardly or "completely objective" (October 1988, p. 1). Notwithstanding this claim, the categories used in the OB research deserve critical scrutiny, since they unavoidably shape the findings.

One problem is that some OB studies use overlapping categories as the basis for analysis, which poses a problem for the coding process and the interpretation of results. Thus, the overlapping categorization of the type of people accessed as sources in the news can lead to ambiguous findings, as in OB's study of environmental news. OB (October 1989, p. 2) claimed that "environmentalists" were "cited more often than scientists on CBC" without providing a definition of either group, or conceding the possibility that an individual could be both an environmentalist and a scientist.

With regard to news topics, the selection of categories often appears to be arbitrary and not informed by previous research. Thus, the OB issues on the federal election (November 1988, January 1989) have the coverage broken down into various topics. Leadership is not one of those listed, although other content analysis-based studies of the last four elections have identified leadership as consistently one of the most frequently mentioned topics (see, e.g., Soderlund, Romanow, Briggs and Wagenberg, 1984, pp. 50-57). It is also unclear from the OB election issues whether announcements by the Progressive Conservative federal government were coded as statements by and for the Progressive Conservative party.

Even more crucial are those categories which directly underpin OB's evaluations of balance or imbalance. First, OB distinguishes between "factual" and "opinion" statements, and codes only the latter in its efforts to measure the achievement of balance in news reports. Methodologically, the "opinion" category overrides an important distinction used in other research (Lowry, 1970, pp. 205-210), between "inference statements," which concern the apparent meanings or probability of events, and "editorial statements," which comment on "the goodness or badness of something." Lowry found that while non-factual inference statements were fairly frequent in U.S. network news, outright editorializing by journalists was rare. But by combining the two types of statement, OB leaves the impression that news is loaded with overt expressions of opinion. The OB analysis of free trade coverage provides an example of an apparently reasonable inference which was coded as an "opinion" statement: "Many in Congress feel [the current trade situation] allows Japanese and Korean cars made in Canada to sneak into the American market, under cover of the Canada-U.S. Auto pact."

While such a statement may be an "inference" in Lowry's definition, it would be misleading to suggest that it expresses the reporter's opinion on the merits of the free trade deal. Indeed, such inferential statements may well occur frequently in television coverage of issues as complex as the free trade negotiations, given the pressure on journalists to summarize in a few paragraphs or minutes.

Far more important, though, if we are interested in analyzing the ways in which news is oriented towards power and discourse, the factual/opinion dichotomy ignores the ideological selectivity, categorization and labelling within the realm of the "factual" itself. In its search for imbalance in the news, OB actually naturalizes the claims of news discourse to neutrally reflect the real. That is, OB scans the media for alleged departures from journalistic criteria of "balance" and "fidelity," rather than subjecting those criteria themselves to critical examination. Our section on balance and benchmarks below will elaborate this theme.

Equally problematic is OB's heavy reliance on another rather crude dichotomy -- between "favourable" and "unfavourable" statements on issues. Such a dichotomy often obscures as much as it reveals. For example, it conflates the distinction between the "efficacy" (power) and "morality" of news actors, a distinction usefully employed in news research as early as 1945 (Sussmann). Moreover, the category of "unfavourable" statements seems to involve any comment that is critical on a given issue, which involves an enormous conflation of a range of possible comment. One example is the abortion debate (discussed in OB, May 1989): both "pro-life" and "pro-choice" groups opposed the government's legislation. Surely it is misleading to combine them into the category "unfavourable."

OB's analysis of labour coverage provides examples of how the simplistic favourable/unfavourable dichotomy overrides such important nuances and context, and can thus be rather misleading. OB (December 1988, p. 5) concluded that because labour sources are quoted more frequently than are management sources, "management was given the worst press." Other studies have also found that labour spokespeople outnumber management in the news, which is understandable given the sheer number, availability and newsworthiness of trade unionists and their actions (Knight, 1982; Hackett, 1983; Hackett, 1985). But these studies have also revealed some very important qualifications to such numerical imbalance, qualifications which OB ignores. For example, trade unionists are more likely to express internal divisions. They often appear in a context in which they are placed on the defensive and compelled to justify work stoppages, for which the language and inferential structure of news holds labour largely responsible. OB described a picketer yelling on national television, "Hey you slimy scabs, wait till you're out on the street, you little pigs," as an opportunity for labour representatives "to communicate about other players"; yet the woman's anger, language, and implicit threat, especially in the eyes of viewers not predisposed to support union militancy, probably do more to hurt labour's image than to enhance it. Indeed, OB itself (December 1988, p. 6) described labour sources as the most "emotional" and "vehement." By contrast, management representatives typically are quoted about a broader range of issues and in a context which affords them greater status, apparent rationality, and influence over the content of their speech clips (Hackett, 1985). For all these reasons, the image of labour and business conveyed by the news media cannot adequately be analyzed simply by counting the number of labour and management people quoted, and by coding the apparent (un)favourability of an aggregate of individual statements about the two groups.

There is another conceptual flaw with the favourable/unfavourable dichotomy: It ignores the active role of audiences in creating the meanings of news. The viewer's interpretation of whether a given statement can be deemed "favourable" or "unfavourable" (or irrelevant) towards a particular position depends in part on his/her socially- and culturally-located "decoding framework" or perspective (Morley, 1980). Consider, for example, whether the following statement by a CBC reporter should be regarded as "favourable" towards free trade, by American and Canadian TV viewers respectively: "The U.S. Congress is being told that the agreement is a sell-out."

If the agreement was an American defeat in a situation of bilateral negotiations, then surely it was conversely a Canadian victory. Yet the OB researchers coded that statement as "anti-free trade." To regard statements as "unambiguously" positive or negative on a given issue (OB, October 1988, p. 1) involves a prior assumption -- namely, that there is a near-universal normative consensus in Canadian society. Such an untheorized a priori assumption would seem to preclude raising for analysis the ways in which the media (and other ideological institutions) help to construct consensus, and to shape the criteria by which audiences evaluate public events, issues and actors.

Conceptual Problems: Balance and Benchmarks

Many of the methodological problems discussed above are merely symptomatic of OB's failure critically to address some very fundamental conceptual and epistemological questions. OB has failed adequately to theorize about the concept of balance and whether it constitutes an appropriate criterion for evaluating the news, about the relationship between news texts and the political world they purport merely to describe, and about the constraints and determinants of the news production process.

The first set of problems concerns the concept of "balance," the value standard which explicitly drives the OB research agenda, and which leads to further questions such as the "benchmarks" against which to evaluate the adequacy of news.

In its first issue, OB defined as its project the examination of the "fidelity" with which national media carry out their public information function. Yet what is meant by "fidelity" is not explored. If "fidelity" means the ability of media neutrally to reflect the contours of public debate, it implies that media can stand apart from and outside the political struggles which they narrate. Several powerful arguments can be advanced against such an assumption: News media unavoidably structure their representation of social and political events in ways which are not pregiven in the events themselves. Language itself cannot function so as to transmit directly the supposedly inherent meaning or truth of events. Far from constituting a detached observer, the media help actively to construct the social world, which is not a pregiven reality available for the media to "reflect" (Hackett, 1984, pp. 234-235).

Even if the possibility of such "fidelity" were conceded, an analysis of how the media amplify, negotiate or contest the competing positions on public issues would require at least a sketch of those contending positions and definitions of issues. But unlike other researchers (e.g. the Glasgow University Media Group 1976, 1980), OB does not typically contextualize its evaluation of media performance in such a way. Instead, OB appears simply to equate media "fidelity" with the "balanced," symmetrical treatment of two competing sides of public issues (whose origins are considered unproblematic). There are a number of problems with this approach, which, to be sure, OB shares with some other mass communication research done on news "bias" or "balance" (Hackett, 1984).

First, OB seems to assume that all issues naturally have two antagonistic and equally weighted sides. Yet, in terms of political weight and/or generally accepted normative standards, many issues are not susceptible to such dyadic balance. If news emphasizes the negative effects of sexual assault, should these be counterbalanced by portrayal of positive effects? The OB edition on poverty decried the sympathetic depiction of poverty, but it is difficult to imagine an unsympathetic depiction of the poor in order to achieve balance.

Second, OB fails to notice a potential contradiction between balance and fidelity. The symmetrical presentation of competing viewpoints on public issues would often not be a faithful representation of public discourse. To take one example, Liberal strategists in the 1980 federal campaign limited their leader, Pierre Trudeau, to one or two public events per day. The Progressive Conservative and NDP leaders were much more active, but the Liberals counted on journalists' "rules of fair play" to override their concerns about a cynical campaign, and to provide the Liberals with equal time and space (Gilsdorf, 1981, p. 64). To the extent that "balance" was achieved, it was at the expense of a "faithful" representation of the campaign.

Third, in the absence of analysis of the origins and perspectives on public issues, OB seems to draw upon the implicit knowledge or perspectives of the researchers in order to identify the relevant "two sides" of a given issue. Thus, the OB edition on health care (February 1989) did not offer an analysis of the contending positions on the issue or their relative weight. Instead, OB appeared simply to posit "user pay" versus "public funding" perspectives, and then proceeded to demonstrate that the former received far less coverage. Sometimes, in the effort to measure dyadic balance, OB is led to posit a virtually non-existent opposing position, as in the edition on abortion (May 1989). OB found that negative comments greatly outweighed positive statements on the government's strategy of introducing a number of options for debate instead of a standard bill. But since almost all outsiders, both "pro-life" and "pro-choice," were critical of the government's approach, "balanced" coverage would have been misleading and inaccurate. Sometimes OB (e.g., February 1989) has attempted to use negative content analysis, which can be a legitimate technique to identify significant omissions in the news agenda. However, this technique requires a comprehensive analysis of the issue in order to indicate what absences would indeed be significant; this, OB does not appear to do.

Fourth, as we have noted, the effort to define statements in the news as "for" or "against" a particular policy option, such as free trade, involves an enormous conflation of a range of possible comment.

Fifth, OB itself sometimes recognizes, at least implicitly, that "balance" is not always a sufficient or adequate evaluative criterion. However, its efforts to supplement or replace the concept of balance are no better theorized. Thus, in its edition on the environment, OB (October 1989) expands its focus to "objectivity." Here, OB adopts a very simplistic operationalization of this concept, defining it as reactive journalism, by contrast with the "advocacy journalism" which newsworkers supposedly are practising when they initiate coverage of an issue. Such a definition ignores the complexities and constraints of the news production process, and of the ensemble of ideas and practices which constitute news objectivity. Moreover, objective journalism as OB defines it would simply accord even more power to the social and political groups most capable of initiating newsworthy events or of making statements which can be translated into public policy -- notably government.

Another questionable effort to find a replacement for "balance" as a benchmark for the evaluation of news coverage occurs in OB's edition on AIDS (July/August 1989). Presumably, no sane person would be in favour of AIDS, so the balance criterion does not apply. Instead, OB contrasts news mentions and official statistics on the means by which the HIV virus is spread. OB argues that CBC under-reports HIV transmission by homosexuals and bisexuals, by comparison with its relative frequency as a cause of the disease. What is the relevance of this finding if OB's purpose is to help its readers to understand and evaluate news media? Should CBC have connected 85.3% of its coverage of AIDS with homosexual transmission in order to achieve "balance"? From a public health viewpoint, how productive would such coverage be if the fastest-growing groups of victims are young people and heterosexual women? In using such an untheorized benchmark, OB comes very close to inviting the media to indulge in intensified homophobia.

Sixth, "balance" is far from being a guarantee of participatory and democratic communication. As Cook and Ruggles argue elsewhere in this volume, balance in regulatory and media practice generally means balance amongst elite viewpoints, to the exclusion or marginalization of oppositional grassroots perspectives. Balance works to establish dominant social and political institutions as primary definers of political issues. Moreover, balance is part of a regulatory regime which generally legitimizes the professionalization of media production to the virtual exclusion of community participation. These implications are unexplored by OB in its uncritical adoption of the balance criterion.

Seventh, even if one discounts these fundamental theoretical objections to the concept of balance, its empirical measurement is far more problematic than OB acknowledges. Balance cannot be determined simply by quantifying statements "for" and "against," especially in a medium such as television that carries a heavy emotive load, often embedded in the visuals. Moreover, when quantifying balance, the number of mentions and the amount of time or space are not equivalent components; in this respect, reliance on percentages can become rather misleading, especially when the derivation of numbers and their totals are not clarified.

Assumptions about the Production and Meanings of TV News

A related problem with OB's research is that it does not appear to be informed by theoretical reflection upon the production of news, and the meanings which it generates.

To explain our critique, we first outline briefly some of our own assumptions about the production and meanings of news in television, the medium which is the main focus of OB's research. News is a product of organizations situated within a matrix of social relations. Historically, North American television has been driven by a commercial imperative. Almost from its inception, television has been conceived as primarily an entertainment medium. Programs are developed and produced to deliver audience attention to advertisers. Even within the State-owned CBC, audience size and the commercial imperative are important -- given competition with commercial broadcasters, partial dependence on advertising revenue, and the need to show a return for the public funding which CBC receives. Although CBC newscasts as such do not carry advertising, CBC personnel, including those at the National and the Journal, follow the ratings and their own audience research with interest. However, the functions of CBC's news and public affairs programming, especially its political coverage, are not strictly commercial. Such programming has a highly respected reputation and constitutes an important legitimizing agent for the corporation. News coverage is seen as at the core of the CBC's legislative mandate to serve the information needs of all Canadians, and legitimizes the corporation in the eyes of MPs and civil servants who must review, recommend and approve funding for that crown corporation (Gilsdorf, 1990).

Within the broad context of such commercial and political imperatives, the production of TV news about elections or political issues is the result of the complex interaction of a number of factors. These include, for example, a series of organizational constraints (for example, limited resources, deadlines, the structure of the newsgathering "net"), journalists' relations with sources (for example, politicians and their aides) and source organizations; the technological biases of the medium itself; conflict, negotiation, bureaucratic precedent and routine within the news organization; professional standards and working ideologies (such as criteria of newsworthiness, notions of objectivity, assumptions about competing media and about the news audience), the socialization and training of newsworkers as organizational actors, which Ericson and his colleagues (1987, pp. 133-138) call the "vocabulary of precedents," as well as individual characteristics (energy, knowledge, attitudes, and the like).

Like news production itself, the generation of the meanings which journalists and their audiences attach to the news is a complex social process, neither free-floating nor entirely determined. Meaning is constructed through structures of social and cultural relations, relations which both position people as self-organizing subjects, and which frame or constrain their interpretive processes. Audiences interpret the news through their own socially and culturally-located decoding frameworks (Morley, 1980). For their part, along with other political, economic and cultural institutions, mass media play a major role in the construction of meaning. While journalists have more autonomy than instrumentalist theories of the news postulate, they tend to share the decoding frameworks of their audiences, and are additionally subject to the organizing relations and practices, summarized above, of the institutions within which they work.

However, OB's research seems to take into account neither the complexity of the production of news and its meanings, nor the relevant literature (Tuchman, 1978; Ericson, Baranek and Chan 1987 and 1989; Hall, Critcher, Jefferson, Clarke and Roberts, 1978; Gilsdorf, 1990, to name only a few). We have already noted that OB seems to assume, problematically, that news can stand outside of the political world as a detached and neutral observer. This point can be expanded: OB apparently assumes that the meaning or facticity of an event it something to be discovered "out there," existing independently of those who have an interest in creating, reporting or consuming it. The role of interpretation that invades all levels of the selection, gathering and presentation of news seems to have been evacuated from OB's own interpretation and presentation process.

In addition, OB sometimes makes inferences about the newsgathering and reporting process which underlies the content patterns of news. While well-argued inferences can be useful, OB sometimes makes unsubstantiated allegations or implications of editorial partisanship, and frequently the news production process is oversimplified or misinterpreted. Five examples follow:

  • OB (October 1988, p. 1) states that "the news media structure events of the day in order of perceived importance." This may be true of television's lead story (though the availability of visuals which can "grab" the viewer also play a role). But other stories do not necessarily follow in diminishing order of perceived importance. Rather, they are typically organized in regional or topic groupings to establish "flow," in the sense discussed by Williams (1974, pp. 86-118).
  • OB (October 1988, p. 3) asserts that opinions and positions are "the direct result of editorial decisions by journalists." This is indeed the case some of the time, but as extensive research on journalistic routine shows, decisions are typically made much less consciously. Routines are developed due to the many constraints in newswork, particularly time, budgets and socialization practices.
  • OB (November 1988, p. 3) states that "journalists obtain more information than they can possibly report in the time allotted...Views presented comprise only a fraction of those collected for a story." While editorial selectivity is certainly necessary, it is usually a process of winnowing comments from a long press conference or one or two respondents. Rarely do journalists have the luxury of interviewing large numbers of sources for a single story. Again, OB overlooks the concept of routine and the very considerable constraints of time.
  • In several issues, OB seems to decry the absence of economists in CBC coverage. This conclusion needs to be contextualized by a knowledge of CBC's "news net" and other constraints on news production, including notions of who its audience is.
  • Most of OB's analysis ignores the role that sources and source organizations play in determining news practice and content. The self-interest and relative accessibility of different organizations, and their differential capacity efficiently to produce apparently authoritative but often self-serving information, all significantly influence what gets covered and who gets to make statements to the media (Gandy, 1982; Gans, 1980, pp. 116-145; Ericson, Baranek and Chan, 1989). When considering sources in the news in relation to questions of balance, researchers could usefully distinguish between "primary" and "secondary definers" (Hall, Critcher, Jefferson, Clarke and Roberts, 1978, pp. 57-60). That is, those sources who are able to set the terms of debate and the definitions of reality amplified in the news are often the key factor in establishing or maintaining a point of view. A mere head-count of types of sources fails to address the crucial question of the frames of analysis underpinning news coverage.

Reporting Results: The Rhetoric and Political Uses of Content Analysis

In addition to methodological and conceptual difficulties,OB often undercuts its stated goal of scientific objectivity by an editorial mode of reporting results. One disturbing tendency is the use of graphs suggesting that the reporting of particular, named journalists is overwhelmingly imbalanced on a particular issue -- sometimes on the basis of only one or two news stories (e.g. OB October 1988, p. 4). Given the contestable and interpretive nature of OB's own methodology, such a "trial by graphics" cannot be taken at face value, and yet it can be intimidating to individual journalists.

The use of headlines and quotations in sidebars in OB, while it can be a legitimate reporting device and an aid to readers, often presents a clear point of view. Moreover, the reader of OB is sometimes confronted with gaps between evidence and conclusions, gaps which are seemingly bridged by interpretations which more often than not are consistent with the ideological stance of the Fraser Institute. One example can be cited from the OB issue (March 1989, p. 4) on international news coverage, written by Professor Barry Cooper and Denise Savage. A section headlined "Soviet prominence" referred to CBC's "relatively extensive use of Soviet and Eastern bloc sources." Two specific examples of CBC quotes from official Soviet sources justifying Soviet military policies were then offered. Yet OB offered no statistical analysis of the relative frequency of official and dissident or anti-Communist sources in CBC news from the USSR. Moreover, the data themselves -- on the same page -- indicated that the USSR and all Eastern European countries combined form the geographic source in fewer cases than Israel alone, and only half as many as the USA. Why then was the section written in such a way as to imply a pro-communist tendency in terms of geographic news sources?

Indeed, that is how the rhetoric of OB reports largely works--by a process of implication. OB studiously avoids the term "bias," and nowhere does it render explicit the thread which guides much of its research; yet it fairly consistently develops an inferential structure favourable to what Adams (1978, p. 18) calls "a political-attitudinal theory" of news, one which "sees the personal political opinions of newspeople as regularly influencing decisions about what to cover and how to cover it." OB (June 1989, p. 4) perhaps comes closest to stating such an overall interpretation of the news in its review of its own previous findings on news coverage of five major topics. In a section headed "Reporters' slant is patterned," OB argued that reporters "expressed decidedly firm opinions on the economic and social issues examined" -- against privatization and free trade, for pro-life more than pro-choice positions on abortion, for public rather than user funding of health care, and for labour's position rather than management. The section then concludes:

This analysis indicates that, as one might expect, reporters are not completely value-free in their coverage of issues. What is somewhat disconcerting is to discover that reporters' opinions follow a consistent pattern.

That alleged pattern is not specified, but columnist Andrew Coyne was surely not far from the mark in his interpretation of the On Balance research agenda:

Ostensibly, it's just disinterested empirical observation, like bird-watching, but the intention is clear: to gather proof of the leftist conspiracy conservatives are convinced is running the media (Financial Post, August 9, 1989, p. 11).

Such a view of the media can be sustained only by ignoring a great deal of reputable research which suggests that mainstream media tend to amplify official definitions of reality and to marginalize and delegitimize fundamental opposition (see, e.g., Gans, 1980; Gitlin, 1980). Moreover, we have argued above that the OB research is methodologically and conceptually questionable. It adopts a positivist and non-reflective view of content analysis; it uncritically accepts a rather limited (and limiting) notion of "balance"; it does not take into account the complexities of producing news and its meanings; and it is prone to oversimplification, and a lack of theorization and contextualization. OB's stated aim of objectively analyzing the fidelity of news coverage is in contradiction with the very conditions under which it is produced. It is funded by an advocacy group whose very purpose is to promote laissez-faire economics, and by presumably like-minded (and affluent) subscribers to the $95 per year newsletter. In addition to such potential ideological pressures, the necessity of producing a large-scale study every month creates its own constraints. How else can one explain the publication of results which can be produced easily (such as computer-generated word counts), but which are neither theoretically interesting nor even descriptively meaningful (such as the number of references in news transcripts to "socialism," "liberalism," and "conservatism" without any analysis of the source, context or direction of their utterance)? How else can one explain such published inaccuracies as a list of the ten most frequent international newsmakers which identifies Manuel Noriega as "past-president of Nicaragua," and which includes Soviet politicians Andrei Sakharov and Boris Yeltsin, but excludes Mikhail Gorbachev (OB, January 1990, p. 7).

Yet despite such flaws, OB has received considerably more public recognition than have most content analyses of news. Such recognition is understandable if we make sense of OB not as media scholarship, but as a means of calling media attention to itself and its political viewpoint. Political advocacy groups, especially in the U.S., have increasingly launched public attacks on media institutions on the basis of "balance" in the hopes of garnering public attention and political support (see, e.g., Gans, 1985; Stoler, 1986; Montgomery, 1989). Indeed, OB can be seen as part of a process of "ideological mobilization" (Dreier, 1982) by the political Right and its corporate allies against the media -- particularly public broadcasting.

It is significant that economist Michael Walker, Fraser Institute founder, has commented favourably on the manifestly conservative U.S. media watchdog Accuracy in Media (AIM) (Fraser Forum, March 1986, p. 8). AIM and the Fraser Institute have two important parallels. First, they both seek to maximize their exposure in the same media they criticize. A good portion of the Fraser Institute staff time is devoted to preparing written and taped commentaries for free distribution to media across Canada, and Michael Walker has positioned himself as a key media spokesperson for the Right (Alberta Report, February 8, 1987). Second, both organizations focus their media critiques on publicly-funded broadcasters. The common element in all OB newsletters is a critique of the CBC, sometimes in comparison with the Globe and Mail and/or CTV, sometimes on its own.

Certainly opponents of public broadcasting have found sustenance in the OB research. We have noted above that John Crispo, shortly before his controversial appointment to the CBC Board of Directors in April 1991, favourably cited OB's study on free trade. In this context, his perspective on CBC suggests what is at stake for public broadcasting (and indeed for media autonomy and diversity) in Canada: he has decried CBC as a "propaganda agency" for "left-wing producers" which deserves to be closed down if it does not "clean up its act in terms of bias and efficiency," and he suggested that the motivation for government cutbacks to CBC was as much political as economic (Michael Valpy, Globe and Mail, April 4, 1991, p. A6).

We are not denying that CBC and other news media need to have as many people critically assessing their work as possible, from various points on the political spectrum. There are legitimate roles both for advocacy groups and for content analysis in that process of critique. However, the normative commitments necessarily underlying such critiques ought not to be obscured by repeated claims of transcendent objectivity.

Rather, such critical practices should lead to an exploration, analysis, discussion and understanding of the complexities of the social process of making meaning -- whether as a journalist, an academic, or a representative of particular interests. We hope that we can all avoid oversimplification, and that we are all ready to acknowledge that we are interpreters of human behaviour and the human experience, implicated in the very processes that we observe.


The opinions expressed in this article are those of the author and they do not necessarily represent an official position of the CBC.
It must be recalled that our discussion is based on the first fourteen issues of On Balance. Since then, to its credit, the NMA has at least started to provide percentages of inter-coder agreement, and occasional references to previous studies.
This statement from a CBC-TV news report was brought to our attention through a journalist whose reporting was labelled "anti-free trade" in OB. The reporter subseqently exchanged correspondence with the NMA regarding how it had reached this conclusion. He was given some examples of his statements, including this one, which were coded as "anti-free trade." OB has unfortunately been more reluctant to reveal to its readers such examples of its coding decisions.
For background, see note 3.
The Fraser Institute has never hidden its anti-CBC position. Numerous articles in the Fraser Forum have called on the government to privatize the Crown corporation. In January 1986, the Forum published assertions that nobody watches the CBC, that it benefits only those "who draw salaries from the operation," "over-manning [sic] is near rampant," and on-air staff are "biased left-ward."
For instance, the analysis in this article is undoubtedly informed by the authors' support for the principles of public broadcasting, and for forms of democratic communication which are not confined to formal "balance" between representatives of elite opinion.


Adams, William C. (1978). Network news research in perspective: A bibliographic essay. In W.C. Adams and F. Schreibman (Eds.), Television network news: Issues in content research (pp. 11-46). Washington School of Public and International Affairs, George Washington University.

Business Council of British Columbia. (1989). Industrial Relations Bulletin. 21, (2) (January 17).

Dreier, Peter. (1982). Capitalists vs. the media: An analysis of an ideological mobilization amongst business leaders. Media, Culture and Society, 4, 111-132.

Ericson, Richard P., Patricia M. Baranek and Janet B.L. Chan. (1987). Visualizing deviance: A study of news organization. Toronto: Buffalo and London: University of Toronto Press.

Ericson, Richard P., Patricia M. Baranek and Janet B.L. Chan. (1989). Negotiating control: A study of news sources. Toronto: Buffalo and London: University of Toronto Press.

Fraser Institute. (1989). Annual Report 1989. Vancouver, B.C.

Gandy, Oscar H., Jr. (1982). Beyond agenda setting: Information subsidies and public policy. Norwood, New Jersey: Ablex.

Gans, Herbert J. (1980). Deciding what's news. New York: Vintage Books/Random House. (Originally published 1979).

Gans, Herbert J. (1985). Are U.S. journalists dangerously liberal? Columbia Journalism Review (November/December), 29-33.

Gilsdorf, William O. (1979). The Liberal Party and the media in the 1979 Canadian federal election: Some preliminary conclusions. Recherches en Communication, Vol.1,(pp. 22-33) Montreal: University of Montreal.

Gilsdorf, William O. (1980). The structure of media coverage in the 1979 and 1980 Canadian federal elections. Paper presented to the Annual Meeting of the Association for Education in Journalism.

Gilsdorf, William O. (1981). Getting the message across: Media strategies and political campaigns. In Liora Salter, (Ed.), Communication studies in Canada. (pp. 52-67) Toronto: Butterworths.

Gilsdorf, William O. (1990). The organizing processes of the CBC-TV national Election Unit in the 1988 Canadian federal election. Paper presented to the Annual Conference of the Canadian Communication Association, Victoria, B.C. (52 pages).

Gitlin, Todd. (1980). The whole world is watching: Mass media in the making and unmaking of the New Left. Berkeley, Los Angeles and London: University of California Press.

Glasgow University Media Group. (1976). Bad news. London: Routledge & Kegan Paul.

Glasgow University Media Group. (1980). More bad news. London: Routledge & Kegan Paul.

Hackett, Robert A. (1983). The depiction of labour and business on national television news. Canadian Journal of Communication, 10(1), 5-50.

Hackett, Robert A. (1984). Decline of a paradigm: Bias and objectivity in news media studies. Critical Studies in Mass Communication, 1(3), 229-259.

Hackett, Robert A. (1985). A hierarchy of access: Aspects of source bias in Canadian TV News. Journalism Quarterly, 62(2), 256-265, 277.

Hackett, Robert A. (1991). News and dissent: The press and the politics of peace in Canada. Norwood, N.J.: Ablex.

Hall, Stuart, Chas Critcher, Tony Jefferson, John Clarke, and Brian Roberts. (1978). Policing the crisis: mugging, the state, and law and order. London: MacMillan.

Holsti, O.R. (1968). Content analysis. In G. Lindzey and E. Aronson (Eds.), The handbook of social psychology, vol 2, (2nd ed.), (pp. 596-692). Reading Mass.: Addison-Wesley.

Kiefl, Barry. (1978). Happy medium: A content analysis of network television news. Ottawa: CBC Research.

Knight, Graham. (1982). Strike talk: A case study of news.Canadian Journal of Communication. 8(3), 61-79.

Lichty, Lawrence W. and George A. Bailey. Reading the wind: reflections on content analysis of broadcast news. InTelevison network news: Issues in Content Research (pp.111-137). Washington: School of Public and International Affairs, George Washington University.

Lorimer, Rowland, and McNulty, Jean. (1987). Mass communication in Canada. Toronto: McClelland and Stewart.

Lowry, Dennis T. (1971). Agnew and the network TV news: A before/after content analysis. Journalism Quarterly, 48 (Summer), 205-210.

Montgomery, Kathryn C. (1989). Target: Prime time, Advocacy groups and the struggle over entertainment television. New York and Oxford: Oxford University Press.

Morley, David. (1980). The 'Nationwide' audience: Structure and decoding. BFI Television Monograph No.11. London: British Film Institute.

Soderlund, Walter C., Walter I. Romanow, E. Donald Briggs and Ronald H. Wagenberg. (1984). Media and elections in Canada. Toronto: Holt, Rinehart and Winston.

Stoler, Peter. (1986). The war against the press: Politics, pressure and intimidation in the 1980's. New York: Dodd, Mead & Company.

Sussmann, Leila. (1945). Labor in the radio news: An analysis of content. Journalism Quarterly (September),207-214.

Williams, Raymond. (1974). Television: Technology and cultural form. Glasgow: Fontana/Collins.

  •  Announcements
    Atom logo
    RSS2 logo
    RSS1 logo
  •  Current Issue
    Atom logo
    RSS2 logo
    RSS1 logo
  •  Thesis Abstracts
    Atom logo
    RSS2 logo
    RSS1 logo

We wish to acknowledge the financial support of the Social Sciences and Humanities Research Council for their financial support through theAid to Scholarly Journals Program.