A SUMMARY OF EXPERIMENTAL RESEARCH ON THE
EFFECTS OF EVIDENCE IN PERSUASIVE COMMUNICATION
James C. McCroskey
Traditional theories of rhetoric, person-to-group persuasive communication, stress the value of documented supporting materials, commonly called evidence, in the production of attitude change. Contemporary theorists for the most part concur with traditional theorists in this regard. However, because of the conflicting findings of the few reported experimental studies of the effect of evidence on attitude change, some writers have questioned whether evidence actually has much, if any, impact in persuasion. After reviewing some of these studies, for example, Gregg concluded that "the audience reaction to an argument may have little or nothing to do with whether the argument includes fully documented or completely undocumented evidence, relevant or irrelevant evidence, weak or strong evidence or any evidence at all.1
The purpose of this paper is to examine the results of several studies
of the effects of evidence, most of which have been conducted by the present
writer, and to suggest when and how evidence may function in persuasive
communication. The studies conducted by the writer that underlie this paper
are reported in detail elsewhere.2
STUDIES REPORTED BY OTHER RESEARCHERS
Nine studies have been reported by other investigators that involve research on the effect of evidence on attitude change in persuasive communication. Two of these found that inclusion of evidence increased the amount of attitude change produced by the message; two found a trend in this direction; and five observed no significant effect on attitude change attributable to evidence.
The two earliest reported studies, those reported by Cathcart3 and Bettinghaus,4 were the only ones to produce statistically significant results favoring inclusion of evidence in a speech designed to achieve attitude change. Studies reported by Gilkinson, Paulson, and Sikkink5 and Ostermeier6 demonstrated trends favoring inclusion of evidence that did not meet normal criterion levels for statistical significance. Anderson,7 Costley,8 Dresser,9 Gardner,10 and Wagner11 found no significant superiority in the production of attitude change for a speech including high quality evidence over a speech including either no evidence or low quality evidence.
In the only reported study of the effects of evidence on perceived source credibility conducted by an individual other than the writer, Ostermeier found that including a form of evidence significantly increased perceived credibility.12
From these studies no firm generalization can be drawn concerning the effect of evidence on attitude change and one can only tentatively conclude that evidence has an impact on source credibility.
In an attempt to provide data upon which meaningful generalizations concerning the effect of evidence in persuasive communication could be based, the writer has conducted twelve studies. The following discussion is organized around the variables that were suspected as interacting with evidence in producing either attitude change or perceived source credibility. In each section there is presented a theoretic rationale for the hypothesized relationship between evidence and the other variable and a discussion of studies investigating this relationship. Since all of these studies have been reported in considerable detail elsewhere, the discussion in the following sections will stress results rather than procedures. The reader may find information on procedural details in previous reports that are cited.
A consideration of relationships between evidence and other communication variables require several operational definitions to avoid confusion in interpreting the material that follows.
The meaning of the term "evidence" as used by various authors of articles and books on persuasive communication is not always the same. The definition of "evidence" employed by the researchers involved in the studies discussed in this paper is compatible with that of the majority of writers. "Evidence" is taken to mean factual statements originating from a source other than the speaker, objects not created by the speaker, and opinions of persons other than the speaker that are offered in support of the speaker's claims. Only opinions and factual statements clearly identified with a source other than the speaker have been used in most of the studies; the use of audio-tape as the medium of transmission in most of the experiments precluded the use of tangible objects as evidence.
Source credibility or ethos was operationally defined in this series
of studies as perceived "authoritativeness" and "character"
as measured by either Likert or semantic differential instruments designed
for this purpose.13 Attitude change was operationally defined
as the difference between attitude prior to exposure to the experimental
stimuli and attitude subsequent to exposure as measured by Likert or semantic
differential instruments designed for this purpose.14
EVIDENCE AND SOURCE CREDIBILITY
The first variable suspected of interacting with evidence in the production of attitude change was source credibility. Specifically it was hypothesized that a speech including evidence would be more successful in producing attitude change than a speech not including evidence when the speech was presented by a moderate-to-low-credible source but that inclusion of evidence would have no effect when the speech was presented by a high-credible source. The rationale behind this hypothesis is twofold. First, the use of opinions or facts attested to by a source other than the speaker is a direct attempt to employ the credibility of the cited source as a persuasive tool. When the speaker's credibility is initially high, bringing even more credibility to bear on the case may be unnecessary. In short, there may be a point beyond which increasing credibility does not increase attitude change. Second, consistency theory suggests that, within limits, the greater the inconsistency between attitude toward source and attitude toward concept the greater the pressure to change attitude. When the speaker's credibility is initially high, the fact that other high-credible sources agree with him is consistent and thus unlikely to have much effect on the speaker's credibility. The initially low-credible source, on the other hand, has much to gain in credibility by demonstrating that high-credible sources agree with him. As his credibility increases the inconsistency between the audience's attitude toward him and toward the concept he favors is increased. Thus, while the initially high-credible source has little to gain from evidence, the low-credible source may increase his credibility by citing evidence and, in turn, increase the amount of attitude change produced in his audience.
The previously cited study by Ostermeier found that evidence did increase a source's credibility. For the most part, my studies have produced similar results, but only when the source was perceived by the audience initially as moderate-to-low-credible. My first study involved an unknown, unidentified, tape-recorded speaker. The subjects were college students participating in an experiment outside their normal classroom in the evening with only an unknown experimenter present. The results of this study indicated that inclusion of evidence increased both perceived credibility and attitude change.15 A partial replication of this study employed subjects who were high school students participating in an experiment under classroom conditions with two known and respected teacher-experimenters present.16 The results of this study indicated no effect of evidence on either credibility or attitude change. These conflicting results led to the speculation that the initial credibility of the unknown, unidentified, tape-recorded speaker was artificially increased in the second study by the presence and tacit sponsorship of the known and respected teacher-experimenters.17 This speculation was borne out by the results of a study reported by Holtzman that employed the same instruments included in the previous studies under more controlled conditions.18
The combined findings of these three studies provide support for the hypothesis that inclusion of evidence can increase the credibility of and the attitude change produced by an initially moderate-to-low-credible source but has no effect when the source is initially high-credible.
Six subsequent studies have provided additional support for this hypothesis.19 In no case was evidence observed to increase the attitude change produced by an initially high-credible source while, in most cases, inclusion of evidence by an initially moderate-to-low-credible source was observed to increase attitude change. Similarly, inclusion of evidence rarely was observed to increase the perceived credibility of initially high-credible sources but, in most cases, was observed to increase perceived credibility of initially moderate-to-low-credible sources.
Taken as a whole, these nine studies provide substantial justification
for the generalization that initial credibility and evidence usage interact
to produce attitude change and perceived credibility. Briefly stated, the
initially high-credible source gains little from including evidence but
the initially moderate-to-low-credible source can substantially increase
his perceived credibility and the attitude change he produces in his audience
by including evidence to support his position. This generalization, however,
will be somewhat tempered by the results concerning evidence and other
communication variables discussed below.
CREDIBILITY OF SOURCES OF EVIDENCE
Traditional theory insists that for evidence to have a favorable impact
on an audience it must come from sources the audience accepts as credible.
Because of conflicting results across topics in two studies20
it was suspected that evidence included in the experimental messages employed
may not have been the "best" evidence. Most of the evidence included
was of the "unbiased" type. Several writers have asserted that,
while unbiased evidence is better than biased evidence, reluctant evidence
(a biased source testifying against what appears to be his best interests)
is the best of all. Two studies by Arnold and McCroskey21 and
two by McCroskey and Wenburg22 found no support for this theory.
In each study biased sources were perceived to be less credible than unbiased
or reluctant sources, but unbiased sources were found regularly to be more
credible than reluctant sources. While these four studies do not provide
definitive answers to questions concerning the credibility of sources of
evidence, their results provide ample support for the contention that the
evidence included in the other evidence studies was of sufficiently high
quality so that the results of the studies were not contaminated by audiences
perceiving the evidence as emanating from low-credible sources. Additional
support for this contention was provided by ratings on evidence quality
obtained from subjects in several of the studies. In each case where such
ratings were requested, subjects rated speeches including evidence much
higher on "evidence usage" than speeches not including evidence.
EVIDENCE, DELIVERY, AND MEDIA OF MESSAGE PRESENTATION
In two experiments results relating to the effects of evidence on attitude change and perceived credibility were conflicting across topics when the message was attributed to a moderate-to-low-credible source.23 In both studies evidence had its predicted effect when the speech topic was federal control of education, but in neither study did evidence have its predicted effect when the speech topic was capital punishment. Post-experiment interviews with selected subjects indicated that they perceived the delivery of the speaker on federal control of education to be very good but the delivery of the speaker on capital punishment to be dull and monotonous. Therefore the quality of presentation of a message was suspected as a variable that could interact with evidence usage and, in turn, could produce the conflicting findings across topics.
Delivery long has ben thought to be a significant variable in oral communication. Poor delivery theoretically distracts from the content of the message by drawing attention of the audience to poor delivery characteristics and by reducing the clarity of the verbal message. Because poor delivery might cause an audience to miss evidence at it is presented by causing them to attend to something else and because it might prevent the audience from clearly understanding evidence which they do hear, poor delivery could interact with evidence usage in persuasive communication.
Two studies were designed and conducted to test the following hypothesis: Inclusion of evidence in a persuasive speech increases attitude change and perceived credibility when the speech is well delivered but has no effect on either attitude change or credibility when the speech is poorly delivered. The results of both studies provided partial support for this hypothesis.
Arnold and McCroskey employed a live speaker to present an "evidence speech" and a "no-evidence speech" under conditions of both good delivery and very poor delivery.24 Greater attitude change and higher perceived credibility were produced by the condition including evidence and good delivery than any of the other three conditions. The other three conditions did not differ in amount of attitude change produced. However, even in the poor delivery condition, the speech including evidence produced higher perceived credibility than the speech with no evidence, though not as high as in the condition including evidence and good delivery.
In a subsequent and much broader study, the results were consistent with those of the Arnold and McCroskey study. In this study for main variables were manipulated--evidence, delivery, initial credibility, and media of transmission.25 The results indicated that inclusion of evidence increased immediate attitude change only under conditions of good delivery accompanied by initial low credibility. The results were consistent across transmission media--audio-tape and video-tape. Although poor delivery was found significantly to reduce perceived credibility, again, as in the Arnold and McCroskey study, the speech including evidence consistently produced higher perceived credibility than the speech not including evidence.
The results of these studies, taken together, support the conclusion
that poor delivery can inhibit the effect of evidence on immediate attitude
change but does to inhibit its effect on credibility. Because the results
of these studies were consistent for live, audio-taped, and video-taped
speakers, one has no reason to believe that media of presentation is related
to the effect of evidence in persuasive communication.
EVIDENCE AND PRIOR KNOWLEDGE OF AUDIENCE
Although the findings discussed in the previous indicate that evidence and delivery are related in the production of attitude change, this does not provide a full explanation of the conflicting results across topics in the two studies previously noted. In those studies the results on credibility also conflicted. As a result of post-experiment interviews with subjects involved in the studies in which conflicting results were obtained, another variable was suspected to be related to evidence in the production of attitude change and the perceived source credibility. This is the variable of prior familiarity of the audience with evidence cited by a speaker.
In the post-experiment interviews the almost universal reaction from the subjects was that the evidence cited in the capital punishment speech was "old hat." On the federal control of education topic, on the other hand, the most frequent comment was one of interest and surprise at what was described by several subjects as the "shocking facts" presented.
If evidence must be "new" to have an effect, the inconsistent results become explainable. Such an assumption is highly consistent with some information theories. It is also consistent with dissonance theory. Old evidence has already entered the cognitive domain of the subject. If it created dissonance, that dissonance would have already been resolved or defense mechanisms constructed to avoid the recurrence of dissonance as a result of that evidence. Thus the presentation of that evidence to the subject would have no effect.
On the basis of this theory it was hypothesized that presenting evidence to people who previously have been exposed to that evidence will have no effect on either attitude change or credibility, but presenting the same evidence to people who are not familiar with it will significantly increase attitude change and credibility if the source of the message is initially moderate-to-low-credible.
The study designed to test this hypothesis involved the manipulation of three variables--evidence, initial credibility, and degree of prior familiarity with the evidence on the part of the subjects.26 Evidence and credibility were manipulated as in previous studies. The speeches on the topic of federal control of education were selected as experimental stimuli because it was assumed that the subjects were not familiar with the evidence included in these speeches. Half of the subjects, therefore, were systematically exposed to this evidence under appropriate cover conditions prior to being exposed to the experimental speeches.
The results of this study were precisely predicted by the hypothesis.
Evidence increased attitude change and credibility only in the condition
including low initial credibility and no prior familiarization with the
evidence. These results seem to provide an explanation of the conflicting
results of the earlier studies. But more importantly, the results indicate
that evidence must be "new" to the audience before it can have
an impact on their immediate attitude change or their perception of the
message source.
THE EFFECTS OF EVIDENCE ON SUSTAINED ATTITUDE CHANGE
The previous sections of this paper have been concerned only with the effects of evidence on credibility and attitude change measured immediately subsequent to exposure to persuasive messages. Evidence has an impact on these important variables only under relatively limited circumstances. In several of these studies the effects of evidence on sustained attitude change for periods up to seven weeks were measured.27 In four of the five cases in which the effect of evidence on sustained attitude change was measured, inclusion of evidence was found significantly to increase the amount of attitude change retained over time. In the fifth case the difference was in the same direction although not statistically significant. Further, no interactions between evidence usage and other communication variables were found relating to sustained attitude change.
Precisely why evidence has an effect on sustained attitude change even
when it has no effect on immediate attitude change is not at all clear.
Since all speeches produced significant immediate attitude change, one
possible explanation is that evidence may interfere with the process of
selective recall. The evidence included in the experimental messages in
the studies that measured sustained effect was, for the most part, quite
vivid and memorable. Such material may have been more memorable than other
elements of the messages. Whether less striking evidence would have a similar
impact is unknown.
CONCLUSIONS
Some twenty-two studies concerned with the functioning of evidence in
persuasive communication have now been reported in the literature. The
purpose of this paper has been to examine these studies to determine what
generalizations of value to the practicing communicator or communication
researcher tentatively may be drawn at this point in time. The following
are generalizations which seem appropriate:
1. Including good evidence has little, if any, impact on immediate audience attitude change or source credibility if the source of the message is initially perceived to be high-credible.
2. Including good evidence has little, if any, impact on immediate audience attitude change if the message is delivered poorly.
3. Including good evidence has little, if any, impact on immediate audience attitude change or source credibility if the audience is familiar with the evidence prior to exposure to the source's message.
4. Including good evidence may significantly increase immediate audience attitude change and source credibility when the source is initially perceived to be moderate-to-low-credible, when the message is well delivered, and when the audience has little or no prior familiarity with the evidence included or similar evidence.
5. Including good evidence may significantly increase sustained audience attitude change regardless of the source's initial credibility, the quality of the delivery of the message, or the medium by which the message is transmitted.
6. The medium of transmission of a message has little, if any, effect
on the functioning of evidence in persuasive communication.
NEED FOR FUTURE RESEARCH
Although the number of studies concerning the functioning of evidence in persuasive communication has increased sharply in recent years and several studies not discussed in this paper are in progress, there is a major need for more, and more imaginative, research in this area. Only a few very tentative generalizations about the place of evidence in persuasive communication are available. What are some of the questions that remain to be answered? The following are some that seem worthy of consideration: (1) What is the effect of evidence on overt behavior change? All the studies to date have been concerned with attitude. (2) Can evidence from non-credible sources serve as well as evidence from credible sources? Dresser's results suggest they can, but in his study credibility of the speaker was not manipulated.28 (3) What type of evidence (opinion, statistics, example) produces the most favorable impact? Some researchers have investigated this area, but too many uncontrolled factors were in their designs to make interpretation of the results possible. (4) Are there factors that interact with evidence in producing sustained attitude change? The studies to date have found none, but they have been very limited in scope. (5) What factors other than delivery, source credibility, and prior knowledge interact with or inhibit the effects of evidence? Some that might are structure of the message, intelligence of the audience, and salience of the topic. (6) Do non-students respond to evidence the same way as students? Most of the subjects in the studies to date have been college students. Have we merely contributed one more "tidbit" to the rhetoric of the college sophomore? (7) Does evidence function the same way in interpersonal or group communication as it does in person-to-group communication? This area has received no previous attention. (8) Does evidence function in the written media the same way it does with live, audio-taped, and video-taped speakers? There is no previous research here either. (9) Can evidence enhance a communicator's efforts to inoculate his audience against counterpersuasion? The consistent effects favoring evidence on sustaining attitude change suggest that it might. (10) Does evidence function the same way in various cultures? The response may be very different if one studies evidence in something other than the U.S. middle-class culture in which the previous studies have been conducted.
The above questions are included in this paper to make clear that, while
considerable information has accumulated upon which generalizations may
be formed about the place of evidence in persuasive communication the surface
of this problem area has barely been scratched. If we as communication
researchers are to continue to focus our attention on message variables
within the communication process, evidence should continue to be one of
the major variables we study.
NOTES
1. Richard E. Gregg, "Some Hypotheses for the Study of the Psychology of Evidence," paper read at the 1964 SAA Convention.
2. An extensive report of these studies is available in James C. McCroskey, Studies of the Effects of Evidence in Persuasive Communication, Report SCRL, 4-67, Speech Communication Research Laboratory, Michigan State University, 1967. Copies of this report are available from the writer upon request.
3. Robert S. Cathcart, "An Experimental Study of the Relative Effectiveness of Selected Means of Handling Evidence in Speeches of Advocacy" (unpublished Ph.D. dissertation, Northwestern University, 1953). See also "An Experimental Study of the Relative Effectiveness of Four Methods of Presenting Evidence," Speech Monographs, XXII (August, 1955), 227-233.
4. Erwin P. Bettinghaus, Jr., "The Relative Effect of the Use of Testimony in a Persuasive Speech upon the Attitudes of Listeners" (unpublished M.A. thesis, Bradley University, 1953).
5. Howard Gilkinson, Stanley F. Paulson, and Donald E. Sikkink, "Effects of Order and Authority in an Argumentative Speech," QJS, XL (April, 1954), 183-192.
6. Terry H. Ostermeier, "An Experimental Study on the Type and Frequency of Reference as Used by an Unfamiliar Source in a Message and Its Effect upon Perceived Credibility and Attitude Change" (unpublished Ph.D. dissertation, Michigan State university, 1966).
7. Delmar C. Anderson, "The Effect of Various Uses of Authoritative Testimony in Persuasive Speaking" (unpublished M.A. thesis, Ohio State University, 1958).
8. Dan L. Costley, "An Experimental Study of the Effectiveness of Quantitative Evidence in Speeches of Advocacy" (unpublished M.A. thesis, University of Oklahoma, 1958).
9. William R. Dresser, "Studies of the Effects of Satisfactory and Unsatisfactory Evidence in a Speech of Advocacy" (unpublished Ph.D. dissertation, Northwestern University, 1962). See also "Studies of the Effects of Evidence: Implications for Forensics," AFA Register, X, No. 3 (1962), 14-19; "Effects of 'Satisfactory' and 'Unsatisfactory' Evidence in a Speech of Advocacy," Speech Monographs, XX (August, 1963), 302-306; and "The Impact of Evidence on Decision Making," paper read at the 1965 SAA Convention.
10. James C. Gardner, "An Experimental Study of the Use of Selected Forms of Evidence in Effecting Attitude Change" (unpublished M.A. thesis, University of Nebraska, 1966).
11. Gerard A. Wagner, "An Experimental Study of the Relative Effectiveness of Varying Amounts of Evidence in a Persuasive Communication" (unpublished M.A. thesis, Mississippi Southern University, 1958).
12. See note 6.
13. James C. McCroskey, "Scales for the Measurement of Ethos," Speech Monographs, XXXIII (March, 1966), 65-72.
14. James C. McCroskey, "Experimental Studies of the Effects of Ethos and Evidence in Persuasive Communication" (unpublished D.Ed. dissertation, Pennsylvania State University, 1966).
15. Ibid. See also "The Effects of Evidence in Persuasive Communication," Western Speech, XXXI (Summer, 1967), 189-199.
16. McCroskey, Studies of the Effects . . .
17. James C. McCroskey and Robert E. Dunham, "Ethos: A Confounding Element in Communication Research," Speech Monographs, XXXIII (November, 1966), 456-463.
18. Paul D. Holtzman, "Confirmation of Ethos as a Confounding Element in Communication Research," Speech Monographs, XXXIII (November, 1966), 464-466.
19. McCroskey, Studies of the Effects . . .
20. Ibid.
21. William E. Arnold and James C. McCroskey, "The Credibility of Reluctant Testimony," Central States Speech Journal, XVIII (May, 1967), 97-103.
22. McCroskey, Studies of the Effects . . .
23. Ibid.
24. Ibid.
25. Ibid.
26. Ibid.
27. Ibid.
28. Dresser, "Studies of the Effects of Satisfactory . . . "