Since 1915 coaches have lamented the alleged lack of agreement on debate judging criteria.¹ While some individuals might feel that the "Wells-O'Neill controversy" generated considerably more heat than light, certainly they must be accorded credit for initiating an ongoing tradition of questioning and examination. In 1946 Musgrave renewed the fray with a series of articles involving debate judging which reviewed the "Wells-O'Neill" articles.² Apparently, others in the profession felt the issues involved in the discrepancy between theory and practice were not settled. In 1953 Kline reported research concerning delivery and its relation to other criteria on a rating scale,³ and in 1954 Scott reported research indicating that judges' bias on the debate question did not affect their judging.⁴ The 1959 and 1960 studies by Roever and Giffin provided more information about judging criteria.⁵

After this array of material one might question the need for further research. However, studies of the relative importance of the stock issues in judges decisions,⁶ the ability of debaters to judge themselves,⁷ the criteria used in debate judging,⁸ and the effect of judges' bias on debate decisions,⁹ which have been published indicate various--and apparently conflicting--conclusions. Research reported by Laase and Baccus, for example, indicated that debaters' judgments correspond closely with the critics' judgment on ratings and rankings.¹⁰ King and Clevenger, however, found just the opposite was true of decisions.¹¹

In a study conducted at the 1964 Southern Speech Association Tournament, the present writers investigated various factors relating to stock issues, judging criteria, and decisions in debate.¹²

Two of the conclusions concerning judging criteria which were drawn on the basis of this investigation were particularly tentative and somewhat controversial:

(1) "High school and college coaches are in agreement on the relative importance of criteria in arriving at decisions in debate," and (2) "Delivery should not be included as one of the items on debate ballots." The latter conclusion was based on results which were in conflict with a similar study conducted by Giffin in 1959.¹³ It was thought desirable to replicate this portion of the 1964 study to determine whether the attitudes of these particular judges were typical.

A study reported by Scott in 1954¹⁴ indicating that judges' bias had no effect on their decisions led the writers to include this question in the 1964 study. Because of the reluctance of over half of the judges to indicate their position on the topic, no firm conclusion could be drawn from the results. It was felt that a different procedure which would give the judge a greater feeling of anonymity would be needed to obtain reliable results.

The purpose of this study reported below was to replicate these two portions of the 1964 study.

The data for this study were obtained by a questionnaire submitted to judges at the annual Southern Speech Association Tournament in Durham, North Carolina, April 4-9, 1965.¹⁵ This tournament was selected for two reasons. It is one of the few tournaments in the nation that includes both high school and college divisions. This was considered to be very important because comparisons between high school and college coaches were crucial. In addition, this tournament draws teams from an unusually large area. Both high school and college teams from all over the south participate. The writers were concerned about the number of 1965 tournament judges who also completed forms in the 1965 tournament. A comparative survey of the names of judges revealed, however, that less than 10% of the judges who completed forms for the 1964 study were present in 1965.

Each judge was asked to complete a questionnaire once during the tournament. On this questionnaire he was asked to indicate his occupation, his personal opinion on both the high school and college topic, and to rank from one to seven various criteria for judging debate. Space was provided for adding criteria not listed on the questionnaire.

The judge was not requested to sign his name on the questionnaire. When they were returned they were identified and coded so that at the end of the tournament the judge's decisions could be determined form the tournament ballots and compared with his expressed position on the topic. All comparisons were made on the basis of this code, omitting the judges' names.

Table I reports the results of the study on judging criteria. The criteria used were the same as those originally reported by Giffin. Included are the results of the three divisions of the 1964 study, the two divisions from the 1965 study, the combined coach-rankings across both studies, and the results of the Giffin study converted into rank order. There were too few (four) non-coach judges in the 1965 study to constitute a reportable category.

*Only four judges were not debate coaches in the 1965 tournament.

Several observations based upon these data are noteworthy. The results of the 1964 and 1965 studies were nearly identical. The college coaches differed only in their ranking of evidence and organization. The high school coaches differed only in their ranking of case and analysis. Rankings for refutation, language, and delivery were identical for high school and college coaches in both studies. It should be noted that the rankings reported are based on the mean rank of each group, a statistically questionable procedure. However, in both studies the median and modal ranks are nearly identical. In no case in either study does the mean ranking procedure change the rank order of the criteria.

Table II reports the results of the study concerning judges' bias. The two divisions of the 1965 study are reported separately. The results of the 1964 high school and college divisions are combined. The problem of judges refusing to indicate their opinion on the topic was again a problem in the college division in 1965. Approximately one-third of the debates were judged by such individuals. This was not however, a serious problem in the high school division. Only thirteen debates were judged by individuals who refused to state their opinions on the topic.

*Does not include rounds forfeited.

While conclusions drawn from these two studies must be qualified to the extent that they involved only judges from the South, three conclusions seem to be in order. First, it is clearly indicated that these high school and college coaches are in agreement on the rank order of criteria for judging academic debate. It should be expected that decisions rendered by the two groups would not differ markedly.

Secondly, the suggestion that delivery ought to be omitted from debate ballots, which was tentatively offered by the writers on the basis of the 1964 study, again seems appropriate. As in the 1964 study, the median and modal rank for this criterion was seven. If we wish to continue to include only five criteria on debate ballots, it would seem reasonable to omit delivery since it was not considered by judges as one of the five most important criteria.

This conclusion, of course, is dependent on the interpretation of why the coaches studied chose to place delivery last in order of importance. One interpretation may be that the coaches do not consider delivery important in effective debating. If this is the case, we might conclude that delivery should receive more stress in our debate programs. As most coaches and judges of debate have noticed, ineffective delivery characteristics such as over-projection and excessively rapid rate are not uncommon. Further study of the frequency of poor delivery practices and judges' and coaches' attitudes toward such practices is needed.

Another possible interpretation is that delivery was judged by the coaches studied to be an element inseparable from the other criteria used in judging debate. As indicated above, the other criteria are directly affected by the quality of delivery. Delivery mannerisms can and do distort the perception of the quality of analysis, refutation, etc. Some believe that to consider delivery as a separate criteria is to judge delivery twice. If this is the case, and the writers believe that it is, publishers of debate ballots and tournament directors printing their own ballots should be encouraged to consider removing delivery from the usual list of five criteria.

Finally, Scott's conclusion that judges' bias on the topic has no effect on their decisions was supported. It is apparent from the results reported in Table II that no significant topic bias factor was operating in these tournaments. This is encouraging, for while an individual judge may allow his bias toward the proposition to influence his decision, over a period of time debaters can feel secure that decisions will be based on the quality of their debating--not on what the judge thinks about the topic.

1. See the series of articles by O'Neill, Wells, et al. in the Quarterly Journal of Speech, 1915-1918. For an annotated bibliography of articles on debate judging, see James C. McCroskey, "Fifty Articles in Fifty Years: A Selected, Annotated Bibliography on Debate Judging, 1915-1964," Journal of the American Forensic Association, I (May, 1964), 67-69.

2. George M. Musgrave, "The Wells-O'Neill Controversy," Debater's Magazine, II (December, 1946), 218.

3. Donald F. Kline, "Effect of Debate Skills on Delivery," Gavel, XXXV (March, 1953), 59-60.

4. Robert L. Scott, "The Objectivity of Debate Judges," Gavel, XXXVII (November, 1954), 14-15.

5. Kim Giffin, "A Study of the Criteria Employed by Tournament Debate Judges," Speech Monographs, XXVI (March, 1959), 69-71 and James Roever and Kim Giffin, "A Study of the Use of Judging Criteria in Tournament Debate," AFA Register, VIII (1960), 12-14.

6. Kim Giffin and Kenneth Megill, "Stock Issues in Tournament Debates," Central States Speech Journal, XII (Autumn, 1960), 27-32; Kim Giffin and Kenneth Megill, "A Study of the Use of Key issues in Tournament Debates," Gavel, XLIII (November, 1960), 67-68; and James C. McCroskey and Leon R. Camp, "A Study of Stock Issues, Judging Criteria, and Decisions in Debate," Southern Speech Journal, XXX (Winter, 1964), 154-168.

7. Leroy T. Laase, "An Evaluation of the quality Rating System in Measuring Debate Achievement," Quarterly Journal of Speech, XXVIII (December, 1942), 424-430; Joseph Baccus, "Debaters Judge Each Other," Quarterly Journal of Speech, XXIII (February, 1937), 74-80; McCroskey and Camp, loc. cit.; and Donald Klopf, Diana Evans, and Sister Mary Linus De Lozier, "Comparative Studies of Students, Laymen and Faculty Members as Judges of Speech Contests," Speech Teacher, XIV (November, 1965).

8. Giffin, loc. cit.; Roever and Giffin, loc. cit.; and McCroskey and Camp, loc. cit.

9. Robert L. Scott, "The Objectivity of Debate Judges," Gavel, XXXVII (November, 1954), 14-15 and McCroskey and Camp, loc. cit.

10. Laase, loc. cit. and Baccus, loc. cit.

11. Thomas King and Theodore Clevenger, Jr., "A Comparison of Debate Results Obtained by Participant and Critic Judging," Southern Speech Journal, XXV (Spring, 1960), 223-232, and McCroskey and Camp, loc. cit.

12. McCroskey and Camp, loc. cit.

13. Giffin, loc. cit.

14. Scott, loc. cit.

15. The writers would like to express their appreciation to Dr. Rex Kyker and Professor Edward Brown, directors of the college and high school debate divisions, for their assistance in securing the data for this study.

Click Here To Go Back To PERIODICALS