Judging Quality & The Survival of CX (Part 1)

This weekend gave me another chance to go into the lab of “judging quality,” where I’ve continued to examine the influence of the state of judging in policy debate and its impact on the growth or decline of the activity. Judging seven rounds of varsity policy at Iowa’s IHSSA state this weekend and visiting with some of our coaches who have a policy background, I had the chance to think more about a thought that has continued to bother me, having returned to policy as a coach after many years of absence.

After some rather interesting squirreling at previous tournaments by some fellow judges (not me, thankfully) in elimination rounds this year, and from the review of some ballots reflecting a wide range of views on what policy debate is, I’ve been considering the conjecture that the decline policy debate may be partially attributed to the inconsistency in paradigms practiced, communicated and judged in the round. Applying the perspective from the risk management world, I believe we have indications of substantial error or “noise” in the debate round process which is leading to an inconsistent and lower quality experience, and furthermore, I conjecture that this error is contributing to the decline in the sport.

Consider this: If policy debate is supposed to be an educational experience (as many frameworks and abuse arguments will claim), how is it educational when the same input yields highly inconsistent outputs? Is it educational if a student wins one round and loses another on the same level of quality against the same opponent in the same debate (three judges in eliminations, for instance)? Is it educational, only in communicating the inherent subjectivity and unfairness of the world, and if so, do debaters need to invest such great amount of times to learn this lesson when it may be available elsewhere for less cost? If the same “performance” (not that I’m advocating this specific judging paradigm) by two teams can yield very different results on a consistent basis by judges would certainly have to have a detrimental debate in the recruiting and retention of students interested in the activity. Very few would want to continue an already difficult sport that requires extensive hours invested, practice, camps and weekend after weekend in rounds consuming both Friday and Saturday nights. Remember, these are teenagers we’re talking about. So when that same activity gives a participant a loss when another judge would have declared it a win, it seems probable that this inconsistency would encourage students to find other things to do with their time.

In the conjecture, I should note that I’ve made a clear distinction between paradigms “practiced,” “communicated” and “judged.” If the conjecture is accurate that there is error, I believe it would have to be in at least one of these three areas. Specifically:

  • Paradigms Practiced: This would be the policy-debate paradigm that is practiced by each team in the round. To assume error here would be to suggest that one or both teams came into the round with the wrong model of what policy debate is or isn’t. The better quality, more capable team lost, leading to an error in the determination of the better debate team, because of the wrong paradigm was used by one or more of the teams. This assumes there is more than absolute judge subjectivity in the round, which is necessary to have if you are to believe that a debate team has any influence on the outcome of the round, and also assumes that Quality is external to the paradigm practiced by the respective teams and is lurking out there in space for a competent judge to read, record and judge. Because of this assumption, I believe we’ll have to be careful about how far we go down this path given that we may end up in endless arguments in metaphysical philosophy which probably won’t be resolved in this blog any time soon. Still, we have to examine this area as it gives the debaters a specific lens into how they encountered the educational experience. A debater that views the round incorrectly will be surprised at the decision at the end of the round, which he or she would view as incorrect from this perspective.
  • Paradigms Communicated: This would suggest that error in the round led to a less capable winning because something interfered with the judges paradigm being communicated to the debaters, causing at least one of the teams to debate on the wrong ground. I’ve seen this communication error first hand in out-rounds, co-judging with others who declare themselves to be “a non-interventionist tab judge” only to listen to their rendition of the ballot using a highly projected policy-maker framework. This assumes that judges paradigms are relevant in determining the quality of the round, bringing that subjective Quality into the mix but also making all actors an important part of this determination. In the experience of a risk manager, one of the first places to look for large error is where multiple perspectives clash. This could be an important part of the puzzle.
  • Paradigms Judged: This would suggest an error in the judging process, such as the judge missing arguments that should be regarded as critical and influential to a different outcome, or the judges application of “wrong thoughts” or thoughts inconsistent with what the judge explained was his or her paradigm in the determination of the round. The better team won in the Objective Universe, but the judge didn’t recognize it somehow and gave the ballot to the wrong team. This is often the viewpoint expressed by the losing team in a round, e.g. “We won the round but the stupid judge didn’t get it. How could he value Solvency over our Fairness Framework?” While I believe judges to be far from infallible, I’m inclined to borrow from the world of process quality and Six-Sigma which tends to find a small percentage associated with people error (often less than 4%) and a large percentage attributed to process error (often more than 96%). Still, we do need to examine this one closely as the debaters tend to believe this is where error round originates.

    Looking through this situation from the perspective of an risk manager trying to figure out where the error is coming from, I believe we owe each area some attention. Because the concept of a judging paradigm is the subject of our search for error, and the concept itself is the topic of many differences and disagreements, part II will have to examine this creature in further detail to make sure we can continue with this examination of round error.

    A final note on my conjecture: I’d be remiss if I didn’t acknowledge that there certainly are other factors (or “coefficients” for the statistically inclined) contributing to the decline of policy. One would have to certainly question if the expense of running a policy program is a factor. Policy is not something you dabble in and continue. It requires a full commitment from the school, coach, team and parents. “Timesuck” (to borrow the in-round policy concept) from other activities also has to be a factor. Debate alternatives like Public Forum and Congress certainly could have an erosive effect, taking a portion of the quality policy-capable debaters away from the field. These all are items I’ve seen referenced in other discussions and should continue to be examined external to this exploration on the Quality of what occurs within an individual debate round and its contribution to the development or decline of the sport.

    (Continue on to Part 2 – Policy Debate Judging Frameworks)

  • Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s