CompPanels: Images from the Annals of Composition #15

History of a Model of Writing Evaluation

Leo P. Ruth and Sandra Murphy's model of "the writing assessment episode" was published in College Composition and Communication in 1984 (vol. 35, p. 414), and later in their Ablex monograph Designing Writing Tasks for the Assessment of Writing (1988, p. 128). It is still one of the most useful constructivist takes on the complex interactions that occur when exam-designers, examinees, exam-raters, prompts, essays, and rubrics all collide. You might call it a classic post-structuralist deconstruction of that structuralist apparatus we call holistic evaluation.

But the diagram I reproduce here is not the finished version of Ruth and Murphy's model. It is an earlier avatar of it. It appears on p. 8 of a National Institute of Education research grant final report, called Properties of Writing Tasks: A Study of Alternative Procedures for Holistic Writing Assessment, dated November 1982 and still available in the ERIC Document Reproduction Service database (ED 230 576). A footnote attached to this NIE-grant model provides some history. The "basic categories" of the model were first proposed by Leo Ruth, who was the project director, at a staff meeting on October 22, 1980. Later that academic year, "many refinements" were made during "weekly conceptual brainstormings meetings" by the project's research coordinators and associates, as well as by "other visitors." Named contributors to the model, along with Sandra Murphy, are Catherine Keech, Paul Ammon, Judith Langer, Donald Leu, Charles Kinzer, and Karen Carroll.

I will resist the temptation to detail changes that the model underwent from 1980/1981 multi-authored grant-report form to the 1984 bi-authored Ablex monograph form. They are fascinating. (Compare just the box labeled "The Test-Rater," where the "Underlying Factors" became "Influences on Process").

But I will put a plug in for the 1982 NIE report, Properties of Writing Tasks. Though some of the material got into Ruth and Murphy's 1988 Ablex volume, much of it remains unpublished, except in ERIC microcard. Yet it contains ground-breaking studies (indexed in CompPile) by Catharine Keech [Lucas], Charles Kinzer, Don Leu, Mary Ellen McNelly, Gerald Camp, Karen Carroll, and Ann Robyn—investigations delving into test-maker intentions, rater biases, and student reactions to holistic prompts. In the history of writing assessment, which from 1980 on might be called "beyond the holistic," Properties of Writing Tasks is a largely unrecognized landmark.

RH, December 2003